RESEARCH ARTICLE
Using HIV Surveillance Data to Monitor Missed Opportunities for Linkage and Engagement in HIV Medical Care
Jeanne Bertolli*, 1, R Luke Shouse1, Linda Beer1, Eduardo Valverde1, Jennifer Fagan1, Samuel M Jenness2, Afework Wogayehu 3, Christopher Johnson 1, Alan Neaigus 2, Daniel Hillman 4, Maria Courogen 4, Kathleen A Brady 6, Barbara Bolden 3Author Comment: for the Never In Care Project
Article Information
Identifiers and Pagination:
Year: 2012Volume: 6
Issue: Suppl 1
First Page: 131
Last Page: 141
Publisher ID: TOAIDJ-6-131
DOI: 10.2174/1874613601206010131
Article History:
Received Date: 14/4/2011Revision Received Date: 14/9/2011
Acceptance Date: 17/10/2011
Electronic publication date: 7/9/2012
Collection year: 2012
open-access license: This is an open access article licensed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted, non-commercial use, distribution and reproduction in any medium, provided the work is properly cited.
Abstract
Monitoring delayed entry to HIV medical care is needed because it signifies that opportunities to prevent HIV transmission and mitigate disease progression have been missed. A central question for population-level monitoring is whether to consider a person linked to care after receipt of one CD4 or VL test. Using HIV surveillance data, we explored two definitions for estimating the number of HIV-diagnosed persons not linked to HIV medical care. We used receipt of at least one CD4 or VL test (definition 1) and two or more CD4 or VL tests (definition 2) to define linkage to care within 12 months and within 42 months of HIV diagnosis. In five jurisdictions, persons diagnosed from 12/2006-12/2008 who had not died or moved away and who had zero, or less than two reported CD4 or VL tests by 7/31/2010 were considered not linked to care under definitions 1 and 2, respectively. Among 13,600 persons followed up for 19-42 months; 1,732 (13%) had no reported CD4 or VL tests; 2,332 persons (17%) had only one CD4 or VL test and 9,536 persons (70%) had two or more CD4 or VL tests. To summarize, after more than 19 months, 30% of persons diagnosed with HIV had less than two CD4 or VL tests; more than half of them were considered to have entered care if entering care is defined as having one CD4 or VL test. Defining linkage to care as a single CD4 or VL may overestimate entry into care, particularly for certain subgroups.
INTRODUCTION
Linkage to HIV medical care ideally serves as a catalyst for maximizing the prevention potential of strategies to expand HIV testing [1-9], particularly when it results in the early initiation of antiretroviral therapy [10-12]. For this reason, MPDF the National HIV/AIDS Strategy includes an objective of linking at least 85% of individuals newly diagnosed with HIV into care within three months of diagnosis [13]. Linkage to care also benefits persons living with HIV [14-17] and improved linkage has the potential to contribute to the reduction of disparities in morbidity and mortality (another key goal of the National HIV/AIDS Strategy) [13].
Evaluating progress towards the National HIV/AIDS Strategy linkage to care objective requires population-level monitoring of receipt of HIV care. In particular, monitoring delayed entry to HIV medical care is needed because it signifies that opportunities to prevent onward transmission of HIV infection and to mitigate the progression of HIV disease have been missed. Ongoing monitoring of receipt of HIV care will also be needed to assess the impact of changes required by the 2010 Patient Protection and Affordable Care Act, including the expansion of Medicaid coverage, which are intended to increase access to care [18].
Since 2000, amendments to the Ryan White CARE Act have required Title I and Title II grantees and planning bodies to determine numbers of persons in their service areas who know they are HIV positive but are not receiving regular HIV-related primary medical care [19, 20]. For population-level monitoring of receipt of HIV care, the Council of State and Territorial Epidemiologists has encouraged states to require reporting of CD4 T-lymphocyte (CD4) and HIV viral load (VL) tests [21]. These tests, which are ordered by HIV care providers on a recommended schedule (starting with the patient’s first visit) to assess stage of disease [11], are considered a proxy for receipt of HIV medical care. As more U.S. states have moved toward requiring reporting of all CD4 and HIV VL tests, regardless of the magnitude of the result, the capacity to use reported CD4 or VL tests to monitor receipt of HIV care has increased. By December 2009, 38 of 50 states and the District of Columbia required reporting of either all CD4 cell counts or all HIV viral loads; the remaining states either did not require reporting of both CD4 and VL or required reporting of CD4 and VL restricted to a specific range of values [22]. Although state CD4 or VL reporting laws have become increasingly standardized, the use of these data to describe linkage to care has varied [19, 20, 23-27] in part because there is no standard definition of linkage to care for surveillance purposes.
A central question is whether to consider a person linked to care after receipt of one CD4 or VL test. Unlike kept appointments, which are used by clinicians to assess retention in care [23], the meaning of a single CD4 or VL test may be equivocal, although estimates of linkage to care using surveillance data have tended to measure linkage by the receipt of a single test [24, 25, 27]. However, some investigators have noted that one test result may not be specific enough to indicate care entry, particularly if these tests are commonly ordered before referral to care [26]; and others have distinguished between intermittent care (receipt of one CD4 or VL test during a 12-month period) and regular care (more than one CD4 or VL test during a 12-month period) [27].
In this report, we explore patterns of reported CD4 or VL tests in five health jurisdictions participating in the Never in Care Project, a supplemental surveillance pilot focused on persons diagnosed with HIV infection who have not received HIV medical care [28]. We also describe the implications of using one CD4 or VL test vs two or more tests as definitions of linkage to care for 1) estimating the size of the population yet to be linked to care and 2) for qualitative descriptions of this population.
METHODS
Data Source
To identify cases for the Never in Care Project, data were drawn from the HIV Surveillance Systems in five jurisdictions: Indiana, New Jersey, New York City, Philadelphia, and Washington State. Data describing adults (≥ 18 years old) with an HIV diagnosis from December 1, 2006-December 31, 2008, reported to the health departments in these jurisdictions through July 31, 2010, including demographic and risk behavior information, were used for all analyses. Laboratory data, specifically CD4 and VL tests, were used to describe care patterns.
Reporting of either all CD4 or all VL values was required by state law in four of the five jurisdictions during the observation period. Since 2005, two jurisdictions have had mandatory reporting of all CD4 and all VL values (Indiana, New York City) and two others have had mandatory reporting of all VL values and CD4 values below a threshold (Philadelphia, CD4 ≤350 cells/mm3 and <25% of total lymphocytes and New Jersey, CD4 <200 cells/mm3). Washington has had reporting of all CD4 and all VL values since 2006.
Analysis
HIV case surveillance data and patterns of associated CD4 and VL tests reported were examined for 1) the entire observation period (follow-up ranged from 19-42 months, depending on diagnosis date), and 2) the period extending 12 months from diagnosis. To minimize the effects of delays in reporting CD4 or VL tests, which ranged from 3-120 days for CD4 tests and 2-60 days for VL tests, we defined the observation period as beginning with the earliest diagnosis date (November 2006) and ending in June 2009, and used CD4 or VL tests performed during this period and reported through July 2010.
Persons who relocated to another jurisdiction were identified through the Routine Interstate Duplicate Review, an HIV surveillance activity performed at least biannually [29]. Those confirmed to have relocated outside the jurisdiction were excluded from analysis. Deaths among those with reported HIV diagnoses are routinely ascertained by matching HIV surveillance records with vital statistics data. The frequency of data matches to ascertain deaths ranges from quarterly to annually in the five jurisdictions. Death dates obtained from these matches were used as “censored by” dates in the Kaplan-Meier analysis described below.
Whether a single CD4 or VL is sufficient evidence of care depends on how frequently it represents a one-time event (as it would be if the first CD4 or VL test does not represent a care visit but rather a test ordered before referral to care, or if a large percentage of people fail to establish care after their first visit). On the other hand, if most of those with a first CD4 or VL test establish care and receive additional tests, a single test might be considered an adequate indicator of linkage to care. To explore whether a single CD4 or VL (either a single CD4, a single VL, or a CD4 and VL on the same date) should be taken to signify entry to care (and thus to exclude people from the “never in care” population), we examined percentages of HIV-infected persons with zero, only one, and two or more reported CD4 or VL tests during the entire observation period. We also compared distributions by race/ethnicity, age, gender, risk group and jurisdiction among those with zero, only one, and two or more reported CD4 or VL tests during the first 12 months after HIV diagnosis (a period considered critical for maximizing the survival benefits of HIV care) [30], and compared first CD4 and VL values among those with only one and two or more reported CD4 or VL tests. For these analyses, all those who died during the relevant period (12 months following diagnosis or before June 2009) were excluded. Differences in proportions were evaluated using chi-square tests.
The difference between using at least one CD4 or VL test vs two or more tests as the operational definition of linkage to care lies in the inclusion or exclusion of persons with only one CD4 or VL test. We explored the implications of using at least one test vs two or more tests to define linkage to care using two multivariable log-binomial regression models. Specifically, we assessed whether persons with only one reported test performed in the first 12 months after diagnosis were qualitatively more like those without any reported CD4 or VL tests (model 1) or alternatively, more like those with two or more reported CD4 or VL tests during this period (model 2) with respect to race/ethnicity, age, gender, and risk group. Both models included all of these factors as well as jurisdiction, and modeled prevalence ratios. For these analyses, all deaths during the 12 months following diagnosis were excluded.
We acknowledge the potential for bias in identifying predictors of not linking to care as defined, given the different CD4 and VL reporting requirements across jurisdictions. Because CD4 counts tend to rise and VL levels tend to fall with antiretroviral treatment [11], and one might reasonably expect that treatment might start for some individuals before the second CD4 or VL test, reporting restricted to CD4 values below 200 or 350 cells/mm3 is likely to affect comparisons of only one reported test vs two or more, but not comparisons of zero reported tests vs only one. Therefore, we performed a sensitivity analysis, which consisted of comparing the results from log-binomial regression models comparing only one reported test vs two or more, first including and then excluding data from the two jurisdictions for which reporting of CD4 and VL tests is required only if the results fall within a specified numeric range.
To describe care entry delays according to the two definitions of linkage to care, we estimated the probability functions for not having entered care by time since diagnosis, under each definition, using the Kaplan-Meier life table method. Time to event was measured from the date of the HIV diagnosis documented in the surveillance system (or June 15 of the year of diagnosis when the month of the diagnosis date was missing) until the date of the first CD4 or VL test, for both those who had at least one reported test and for those who had two or more reported tests. Reported CD4 or VL tests with missing dates were not considered for the assessment of timing of care entry. Right censoring occurred upon death or July 31, 2010, whichever was earliest. Persons who had died but for whom a death date was unavailable and persons who had moved out of jurisdiction during the period of observation, but for whom the date of moving was unknown, were excluded from the analysis. Log rank tests for homogeneity were performed to assess differences in the probability functions across jurisdictions.
Subtraction of those known to have received services from those believed to need them is one method of estimating “unmet need,” for HIV medical care within a defined period (usually 12 months) [27]. We did not apply this method to estimate the number of persons living with HIV who have not entered care because our time period of interest was much longer than 12 months. CD4 and VL tests, our indicators of receipt of services, were not required by law until well after many of those living with HIV had been diagnosed. As a result, tallies of numbers of CD4/VL tests received would have been incomplete.
Instead, the size of the population of HIV-infected adults who had not entered HIV medical care in each participating jurisdiction was estimated (according to each of the two definitions of linkage to care) as follows. In each jurisdiction, the estimated probability of not having entered care at each month since diagnosis, from 0-42 months (method as described above) was multiplied by the size of the population living with HIV (not AIDS, because persons with a diagnosis of AIDS were presumed to be in care, at least under definition 1) for the corresponding amount of time since diagnosis (as of December 31, 2008). These products were summed. The probability of not having entered care for those living with HIV (not AIDS) for whom more than 42 months had elapsed since their HIV diagnosis was assumed to be the same as the probability of not having entered care for those 42 months from HIV diagnosis. This assumption is supported by both surveillance data and cohort study data indicating that many of those who do not enter care soon after diagnosis delay care entry until they meet the definition of AIDS [31, 32].
RESULTS
In the five jurisdictions combined, 16,063 persons were diagnosed with HIV infection from December 1, 2006-December 31, 2008. We excluded the following from analysis, regardless of their care status: persons who moved away from the jurisdiction (N=1,569); persons who had a reported CD4 or VL dated before their diagnosis date (N=312), indicating an incorrect diagnosis or CD4 or VL date, or a problem with considering the CD4 or VL date as the care entry date; and persons who had died but had a missing death date (N=24). Of the 14,158 persons remaining after these exclusions, 558 persons died during the observation period: 133 persons died before their first reported CD4 or VL, 255 died before their second reported CD4 or VL, and 170 died after their second reported CD4 or VL.
As shown in Fig. (1), in the five jurisdictions combined, 13,600 persons were not censored during the observation period: 1,732 persons (13%) had no reported CD4 or VL test during the observation period; 2,332 (17%) had only one reported CD4 or VL test and 9,536 (70%) had two or more reported CD4 or VL tests. Next we focused on the first 12 months after diagnosis. Of 476 deaths that occurred in the 12 months following diagnosis, 128 persons died before their first reported CD4 or VL, 234 died before their second reported CD4 or VL, and 114 died after their second reported CD4 or VL. Of 13,633 persons who survived the first 12 months after diagnosis, 2,089 (15%) had zero reported CD4 or VL tests during this period; 3,139 (23%) had only one reported CD4 or VL test, and 8,405 (62%) had two or more reported CD4 or VL tests (numbers and percentages by jurisdiction are presented in Table 1).
 |
Fig. (1). Analysis flow chart.*The following were excluded from analysis, regardless of their care status: persons who moved away from the jurisdiction (N=1,569), who had a CD4/VL date before their diagnosis date (N=311), and persons who had died (N=476 during first 12 months after diagnosis and N=582 during the entire observation period, including 24 persons who had a missing death date). |
 |
Fig. (2a). Probability of Having <1 CD4 or Viral Load Test) by Months Since HIV Diagnosis. Analysis includes all CD4 and viral load tests as evidence of care entry. Differences in probability functions were statistically significant (p<0.0001), according to a log-rank test of homogeneity. |
 |
Fig. (2b). Probability of Having <2 CD4 or Viral Load Tests by Months Since HIV Diagnosis. Analysis excludes a single CD4 or viral load test as evidence of care entry. Differences in probability functions were statistically significant (p<0.0001), according to a log-rank test of homogeneity. |
Numbers and Percentages* of Persons with Zero, Only One, and Two or More Reported CD4 or Viral Load Tests Performed within 12 Months of HIV Diagnosis, Among Adults Diagnosed from December 1, 2006 to December 31, 2008, in Five Jurisdictions Participating in the Never in Care Project
| Health Jurisdiction | Number of Reported CD4 or Viral Load Tests Performed Within 12 Months of Diagnosis† | |||
|---|---|---|---|---|
| Zero N (%) | One N (%) | Two or More N (%) | Total in Cohort N (%) | |
| Indiana | 211 (21) | 372 (37) | 421 (42) | 1,004 (7) |
| New Jersey | 422 (20) | 858 (40) | 880 (41) | 2,160 (16) |
| New York City | 924 (13) | 1,033 (14) | 5,179 (73) | 7,136 (52) |
| Philadelphia | 431 (20) | 708 (32) | 1,055 (48) | 2,194 (16) |
| Washington | 101 (9) | 168 (15) | 870 (76) | 1,139 (8) |
| Total | 2,089 (15) | 3,139 (23) | 8,405 (62) | 13,633 (100) |
* Percentages are of all adults diagnosed with HIV from December 1, 2006-December 31, 2008, and reported through July 31, 2010, in each area, excluding deaths.
† Since 2005, two jurisdictions have had mandatory reporting of all CD4 and all VL values (Indiana, New York City) and two others have had mandatory reporting of all VL values and CD4 values below a threshold (Philadelphia, CD4 ≤ 350 cells/mm3 and <25% of total lymphocytes and New Jersey, CD4 <200 cells/mm3). Washington has had reporting of all CD4 and all VL values since 2006; CD4 and viral load tests on the same date were counted as one test.
Percentages* of Persons with Zero, Only One, and Two or More Reported CD4 or Viral Load Tests Performed within 12 Months of HIV Diagnosis by Race/Ethnicity, Age, Sex, Location, and Risk Group, Among Adults Diagnosed from December 1, 2006, to December 31, 2008, in Five Jurisdictions Participating in the Never in Care Project
| Characteristic | Number of Reported CD4 or Viral Load Tests Performed Within 12 Months of HIV Diagnosis† | |||||
|---|---|---|---|---|---|---|
| Zero N=2,089 % | Only One N=3,139 % | Two or More N=8,405 % | p value, Zero vs Only One | p Value, Only One vs Two or More | Total N=13,633 % of Total | |
| Race/Ethnicity§ | <0.001 | <0.0001 | ||||
| Hispanic | 18 | 19 | 20 | 20 | ||
| American Indian/Alaska Native | 0.4 | 0.2 | 0.3 | 0.3 | ||
| Asian/Pacific Islander | 2 | 2 | 3 | 3 | ||
| Black, non-Hispanic | 60 | 53 | 47 | 49 | ||
| White, non-Hispanic | 16 | 23 | 24 | 24 | ||
| Multi-race | 4 | 3 | 5 | 4 | ||
| Age at Diagnosis (Years)§ | 0.003 | 0.627 | ||||
| <20 | 4 | 2 | 3 | 3 | ||
| 20-29 | 28 | 25 | 26 | 26 | ||
| 30-39 | 26 | 27 | 26 | 27 | ||
| 40-49 | 26 | 27 | 27 | 27 | ||
| 50+ | 16 | 18 | 18 | 18 | ||
| Gender | 0.484 | 0.640 | ||||
| Male | 73 | 73 | 73 | 73 | ||
| Female | 27 | 27 | 27 | 27 | ||
| Health Jurisdiction | <0.0001 | <0.001 | ||||
| Washington | 5 | 5 | 11 | 9 | ||
| New York City | 44 | 33 | 61 | 54 | ||
| New Jersey | 20 | 27 | 10 | 15 | ||
| Philadelphia | 21 | 23 | 13 | 15 | ||
| Indiana | 10 | 12 | 5 | 7 | ||
| Risk Category§ | 0.03 | <0.001 | ||||
| MSM | 30 | 31 | 40 | 38 | ||
| IDU | 7 | 9 | 6 | 6 | ||
| MSM & IDU | 5 | 6 | 4 | 5 | ||
| Heterosexual Contact | 37 | 35 | 42 | 41 | ||
| Undetermined | 20 | 18 | 8 | 11 | ||
| Other | 0.4 | 0.3 | 0.1 | 0.2 | ||
| First CD4 Result (Cells/mm3) | -- | <0.0001** | ||||
| <200 | -- | 50 | 43 | -- | ||
| 200-349 | -- | 14 | 17 | -- | ||
| 350-500 | -- | 14 | 18 | -- | ||
| >500 | -- | 22 | 22 | -- | ||
* Percentages are of all adults diagnosed with HIV from December 1, 2006 to December 31, 2008, and reported through July 31, 2010, in each area, excluding deaths.
† Since 2005, two jurisdictions have had mandatory reporting of all CD4 and all VL values (Indiana, New York City) and two others have had mandatory reporting of all VL values and CD4 values below a threshold (Philadelphia, CD4 ≤_350 cells/mm3 and <25% of total lymphocytes and New Jersey, CD4 <200 cells/mm3). Washington has had reporting of all CD4 and all VL values since 2006; CD4 and viral load tests on the same date were counted as one test.
§ Age, race/ethnicity, or risk was missing for some individuals; MSM: man who has sex with men; IDU: injection drug user
** p value for comparison of CD4 < 200 cells/mm3vs CD4 ≥ 200 cells/mm3.
Comparison of Predictors of Zero vs Only One Reported CD4/VL Test(s), and Only One vs Two or More Reported CD4/VL Tests(s) Performed within 12 Months of HIV Diagnosis, from Log-Binomial Regression Models, in Jurisdictions Participating in the Never In Care Project*
| Characteristic | Number of Reported CD4 or Viral Load Tests Performed Within 12 Months of Diagnosis† | |||
|---|---|---|---|---|
| Zero vs Only 1 CD4/VL Test(s)(N=5,228) | Only 1 CD4/VL vs 2 or More (N=11,544)* | |||
| Crude Prevalence Ratio (95% Confidence Interval) | Adjusted Prevalence Ratio (95% Confidence Interval) | Crude Prevalence Ratio (95% Confidence Interval) | Adjusted Prevalence Ratio (95% Confidence Interval) | |
| Race/Ethnicity§ | ||||
| Hispanic | 1.2 (1.1-1.4) | 1.2 (1.1-1.4) | 1.0 (0.9-1.1) | 1.0 (0.9-1.1) |
| American Indian/Alaska Native | 1.8 (1.2-2.9) | 1.8 (1.2-2.8) | 0.7 (0.3-1.4) | 0.8 (0.4-1.6) |
| Asian/Pacific Islander | 1.2 (0.9-1.6) | 1.2 (0.9-1.5) | 0.7 (0.6-0.9) | 0.9 (0.7-1.1) |
| Black, non-Hispanic | 1.4 (1.3-1.5) | 1.3 (1.2-1.5) | 1.1 (1.1-1.2) | 1.1 (1.0-1.1) |
| Multi-race | 1.6 (1.4-2.0) | 1.4 (1.2-1.7) | 0.6 (0.5-0.7) | 0.9 (0.7-1.1) |
| White, non-Hispanic | Ref | Ref | Ref | Ref |
| Age at Diagnosis § (Years) | ||||
| <20 | 1.4 (1.2-1.6) | 1.3 (1.1-1.5) | 1.0 (0.8-1.3) | 1.1 (0.9-1.3) |
| 20-29 | 1.1 (1.0-1.3) | 1.1 (1.0-1.2) | 1.0 (0.9-1.1) | 1.1 (1.1-1.3) |
| 30-39 | 1.0 (0.9-1.1) | 1.0 (0.9-1.1) | 1.1 (1.0-1.2) | 1.2 (1.1-1.3) |
| 40-49 | 1.1 (1.0-1.2) | 1.0 (0.9-1.2) | 1.0 (0.9-1.1) | 1.1 (1.0-1.2) |
| 50+ | Ref | Ref | Ref | Ref |
| Gender | ||||
| Male | 1.0 (0.9-1.1) | 1.0 (0.9-1.1) | 1.0 (0.9-1.0) | 1.1 (1.0-1.2) |
| Female | Ref | Ref | Ref | Ref |
| Health Jurisdiction | ||||
| Indiana | 1.0 (0.8-1.2) | 0.8 (0.7-1.0) | 2.9 (2.5-3.3) | 2.8 (2.4-3.3) |
| New Jersey | 0.9 (0.7-1.0) | 0.7(0.6-0.8) | 3.0 (2.6-3.5) | 2.9 (2.4-3.4) |
| New York City | 1.3 (1.1-1.5) | 1.1 (.9-1.3) | 1.0 (0.9-1.2) | 1.0 (0.9-1.2) |
| Philadelphia | 1.0 (0.9-1.2 | .9 (0.8-1.1) | 2.5 (2.2-2.9) | 2.3 (2.0-2.7) |
| Washington | Ref | Ref | Ref | Ref |
| Risk Category§ | ||||
| IDU | 0.9 (0.80-1.0) | 0.9 (0.8-1.1) | 1.7 (1.5-1.9) | 1.4 (1.3-1.6) |
| MSM & IDU | 0.8 (0.7-1.0) | 1.0 (0.8-1.2) | 1.7 (1.5-1.9) | 1.2 (1.1-1.4) |
| Heterosexual Contact | 1.0 (1.0-1.1) | 0.9 (0.9-1.0) | 1.1 (1.0-1.1) | 1.2 (1.1-1.3) |
| Other | 1.1 (0.7-1.9) | 1.3 (0.9-2.1) | 2.3 (1.5-3.4) | 1.5 (1.1-2.3) |
| Undetermined | 1.1 (1.0-1.2) | 1.3 (1.1-1.4) | 2.1 (1.9-2.3) | 1.2 (1.1-1.4) |
| MSM | Ref | Ref | Ref | Ref |
* Among adults diagnosed from December 1, 2006-December 31, 2008, excluding all deaths during the first 12 months after diagnosis.
† Since 2005, two jurisdictions have had mandatory reporting of all CD4 and all VL values (Indiana, New York City) and two others have had mandatory reporting of all VL values and CD4 values below a threshold (Philadelphia, CD4 ≤_350 cells/mm3 and <25% of total lymphocytes and New Jersey, CD4 <200 cells/mm3). Washington has had reporting of all CD4 and all VL values since 2006; CD4 and viral load tests on the same date were counted as one test.
§ Age, race/ethnicity, or risk was missing for some individuals; MSM: man who has sex with men; IDU: injection drug user.
Estimated Numbers and Percentages of Persons Not Linked to HIV Medical Care, According to Alternative Definitions of Linkage to Care, Among Those Living with HIV (Not AIDS)* as of December 31, 2008, by Project Area, Never in Care Project, 2006-2008
| Project Area | Population Living with HIV (Not AIDS) | Estimated Number (%) Under Definition 1† | Estimated Number (%) Under Definition 2§ |
|---|---|---|---|
| Indiana | 4,105 | 738 (18) | 1,839 (45) |
| NewJersey | 16,462 | 2,588 (16) | 7,751 (47) |
| New York City | 41,177 | 4,845 (12) | 9,315 (23) |
| Philadelphia | 7,242 | 1,357 (19) | 2,872 (40) |
| Washington | 4,297 | 300 (7) | 792 (18) |
| Total | 73,283 | 9,828 (13) | 22,569 (31) |
* Persons with AIDS, including those identified as HIV infected because of an AIDS diagnosis, were excluded from the denominator because they were presumed to be in care.
† Under definition 1, persons who had <1 CD4 or VL tests are considered not linked to care.
§ Under definition 2, persons who had <2 CD4 or VL tests are considered not linked to care.
As shown in Table 2, persons who had only one reported CD4 or VL test in the first 12 months after diagnosis were significantly different from both those who had no reported CD4 or VL tests and those who had two or more reported CD4 or VL tests with regard to race/ethnicity, risk, and jurisdiction. Persons who had only one reported CD4 or VL test in the first 12 months after diagnosis were also significantly different from those who had no reported CD4 or VL tests with regard to age. Finally, persons with only one reported CD4 or VL in the first 12 months were significantly more likely than those who had two or more CD4 or VL tests to have a first CD4 count below 200 cells/mm3.
Table 3 presents the results of the multivariable log-binomial regression analysis, which indicate that race/ethnicity and age at diagnosis were independently associated with having zero reported CD4 or VL tests in the 12 months after diagnosis vs having only one reported CD4 or VL test in this period. This suggests that the group having only one reported CD4 or VL test is not equivalent to those who have none, as might be the case if most of those with only one test had never actually had a care visit (i.e., if the test was ordered before referral to care). All races/ethnicities except Asian/Pacific Islander were significantly more likely than White, non-Hispanic persons to have zero reported CD4 or VL tests in the first 12 months after diagnosis vs only one reported CD4 or VL test; as were persons under 20 years old vs those 50 years old or older, and persons with undetermined mode of exposure to HIV vs those in the MSM risk group.
In contrast, race/ethnicity did not independently predict having only one vs two or more reported CD4 or VL tests in the 12 months after diagnosis, although age at diagnosis and risk group did. Persons 20-29 and 30-39 years old were significantly more likely than those who were 50 years old or older to have only one reported CD4/VL vs two or more tests, as were persons in all other risk groups compared with MSM. Also of interest is that jurisdiction was associated with the outcomes after controlling for population demographic differences — to a greater extent for the comparison of only one vs two or more reported tests than for zero vs only one reported test(s).
The sensitivity analysis comparing log-binomial regression model results — including and excluding jurisdictions to assess possible bias related to differences in CD4 and VL reporting requirements-- indicated little change in the magnitude of the associations of race/ethnicity, age, sex, and risk group with having only one reported CD4 or VL test vs two or more tests. When the two jurisdictions that require reporting of a subset rather than all CD4 and VL tests were excluded, the magnitude of the associations slightly increased for all factors except for race/ethnicity, for which there was a change in the other direction (data not shown); and confidence intervals were wider when these jurisdictions were excluded. However, the models including and excluding the two jurisdictions identified the same predictors of the outcome.
Never in Care Population
Among those who had no reported CD4 or VL tests by the end of the observation period, time since diagnosis at the end of the period was 19-24 months for 963 (18%), 25-36 months for 2,525 (48%) and 37-42 months for 1,772 (34%). Fig. (2a, b) show the Kaplan Meier probability functions (probability of not having linked to care by time since HIV diagnosis) for all 5 jurisdictions. Under definition 1, those who had at least one reported CD4 or VL test were considered to have linked to care (and those who had not had a reported CD4 or VL test were considered not to have linked to care, Fig. 2a). Alternatively, under definition 2, those who had two or more reported CD4 or VL tests were considered linked to care (and those with less than two reported tests were considered not to have linked to care, Fig. 2b). As shown in Fig. (2a), in all areas, the probability of not having entered care dropped quickly in the first 3-4 months after diagnosis, and the probability declined within 6 months after diagnosis to a level at which there was very little change through the end of follow up (42 months after diagnosis). However, the slope of the probability function until the rate of change slowed varied from area to area, and the relatively constant probability maintained after this point ranged from 6% to 17% among the areas. The same pattern was observed when linkage to care was defined as two or more reported CD4 or VL tests (Fig. 2b) although care entry was somewhat delayed compared with those who had at least one reported CD4 or VL test. The differences in the probability functions across jurisdictions were statistically significant (p<0.0001) under both definitions.
Table 4 presents the size of the population of persons diagnosed with HIV, according to our two definitions of linkage to care, as well as the percentages this group represents of the total population living with HIV at the end of December, 2008, by jurisdiction. Among the jurisdictions, the range of the estimated numbers of persons not linked to care among those living with HIV (not AIDS) as of December 2008 was 300-4,845 under definition 1 vs 792-9,315 under definition 2.
DISCUSSION
Our findings indicate that persons who have had only one reported CD4 or VL test represent a substantial proportion of those who have a reported CD4 or VL test within 12 months of diagnosis. Estimates of the size of the population yet to be linked to care will differ under alternative definitions of linkage to care (at least one reported CD4 or VL vs two or more). Furthermore, considering those who have had only one reported CD4 or VL test as either linked to care or alternatively, not linked to care, may mask important distinctions between persons linked to care and those not linked to care. HIV-infected persons fall along a continuum of engagement with the health care system, from persons who do not engage at all to those who engage sporadically, to those who regularly access care [33]. If linkage to care is defined as two or more reported CD4 or VL tests, combining persons who have had only one test with those who have not had any tests potentially blurs the distinction between those who do not engage in care and those who engage sporadically. Alternatively, if linkage to care is defined as at least one reported CD4 or VL test, combining persons with only one test with those who have two or more tests potentially blurs the distinction between those who engage in care sporadically and those who regularly access care.
Additionally, we found that demographic factors independently predicted having zero reported CD4 or VL tests performed in the 12 months after diagnosis vs having only one test, but (with the exception of age at diagnosis) did not predict having only one reported test vs having two or more reported tests performed in this period. Conversely, risk group predicted having only one reported CD4 or VL vs two or more but not zero vs only one reported test(s) (with the exception of undetermined risk). These findings underscore that persons with only one reported CD4 or VL may have characteristics distinct from both those who appear to have yet to enter care and those who appear to have engaged in care. Taken together, these data suggest that a single CD4 or VL test may overestimate entry into care for certain subgroups.
Our findings support the idea that those who have received only one reported CD4 or VL test are different from those who have received more than one test. However, to better understand this difference, it is critical to know how many of the persons with a first reported CD4 or VL received their first test as part of confirmatory testing (before the first care visit). Surveillance data would miss follow-up visits during which no CD4 or VL tests are ordered. Therefore, two or more tests are needed to indicate that care visits have occurred following the confirmatory testing. Information about physicians’ practices with regard to ordering CD4 and VL tests would also be useful for evaluating linkage to care programs because it would allow assessment of the rate of uncompleted care intake visits after confirmatory testing as well as the rate at which those who have entered care drop out of care. Differentiating between care non-entry and dropping out of care is important, as different interventions are likely to be needed to overcome each of these less-than-ideal outcomes. For the reasons outlined above, all three outcomes we examined in this paper—zero reported CD4 or VL tests, only one, and two or more—should be monitored.
Although using surveillance data to track HIV care patterns appears to be feasible, there are some limitations to this approach, a primary one being that surveillance programs do not collect information on patterns of ordering CD4 and VL tests that could clarify the meaning of a single CD4 or VL test result. Another limitation is that CD4 and VL reports may be delayed, affecting the utility of these surveillance data for real-time tracking of care patterns which might help support intervention [23]. This limitation is expected to become less important over time as jurisdictions transition to full, operational, electronic reporting of CD4 and VL results. In the meantime, retrospective analysis provides the means to track trends, albeit in a delayed timeframe.
Other factors may also affect how closely surveillance data reflect care patterns. Factors directly related to the surveillance system include incomplete reporting; receipt of medical care in a jurisdiction other than the one where HIV testing was performed; incomplete ascertainment of deaths and migration out of jurisdiction; errors such as linking reported CD4 or VL data for one person to another person’s case report; and duplication of case reports, which may happen when a person changes providers and gets re-tested for HIV [34]. However, the surveillance system has in place procedures and policies intended to mitigate these factors, such as routine linkage of surveillance data to state and national death registries and inter-state de-duplication. Other factors not related to surveillance per se include CD4 tests ordered for a health problem unrelated to HIV, or HIV-related medical care visits that do not include ordering CD4 or VL testing. Jurisdictions must know the specific limitations of their HIV surveillance data and external factors influencing its interpretation to make optimal use of the data to monitor HIV care patterns.
Some of the limitations of surveillance data described above may be more relevant to our analyses than others. The effects of reporting delay, and of possibly incomplete reporting in jurisdictions in the early stages of implementing CD4 and VL reporting, as well as under-ascertainment of deaths and out-migration were likely minimized through retrospective analysis, which allowed more time for entry of reported CD4 and VL data and for deaths and moves out of the jurisdictions to be documented. We focused on persons diagnosed from 2006-2008, and we included CD4 and VL tests reported through July 2010. We assumed that all jurisdictions had caught up with reporting CD4 and VL of persons diagnosed in 2006-2008 by mid 2010 (reporting delays should not have affected the data presented unless the delay was longer than 18 months).
During the course of the NIC Pilot Project, incomplete and inconsistent reporting by laboratories was discovered through routine validation efforts, e.g. follow-up of inconsistent numbers of tests reported by a laboratory from month to month. Incomplete and inconsistent reporting was also discovered through the activities conducted as part of the NIC Pilot Project, e.g., when the investigation of cases or screening for eligibility indicated a person had received care, but the surveillance record included no reported CD4 or VL tests. In cases such as these, in which there was evidence that laboratory tests had indeed been ordered, the relevant laboratory was contacted and reminded of its reporting obligations. In at least one such instance, the laboratory involved was able to produce an electronic file of historical data and the missing CD4 and VL tests were added to surveillance records, and included in this analysis. Reports from these laboratories were monitored prospectively for ongoing compliance. These improvements in reporting likely benefited the Never in Care Project, having been made as they were before July 31, 2010, until which time reported CD4 or VL, regardless of when they were performed, were included in the analysis. Even so, we cannot rule out the possibility that our analysis misclassified some persons as not having a CD4 or VL test, or having only one CD4 or VL test, if CD4 or VL tests had not been reported.
It is possible that we overestimated the numbers of persons who had not received HIV medical care because reporting laws in some participating jurisdictions did not require reporting of all CD4 and all VL tests. Less frequent ordering of VL tests because of their higher cost and/or insurance coverage considerations might have contributed to overestimation of the number of persons who had not linked to HIV care. This is a concern especially for jurisdictions where CD4 counts are not reported unless they are less than 200 or less than 350 cells/mm3, because of the resulting higher probability that a care visit would be reflected by neither a reported CD4 nor a reported VL test.
We may also have overestimated the number of persons who had not linked to HIV medical care under definition 2 (i.e., when we considered those who had less than two reported CD4 or VL tests not linked to care), because persons with only one reported CD4 or VL performed on a date less than 12 months before the end of the observation period were not considered to have entered care. It is possible that the study period ended before they had time to have a second test. However, there were only 94 persons whose only reported CD4/VL test was performed less than 12 months from the end of the observation period. Therefore, even if we considered all 94 persons to have been misidentified as never in care and reclassified them, the never in care population estimate would be reduced by only 2% (from 5,257 persons to 5,163).
Finally, because we had data to estimate probabilities of not entering care only up to 42 months after HIV diagnosis, we assumed that the probability of not having entered care for persons more than 42 months from diagnosis was equal to the probability of not having entered care for persons 42 months from diagnosis. As a result, our estimates of the population having entered care were likely conservative, potentially contributing to overestimation of the numbers and percentages of persons never in care among those living with HIV (not AIDS) and diagnosed for more than 42 months in each jurisdiction.
Results from our log-binomial regression analyses indicate that jurisdiction was independently associated with not being linked to care under the two definitions, suggesting that factors other than the differences in the measured population characteristics affected the relative success in linking HIV-infected persons to care. The volume of cases to be linked to care and the resources and types of systems and services available to facilitate linkage to care varied across the jurisdictions, and these factors warrant further investigation. It is also possible that surveillance system differences may have contributed to the difference in these estimates. In addition to the differences noted above, the proportion of reported CD4 or VL results received electronically vs entered manually differed, as well as requirements for reporting of CD4 and VL tests, and length of time for which reporting of all values of CD4 or VL was mandated. Although we were unable to investigate all these factors, the sensitivity analysis results did not indicate bias in the multivariable model results from including jurisdictions with CD4 and VL reporting requirements limited to a specific numeric range. In the absence of consistent data on completeness and validity of reporting and on the availability and effectiveness of linkage to care services across the jurisdictions, we acknowledge the possibility of mixed effects of these possible influences on the findings from the multivariable models.
Despite these possible limitations, surveillance data represent the best available data for monitoring HIV care patterns on a population level. Use of surveillance data makes possible a relatively standardized assessment of linkage to care rates across an entire jurisdiction, unlike meta-analyses, for which data may not cover as wide a geographic area, for which data sources are limited to published work, and in which heterogeneity in study designs and analytic approaches can present challenges.
Our findings indicate that 85% of persons diagnosed with HIV in the five jurisdictions had a first reported CD4 or VL test performed within 12 months of diagnosis, which is substantially higher than the 69% found in a recent meta-analysis. As the meta-analysis included studies predating our observation period, a possible explanation for this difference is that linkage to care has improved over time. From our analyses, we also learned that 27% of persons with at least one reported CD4 or VL in the 12 months following diagnosis did not have a second test, suggesting that nearly one-third of those who had an initial care visit did not continue to receive regular care as recommended. Our estimate is intermediate between estimates by Ulett et al., [35] and Giordano et al., [36] who measured receipt of care by number of clinic visits and found that 16% and 48% of patients attending a clinic intake visit were not subsequently seen in follow-up, respectively.
In our comparison of first CD4 test results for those who had only one reported CD4 test and those who had two or more reported CD4 tests, those who had only one test were significantly more likely to have a first CD4 less than 200 cells/mm3. This finding is consistent with other analyses of surveillance data and cohort study data indicating that many of those who do not enter or establish care soon after diagnosis delay care entry until they meet the definition of AIDS [31, 32].
Rajabiun et al. [37] have reported the findings of a qualitative interview study involving an underserved population of persons living with HIV in which participants described “a tenuous connection to health care despite having seen an HIV health care provider in the recent past,” a finding that has also been reported by others [38, 39] and that emphasizes that efforts to improve retention in care should be implemented in parallel with efforts to engage persons living with HIV in medical care.
Our findings also emphasize that in all areas, too many HIV-infected persons are entering care too late to derive maximum benefit from antiretroviral therapy [14]. The high proportions of persons living with HIV who had not received HIV medical care suggest that in some jurisdictions, the proportion of HIV- diagnosed persons not linked to care may exceed the proportion who have not yet been diagnosed, and that improvements in linkage to care proportionate to the expansion of HIV testing are needed. Finally, because we were focused on exploring two definitions of linkage to care, one of which did not apply to AIDS cases, we did not evaluate care utilization patterns among persons with AIDS. More work is needed to develop methods for population-level monitoring of care patterns among those with AIDS.
CONCLUSION
Findings suggest that HIV surveillance and associated laboratory data are useful for monitoring HIV care patterns at the population level. However, using a single reported CD4 or VL as an indicator of linkage to care may overestimate the numbers of HIV-infected persons who have established HIV medical care.
ACKNOWLEDGEMENTS
The authors acknowledge Susan Buskin, Mark Stenger, Gia Badolato, Michael Eberhart, Michael Connor, Pamela Garland, Corliss Heath, Anurag Jain, and Thomas Peterman.
CONFLICT OF INTEREST
The authors confirm that this article content has no conflicts of interest.
REFERENCES
| [1] | Marks G, Crepaz N, Senterfitt JW, Janssen RS. Meta-analysis of high-risk sexual behavior in persons aware and unaware they are infected with HIV in the United States: Implications for HIVprevention programs J Acquir Immune Defic Syndr 2005; 39(4 ): 446-53. |
| [2] | Metsch LR, Pereyra M, Messinger S, et al. HIV transmission risk behaviors among HIV-infected persons who are successfully linked to care Clin Infect Dis 2008; 47(4 ): 577-84. |
| [3] | Vernazza PL, Troiani L, Flepp MJ, et al. Potent antiretroviral treatment of HIV-infection results in suppression of the seminal shedding of HIV. The Swiss HIV Cohort Study AIDS 2000; 14(2 ): 117-21. |
| [4] | Coombs RW, Reichelderfer PS, Landay AL. Recent observations on HIV type-1 infection in the genital tract of men and women AIDS 2003; 17(4 ): 455-80. |
| [5] | Quinn TC, Wawer MJ, Sewankambo N, et al. Viral load and heterosexual transmission of human immunodeficiency virus type 1. Rakai Project Study Group N Engl J Med 2000; 342(13 ): 921-. |
| [6] | Tovanabutra S, Robison V, Wongtrakul J, et al. Male viral load and heterosexual transmission of HIV-1 subtype E in northern Thailand J Acquir Immune Defic Syndr 2002; 29(3 ): 275-83. |
| [7] | Kayitenkore K, Bekan B, Rufagari J, et al. The impact of ART on HIV transmission among HIV serodiscordant couples Poster Discussion: AIDS. In: XVI International AIDS Conference; Toronto, Canada. 2006. Abstract No: MOKC101 |
| [8] | Reynolds S, Makumbi F, Kagaayi J, et al. ART reduced the rate of sexual transmission of HIV among HIV-discordant couples in rural Rakai, Uganda In: [Abstr 52a], 16th Conference on Retroviruses and Opportunistic Infections; Montreal, Canada. 2009. Abstract no. 52a |
| [9] | Sullivan P, Kayitenkore K, Chomba E, et al. Reduction of HIV transmission risk and high risk sex while prescribed ART: Results from discordant couples in Rwanda and Zambia In: 16th Conference on Retroviruses and Opportunistic Infections; Montreal, Canada . 2009. Abstract No. 52b |
| [10] | Centers for Disease Control and Prevention. Advancing HIV prevention: new strategies for a changing epidemic MMWR Morbid Mortal Wkly Rep 2003; 52(15): 329-. |
| [11] | Lubinski C, Aberg J, Bardequez AD, et al. HIV policy: The path forward-A joint position paper of the HIV Medicine Association of the Infectious Disease Society of America and the American College of Physicians Clin Infect Dis 2009; 48: 1335-44. |
| [12] | Marks G, Crepaz N, Janssen RS. Estimating sexual transmission of HIV from persons aware and unaware that they are infected with the virus in the USA AIDS 2006; 20(10): 1447-50. |
| [13] | Office of National AIDS Policy National HIV/AIDS Strategy. Retrieved from: http://www.whitehouse.gov/administration/eop/onap/nhas [accessed on May16, 2010]; |
| [14] | Branson BM, Handsfield H, Lampe MA, et al. Revised recommendations for HIV testing of adults, adolescents, and pregnant women in health care settings MMWR 2006; 55(RR-14): 1-17. |
| [15] | DHHS Panel on Antiretroviral Guidelines for Adults and Adolescents Office of AIDS Research Advisory Council (OARAC). Guidelines for the Use of AntiretroviralAgents in HIV- 1-Infected Adults and Adolescents, December 1, 2009, page 35. Retrieved from: http://www.aidsinfo.nih.gov/contentfiles/Adultand AdolescentGL.pdf [accessed on May 16, 2010]; |
| [16] | Giordano T, Gifford A, White A, et al. Retention in care: A challenge to survival with HIV infection Clin Infect Dis 2007; 44(11 ): 1493-9. |
| [17] | Bucciardini R, Fragola V, Massella M, et al. Health-related quality of life outcomes in HIV-infected patients starting different combination regimens in a randomized multinational trial: the INITIO-QoL substudy AIDS Res Hum Retrovir 2007; 23: 1215-22. |
| [18] | Patient Protection and Affordable Care Act. P, L. 111-148, 124 Stat. 119, March 23, 2010. Retrieved from: http://www.gpo.gov/ fdsys/pkg/PLAW-111publ148/pdf/PLAW-111publ148.pdf [accessed on February 22, 2011]; |
| [19] | Estimating unmet need for HIV-related primary medical care: the basics. Mosaica. Retrieved from: http://www.mosaica.org/LinkCli ck.aspx?fileticket=VM7gakz1uCY%3D&tabid=2377&mid=4540 [accessed on February 22, 2011]; |
| [20] | Kahn JG, Janney J, Franks PE. A Practical Guide to Measuring Unmet Need for HIVRelated Primary Medical Care: Using the Unmet Need Framework.Institute for Health Policy Studies University of California, San Francisco, May 2003. Retrieved from: ftp://ftp.hrsa.gov/hab/unmetneedpracticalguide.pdf [accessed on December 20, 2010]; |
| [21] | Council of State and Territorial Epidemiologists. Laboratory reporting of clinical test results indicative of HIV infection: new standards for a new era of surveillance and prevention Position Statement number 04-ID-07. Retrieved from: http://www.cste.org /ps/2004pdf/04-ID-07-final.pdf [accessed on September 30, 2010]; |
| [22] | Centers for Disease Control and Prevention CD4 and viral load reporting by HIV infection surveillance reporting area, December 2009-50 states, District of Columbia, and United States dependent areas (Unpublished) |
| [23] | Mugavero MJ, Davila JA, Nevin CR, Giordano TP. From access to engagement: Measuring retention in outpatient HIV clinical care AIDS Patient Care STDs 2010; 24: 607-13. |
| [24] | Torian LV, Wiewel EW, Liu K, Sackoff JE, Frieden TR. Risk factors for delayed initiation of medical care after diagnosis of human immunodeficiency virus Arch Intern Med 2008; 168(11 ): 1181-7. |
| [25] | Zetola NM, Bernstein K, Ahrens K, et al. Using surveillance data to monitor entry into care of newly diagnosed HIV-infected persons: San Francisco, 2006–2007 BMC Public Health 2009; 9: 17-21. |
| [26] | Bamford LP, Ehrenkranz PD, Eberhart MG, Shpaner M, Brady KA. Factors associated with delayed entry into primary HIV medical care after HIV diagnosis AIDS 2010; 24(6 ): 928-30. |
| [27] | Ikard K, Janney J, Hsu LC, et al. Estimation of unmet need for HIV primary medical care: A framework and three case studies AIDS Educ Prevent 2005; 17(Suppl B ): 26-38. |
| [28] | Fagan J, Bertolli J, McNaghten AD. for the Never in Care Project. Understanding persons who have never received HIV medical care: a population-based approach Public Health Rep 2010; 125: 520-7. |
| [29] | Glynn MK, Ling Q, Phelps Ruby, Li J, Lee L. Accurate monitoring of the HIV epidemic in the United States: case duplication in the national HIV/AIDS surveillance system J Acquir Immune Defic Syndr 2008; 47(3): 391-6. |
| [30] | Mugavero MJ, Napravnik S, Cole SR, et al. Viremia copy-years predicts mortality among treatment-naive human immunodeficiency virus-infected patients initiating antiretroviral therapy Clin Infect Dis 2011; 53(90 ): 927-35. |
| [31] | Althoff KN, Gange SJ, Klein MB, et al. Late presentation for human immunodeficiency virus care in the United States and Canada Clin Infect Dis 2010; 50: 1512-20. |
| [32] | Centers for Disease Control and Prevention. Reported CD4+ T-lymphocyte results for adults and adolescents with HIV/AIDS-33 states, 2005. HIV/AIDS Surveillance Supplemental Report 2005; 11(No.2): 1-31. Retrieved from: http://www.cdc.gov/hiv/state/hasrli http://www.cdc.gov/hiv/surveillance/resources/reports/2005supp_v ol11no2/index.htm [accessed on July 8, 2012]; |
| [33] | Horstmann E, Brown J, Islam F, Buck J, Agins BD. Retaining HIV-Infected patients in care: Where are we? Where do we go from here? Clin Infect Dis 2010; 50: 752-61. |
| [34] | Hanna DB, Tsoi BW, Begier EM. Most positive HIV western blot tests do not diagnose new cases in New York City: Implications for HIV testing programs J Acquir Immune Defic Syndr 2009; 51(5 ): 609-14. |
| [35] | Ulett KB, Willig JH, Lin H-Y, et al. The therapeutic implications of timely linkage and early retention in HIV care AIDS Patient Care STDs 2009; 23(1 ): 41-9. |
| [36] | Giordano TP, Visnegarwala F, White AC Jr, et al. Patients referred to an urban HIV clinic frequently fail to establish care: factors predicting failure AIDS Care 2005; 17: 773-83. |
| [37] | Rajabiun S, Mallinson RK, McCoy K, et al. “Getting me back on track”: The role of outreach interventions in engaging and retaining people living with HIV/AIDS in medical care AIDS Patient Care STDs 2007; 21(Suppl 1 ): S20-9. |
| [38] | Meyerson BE, Klinkenberg D, Perkins DR, Laffoon BT. Who’s in and who’s out: Use of primary medical care among people living with HIV Am J Public Health 2007; 97(4 ): 744-9. |
| [39] | Mallinson RK, Relf MV, Dekker D, Dolan K, Darcy A, Ford A. Maintaining normalcy: a grounded theory of engaging in HIVoriented primary medical care Adv Nurs Sci 2005; 28(3 ): 265-77. |
