Design and Weighting Methods for a Nationally Representative Sample of HIV-infected Adults Receiving Medical Care in the United States-Medical Monitoring Project

First-stage Sampling: Project Areas

The first-stage sampling frame consisted of the 50 U.S. states, the District of Columbia, and Puerto Rico. In 2004, 20 areas were selected with PPS (i.e., the measure of size equaled the number of persons reported to be living with a diagnosis of AIDS at the end of 2002).

A number of states had more than a 1/20 share of the size measure and were selected with certainty. Five U.S. cities and 1 county (Chicago, Houston, New York City, Philadelphia, San Francisco, and Los Angeles County) received federal funding for their own local surveillance programs and thus constituted a special circumstance operationally. All six were contained in states that were sampled with certainty, and each became an independent project area, complementary to the rest of the state. Beginning in 2009, budget constraints made it necessary to reduce the number of funded project areas by dropping a random subsample of three moderate-prevalence states. The 23 project areas that participated in the 2009 cycle of MMP (see Table 1) included not only states with high morbidity but also some with moderate and low HIV prevalence, which may not typically be included in a study monitoring HIV care. Each of the US Census divisions (Northeast, South, Midwest, and West) was represented in the sample. Sampling has not been repeated at the PSU level since project inception; thus, the selected states/territories and cities conducting MMP have remained consistent since 2009.

Second-stage Sampling: Facilities and Second-stage Units

Various sources were used to identify facilities that provide HIV medical care, which were selected at the second stage of sampling [7]. The measure of size used for facility sampling was the estimated number of patients who received care at the facility during January-April of the reference year, the population definition period (PDP). Project areas obtained these estimated patient load (EPL) figures by asking sampled facilities to query their billing databases (ideally) or consult other records to estimate the number of HIV-positive patients seen during the PDP.

Small facilities complicated the PPS sampling design. Below a certain facility size threshold, the patient sampling rate needed to restore equal probability sampling across the two stages of sampling would exceed 100%. To avoid this, small facilities were linked to larger facilities to form groupings whose aggregate measure of size always exceeded this facility size threshold. The facility groupings (which may be thought of as pseudo-clusters) were sampled; that is, all facilities in the linked group were considered to have been sampled if the group was selected. Sampling units are sometimes linked for administrative or logistical reasons, but we linked facilities only to preserve equal probability sampling at the patient sampling stage. Facility samples were selected every 2 years; independent patient samples were selected each year from participating facilities.

Third-stage Sampling: Patients

Information on patient volume obtained from participating sampled facilities was used to determine patient sampling fractions. Although all facilities in the frame were asked for an EPL, the participating facilities provided an actual count of HIV-positive patients aged 18 years and older who received any type of medical care at the facility during the PDP. This updated measure of size, the actual patient load (APL), was used to determine patient sampling probabilities so as to restore equal probabilities of selection across the two stages of sampling. For each sampled facility in a given project area, the product of its selection probability and its patients’ probabilities of selection was constant. The selection probabilities for patients were based on the updated count and were equal for all patients of the same facility, although selection probability differed across facilities.

Sample Sizes and Allocation

Design effects expected for this population under a two-stage clustered design have not been reported. The choice of facility sample size (range: 20-60) was guided by the following considerations. A project area was allocated a slightly larger facility sample size if (1) the area had a large number of facilities, (2) the area had many facilities with few patients, (3) the distribution of the measure of size across facilities was otherwise skewed or irregular, or (4) the state chose to stratify implicitly by geographic region.

The minimum patient sample size per state was set at 400 to ensure sufficient precision [10]. This sample size was not inflated to compensate for patient nonresponse or for the design effect due to clustering. Patient sample sizes were increased for a few project areas (California, Florida, New Jersey, New York City, and Texas), in recognition of the greater burden of HIV/AIDS in those states. Patient sample sizes were reduced for states (Illinois, New York, and Pennsylvania) where the largest urban area bore the greatest burden of HIV/AIDS and was also a participating project area. When the state and the city were combined (Chicago, Illinois; New York City, New York; and Philadelphia, Pennsylvania), sample sizes were adequate for statewide estimates; however, Illinois (excluding Chicago), New York (excluding New York City), and Pennsylvania (excluding Philadelphia) did not have sample sizes large enough to produce independent estimates based on a single year of data.

Patient sample sizes were allocated to participating facilities in accordance with selection probabilities that restored equal probability sampling across the project area. Data on sampled patients were collected in annual, cross-sectional cycles from June of the reference year (the year containing the January-April PDP) through May of the following year. Table 1 indicates the patient sample size for each project area in the 2009 MMP data collection cycle (see Table 1).

Computation of Base Weights

The patient base weight consists of two components, accounting for the facility’s probability of selection and the patient’s conditional probability of selection given the sampled facility. As some facilities were sampled as part of a group of facilities, or SSUs, the facility base weight (W¹) is the inverse of the probability of selection of the SSU,

where P_1j is the probability of selection for the SSU in the given project area and j indexes the patient in an SSU.

The second component of the patient base weight is the inverse of the patient’s probability of selection, given that the facility was selected:

where P_2j is the probability of selection for patient j in the given sampled facility, or SSU. The design weight is the product of these two base weights.

A series of adjustments was performed on the base weights to reflect nonresponse (including facility, demographic, and survey nonresponse adjustments) and multiplicity. To distinguish these response rate and multiplicity adjustments from the base weights, we use the following notation:

Thus,

and so forth, with the caret indicating the cumulative product of all multiplicative adjustments up to the jth factor.

Response Rate Adjustments

The interview data were subject to the highest levels of nonresponse and therefore potentially subject to greatest nonresponse bias. Response was higher for the other data collection components (i.e., MDS and MRA), which were collected without the requirement of patient consent in areas conducting MMP under their public health authority.

Table 2 presents facility-level and patient-level response rates. Facility participation ranged from 45% to 100% (a rate achieved in five project areas). Adjusted rates of patient-level response ranged from 26% to 71%. The effective combined response rates (facility and patient levels) ranged from 17% to 71%. Adjusted response rates are CASRO rates, standard response rates developed by the Council of American Survey Research Organizations [19]; the denominator includes a term for the estimated number of eligible patients among those whose eligibility status was unknown.

Nonresponse Adjustment Classes

The purpose of nonresponse adjustments in weighting is to adjust the weights for respondents so that the data better represent both respondents and nonrespondents [20]. The simplest forms of nonresponse adjustments are based on weighting classes defined by using a few selected variables. Ideal weighting classes are homogenous in terms of response rates in each class as well as for the presumed key outcomes. Weighting classes should not have small sample sizes. Specifically, each class needs to contain a sufficient number of respondents not only to permit calculation of meaningful adjustments but also to prevent instability in the adjusted weights. Finally, from a practical perspective, the variables used to define weighting classes must be available both for respondents and nonrespondents. In general, constructing weighting classes by using many variables would violate these statistical guidelines [21].

The most useful data sets for defining weighting classes were the MDS and the facility attributes data set. These data sets had the highest response rates (MDS response rate = 87.8%; facility attributes response rate =100%), provided sufficient number of respondents, and included information on both respondents and nonrespondents to the interview. The menu of nonresponse adjustment variables from these data sets was limited, based on availability of data, to a few key demographics and patient-level characteristics-race, age, and years since diagnosis-as well as facility-level variables such as facility size, type, and affiliation.

The MMP weighting classes defined in different project areas used different weighting class variables. In a project area, and from the subset of key variables available for weighting classes in the MDS and facility attributes data, we analyzed response rates to select the variable(s) most strongly correlated with response propensity for use in constructing appropriate weighting classes, taking care to avoid small cells that might lead to unstable adjustments. Analyses included t tests to compare response rates for subgroups.

Weight Adjustments for Nonresponse

Weight adjustments were conducted at one level for facilities and at two levels for patients.

Facilities were classified as respondents, eligible nonrespondents, or ineligibles. A respondent facility was one that submitted APLs for the PDP or reported that although it was still in business, it had no patients during the PDP. An ineligible facility was one that did not provide outpatient HIV-related medical care to adults.

During the MMP weighting process, several stages of nonresponse adjustments were applied to the patient-level data. Sampled patients were classified into four categories: (1) eligible respondents, (2) eligible nonrespondents, (3) ineligible patients, and (4) patients whose eligibility was unknown. A patient was deemed ineligible for weighting purposes if the patient was younger than 18 years of age at the time of sampling, was not HIV-positive, or did not receive care during the PDP.

Patients were classified as eligible nonrespondents if the interview took place but was not completed, the interview records were missing, or the patients declined to participate. Other patients who did not complete the interview were classified as eligibility unknown.

For many of the patients for whom interview data were unavailable, data were available from the MDS, and facility attributes data were available for all patients from respondent facilities. Therefore, the process consisted of (1) adjusting facility base weights by using facility attributes data, (2) adjusting weights for patients with only MDS data by using attributes data of the facility from which the patient was sampled, and (3) adjusting weights for patients with interview data by using MDS data. These steps are described in the next subsections.

Variables such as facility size, facility type, and patient age, which were significant predictors of participation, were used to create nonresponse adjustment categories. The three nonresponse adjustments for MMP were facility nonresponse adjustments, demographic nonresponse adjustments, and interview nonresponse adjustments. Weighting classes were defined separately for each adjustment and differed by project area.

Facility Nonresponse Adjustment

At the facility level, the definition of weighting classes was limited by the relatively small number of sample facilities in each project area as well as by the limited number of facility-level variables. Moreover, these variables were highly correlated. Therefore, we used facility size to define nonresponse weighting classes in each project area. Two size classes were defined in each project area by using the median EPL for eligible facilities in the project area.

For a particular weighting class, the facility nonresponse adjustment was the ratio of two weighted sums of EPLs: the sum of weights over all eligible facilities-combining both participating facilities (denoted as group A) and nonparticipating facilities (denoted as group B)-divided by the sum of weights over participating facilities only. The weight adjustment (W₃) for all facilities in weighting class i is then:

and the base weight for patient j, adjusted for facility non-response, is Ŵ_3ij.The weights used in these weighted sums, for both numerator and denominator, were the facility base weights.

Note that the EPL was used because it was available for all facilities in the initial sample, whereas the APL was available only for participating facilities.

Demographic Nonresponse Adjustment

The next adjustment to the patient weights used the MDS to create weighting classes. The MDS was intended to provide demographic information and other, limited patient-level characteristics for the initial sample (respondents and nonrespondents), but the data were not available for all patients in the initial sample because some could not be linked to surveillance registry records. Without patient-level characteristics from MDS or medical records, only less-informative data on facility characteristics could be used for weight adjustments. Thus, when MDS information for respondent patients was missing, we used MRA data as a proxy for the MDS data. Then, for weighting purposes only, we imputed MDS data for the few interview respondents without MDS or MRA values for the key variables used to define weighting classes.

The nonresponse adjustments also used facility-level variables to create project-area–specific weighting classes. We defined binary variables related to responding classes, such as facility size (median, based on APL), university affiliation, and private practice status, which divided the initial patient sample into two approximately equal groups and created groups with differing response rates. Despite many similarities, the factors used for these classes differed by area in this data-driven approach.

The following specifies how we defined the adjustment cells by area:

• Facility size: California, Chicago, Florida, Illinois, New Jersey, Oregon, Texas, and Washington
• Facility’s university affiliation: New York City
• Private practice or public facility: Georgia and San Francisco
• No dichotomized adjustment: Delaware, Houston, Indiana, Los Angeles, Michigan, Mississippi, New York State, North Carolina, Pennsylvania, Philadelphia, Puerto Rico, and Virginia

Within each weighting class, the adjustment factor was the ratio of two sums of weights: (1) the sum of these weights for the entire patient sample and (2) the sum for the patients with MDS data.

The weight adjustment started with the base weights Ŵ_3j adjusted for facility nonresponse, as described in the preceding subsection. The adjusted weight for patient j,

is the ratio of two weighted sums. The weighted sum in the denominator is over MDS participants only (class A), and the weighted sum in the numerator is over all MDS sampled patient records (classes A and B). Thus,

Survey Nonresponse Adjustment

At this stage, every sampled patient with MDS data and every nonrespondent had a value for each of the weight components (W_1j W_2j W_3j W_4j); hence, for Ŵ_4j. The patients without MDS or interview data were then removed from the weighting process, and the respondent sample weights were adjusted so that in each weighting class, the weight sum equaled the sum of the weights for the entire data set (including demographic variables).

For every patient with MDS data in each project area, we identified a categorical variable-a facility characteristic or an MDS variable-that was related to the response rate. As in the preceding step, the variable was chosen on the basis of two conditions-the degree to which it divided the sample of patients with MDS data into two approximately equal groups and the degree to which the response rates of the two groups differed.

The patient nonresponse adjustment cells (or weighting classes) differed by project area. The following specifies how we defined the adjustment cells by area:

• Median facility size: Delaware, Florida, Philadelphia, and San Francisco
• Facility’s university affiliation: Mississippi, Puerto Rico, and Washington
• Private practice or public facility: Chicago, Georgia, Houston, Los Angeles, Michigan, New York City, and North Carolina
• Patient age: Pennsylvania (where the groups were 18-24 years vs. 25 years and older) and Indiana (where the groups were 18-44 years vs. 45 years and older)
• Hispanic/Latino ethnicity: California
• No dichotomized adjustment: Illinois, New Jersey, New York State, Oregon, Texas, and Virginia

For this next adjustment, within each adjustment cell, A was defined as the class of MMP respondents and B as the class of MMP eligible nonrespondents with MDS data (or in some instances, MRA data). We also defined the indicator hj and set hj = 1 if the patient j was eligible, hj = 0 if ineligible; for patients of undetermined eligibility, hj equaled the proportion of eligible patients in the weighting class among patients in the weighting class for whom eligibility was known. The adjustment factor for nonresponse among MDS respondents, W_5j, combines the adjusted weights for classes A and B:

Multiplicity Adjustment

After the nonresponse adjustments, a multiplicity adjustment was considered advisable because patients who had received treatment at other eligible facilities during the PDP might have had a higher probability of selection, introducing bias. As in every weight adjustment, the potential for bias reduction needs to be balanced against any increased variability that may be introduced by the adjustment guidelines [21].

Conceptually, patients with a potential for selection from multiple facilities in the frame had a greater probability of selection. The adjustment applied to eligible facilities in the frame whether or not they had been included in the sample.

Multiplicity adjustments typically involve a series of approximations necessary to estimate the multiplicity factor [22]; in terms of MMP, this adjustment used the number of facilities at which the patient received HIV care during the PDP. This adjustment often is expressed in terms of a factor by which the weight is divided. This factor was 1 if the patient visited only one facility during the PDP. The factor equaled 2 if the patient visited a second facility, and more generally, it equaled m if, in total, m eligible facilities were visited during the PDP by the given patient. In the MMP interview, as is common in other data collections, this number was estimated from self-reported data. The multiplicity adjustments we applied in this study did not divide the weights by the factor, but approximated the probability of selection given that the patient could have been selected through more than one facility.

The MMP interview included questions that allowed for approximating the multiplicity factor (m) where the respondent reported the number of other facilities visited for HIV care. Suppose that a patient reported visiting m eligible facilities in the frame. The probability of selection for this patient, considering the probability of not including the patient in the sample, will be the product of not including the patient in any of the m facilities.

The estimated probability of not including the patient may be expressed as:

Thus, the probability of not selecting the patient in any of the m facilities is estimated as:

and the estimated probability of selecting the patient multiple times then becomes:

For weighting purposes, it was assumed that each patient-facility visit would have the same weight, with the same probability of selection for each pair. We estimated as the adjusted probability of selecting patient j from any one of the m facilities from which patient j could have been selected. The weight can then be expressed as:

This makes the desired weight adjustment factor:

Because relatively few patients went to more than one HIV care facility (7.1% reported going to one additional facility, and only 0.7% reported going to two or more), we imposed a cap of m = 2, which results in the simplified calculation:

Thus, the multiplicity adjusted weight used was:

Following standard practice, we considered one last step before finalizing project area weights. The intent was to trim weights if any weights in a project area were more than three times the median weight. No project area weights were more than three times the median weight, so no trimming was conducted at this level, and Ŵ_6j became the project area weight.

The project area weight was the starting point for the creation of state and national weights, described next.

State and National Weights

The weights described in previous sections applied to the project areas, which are the PSUs of the national sample. In addition to creating project area estimates, we needed to create national estimates and estimates for states in which a city constituted a separate project area from the rest of the state. This subsection describes the computation of weights at the state and the national level.

b. National Weights

The calculation of national-level weights started with the weights computed for each project area, Ŵ_6j. Let W_7j be the inverse of the probability of selection of the state from which patient j was sampled. Given that funded cities were in certainty states, W_7j was 1.0 for all the patients from any of the cities or the certainty states. The new weight, Ŵ_7j, the product of Ŵ_6j and W_7j, became the initial national weight for patient j.

The final step in producing national weights was weight trimming, implemented to limit the variability in national weights (extreme weights at the national level can lead to large variances, but if one trimmed the extreme weights nationally, the distribution of various demographic characteristics would be distorted). Our solution was to conduct trimming by using weighting categories representing key demographics [23]. This approach divides the sample by demographic characteristics defining weighting categories, in this instance, race/ethnicity (African American, Hispanic/Latino, and other), age (<45 years and ≥45 years), and gender. The median of the Ŵ_7j was determined at the national level and for each weighting cell. The weights were capped at three times the smaller of the two medians, and the difference was added proportionately to the patients in the same weighting category.

Let f_kj be 1 if patient j is in cell k and 0 otherwise. Let u be the median of Ŵ_7j across the nation and u_k be the median weight of weighting cell k. Then,

and

constitute the trimmed national weights.

MRA Weights

The MRA weighting procedures were similar to those developed for the interview weights. Also similar were the weight adjustments, including those for nonresponse and multiplicity. However, to control totals, one further adjustment was made at the project area level.

Let W_INT be the sum of the interview weights for a given project area. The adjustment factor was then computed as follows, where the sum is over the project area:

Therefore, the adjusted weight is:

These weights add to the same total, W_INT, for the project area. Thus Ŵ_7j is the MRA weight at the project area level.

We implemented the same procedure at the end for national MRA weights.

Overlaps and Subsets

Weights were created for two subsets of data sets. One was the long form of the interview. The long, or standard, form of the interview was used most of the time. The short form was administered only when the respondent was too ill or otherwise unable to complete the longer standard interview, or when translation was required. A second subset was the overlap, or matched pairs from the interview and MRA data sets. A three-step adjustment process was applied to each data set individually. First, the weights were multiplied by a constant so that their sum equaled the sum of the interview weights for the same project area. Second, the weights were divided by the probability of selection of the project area. Third, the same trimming process was applied to these new weights, using the same procedures and trimming categories as used for the national interview weights.