This analysis of the 11 types of CIT use measured in the IHA-P4P program presented several challenges in developing an adequate statistical analysis, including:
Comparability of disaggregated outcomes – The pace and extent of adoption of the 11 CIT innovations varied considerably and not all of these innovations were introduced into the IHA-P4P measure set at the same time.
Timing of adoption – If independent variables are only temporarily associated with an outcome the use of ordinary logit of probit analysis may be misleading or unreliable (253).
Censored observations – Observing time of adoption using cross sectional survey data presents limitations related to observations with uncertain outcomes. In this analysis two types of censored data were present. The first involves POs not reporting adoption during the study period. Designating these POs as non-adopters does not address the possibility that they will adopt at a future date or that they may have adopted, but not reported. This limitation is characterized as “right” censoring.
A second type of censored observations includes POs reporting CIT use during the first year of the study, but who may actually have adopted earlier. This limitation is characterized as “left” censoring (254).
Several different statistical methods were applied in an effort to address these challenges, to find the most reliable test, and to cross reference results. To start, logistic regression analysis was conducted with use of each individual CIT type (yes/no) as a dependent variable for each year, eliminating all POs not reporting CIT adoption. The results were neither consistent over time nor sufficiently reliable to lend themselves to interpretation (regression results included in Appendix 4). This result suggested the aggregation of individual CIT use outcomes by developing composite scales or indices in an effort to generate more reliable and meaningful results. A variety of scales and indices were created as dependent variables to assess POs’ use of CIT and tested for reliability.
The following three indices were used in the final model:
Population Management CIT Index (POP-index) – a summary measure of the 3 data integration CIT innovations for population management in the P4P IT domain with a range of 0 to 3 (Hereafter referred to as Population Management CIT). This index included 3 types of CIT for the entire study period because the number of measures in this category did not change over the time of the study.
Point of Care CIT Index (POC-index) – a summary measure of the 5 to 8 CIT innovations for point of care in the P4P IT domain with a range of 0 to 5 for 2003, 0 to 7 for 2004- 2006 and 0 to 8 for 2007, reflecting the addition of point of care capabilities and measures to the P4P IT domain during the study period.
Total CIT Index (E-index) – a summary measure of all 8 to 11 CIT innovations in the P4P IT domain with a range of 0 to 8 in 2003, 0 to 10 from 2004-2006 and 0 to 11 in 2007, reflecting the total number of CIT capabilities in the P4P IT domain present each year during the study period.
These indices treated CIT innovations equally because each was weighted equally when determining eligibility for incentive payments. Cronbach’s alpha reliability coefficients were computed to test internal consistency of these constructs. Results were all above 0.70, indicating good to excellent reliability for all three constructs as reflected in Table 13 on the next page (255).
Table 13: Reliability of Population Management ("Pop IT"), Point of Care ("Poc IT"), and All CIT ("E Index") Indices, (2003-2007)
Pop IT Index Poc IT Index E Index
Year # of Variables in Index Chronbach’s Alpha. # of Variables in Index Chronbach’s Alpha. # of Variables in Index Chronbach’s Alpha. 2003 3 0.7159 5 0.8453 8 0.7981 2004 3 0.8682 7 0.8409 10 0.8742 2005 3 0.9303 7 0.8644 10 0.8978 2006 3 0.9578 7 0.8979 10 0.9087 2007 3 0.9678 8 0.8921 11 0.9330
Table 14 on the next page includes the adoption rates for each CIT innovation included in this analysis and the mean index scores for each year from 2003 through 2007.
In 2003 PO IT Survey respondents demonstrated individual CIT use within the population management category at adoption rates ranging from 21% up to 51% with a mean index average of 1.1 on a scale of 3.0. By 2007 this increased to CIT adoption rates ranging from 90% to 98% of IT Survey respondents with a mean average of 2.8 on a scale of 3.0.
In 2003 IT Survey respondent use of point of care CIT was lower than population management CIT, ranging from adoption rates of 10% up to 37% with a mean index average of 1.0 on a scale of 5.0. By 2007 this increased to CIT adoption rates ranging from 32% up to 78% with a mean index average of 4.5 on a scale of 8.0.
In 2003 the use of all CIT had a mean index average of 2.1 on a scale of 8.0 increasing to a mean index average of 7.3 on a scale of 11.0 in 2007.
Multivariate linear regression was used to test each index as a continuous dependent variable for each year in the study. Again, those POs not reporting CIT use during the study period (right censored observations) were eliminated (truncated) from the analysis. POs reporting CIT use during the first year of the study were included, ignoring concerns about POs who adopted CIT before the reporting date (left censored observations).
Using these research constructs instead of individual CIT capabilities as dependent variables yielded results which were more consistent over time. However, some independent variables were only temporarily associated with the outcome, therefore the reliability of this approach remains questionable within the context of a probit model.
Table 14: Percent of PO IT Survey Respondents Reporting CIT Use in IHA-P4P (2003-2007) 2003 . 2004 . 2005 . 2006 . 2007 . Measures . # of POs . % of POs . # of POs . % of POs . # of POs . % of POs . # of POs . % of POs . # of POs . % of POs .
Total IT Survey Respondent
POs 95 112 113 130 122
Population (“Pop”) Management IT Activities: 1. Actionable Report/Data
Warehouse 48 50.5% 88 78.6% 88 77.9% 114 87.7% 119 97.5%
2. Computerized Registries 34 35.8% 62 55.4% 88 77.9% 116 89.2% 111 91.0%
3. HEDIS Results 20 21.1% 78 69.6% 85 75.2% 112 86.2% 110 90.2%
Pop IT Index Points Average 1.1 2.0 2.3 2.6 2.8
Point of Care (“Poc”) IT Activities:
1. Electronic Prescribing 9 9.5% 18 16.1% 22 19.5% 35 26.9% 39 32.0%
2. Electronic Drug Checks 9 9.5% 27 24.1% 31 27.4% 47 36.2% 52 42.6%
3. Electronic Lab Results 35 36.8% 61 54.5% 82 72.6% 86 66.2% 96 78.7%
4. Accessing Clinical Notes
of Other Practitioners 24 25.3% 48 42.9% 65 57.5% 78 60.0% 93 76.2%
5. Physician Preventive and
Chronic Care Reminders 14 14.7% 27 24.1% 36 31.9% 53 40.8% 84 68.9%
6. Accessing Clinical Findings such as Blood Pressure1
− − 24 21.4% 26 23.0% 39 30.0% 57 46.7%
7. Electronic Messaging1 − − 24 21.4% 57 50.4% 75 57.7% 72 59.0%
8. Ordering Lab Tests2 − − − − − − − − 55 45.1%
Poc IT Index Points Average - Activities 1-8
1.0 2.0 2.8 3.2 4.5