Table Of ContentThe Developmentand Testing of
RiskAdjusters Us'ng Medicare Inpatientand
Ambulatory Data
FINALREPORT
Preparedfor
HealthCareFinancingAdministration
ContractNc. 93-026/EE
Preparedby:
TheLewinGroup
AllenDobson,Ph.D.
KevinColeman
TheJohnsHopkinsUniversity
JonathanWeiner,Dr.P.H.
GerardAnderson,Ph.D.
BarbaraStarfield,M.D.,M.P.H.
StephanieMaxwell
Augusts, 1996
Thestatementscontainedinthisreportaresolelythoseoftheauthorsanddonotnecessarilyreflecttheviews
orpolicesoftheHealthCareFinancingAdministration.Thecontractorassumesresponsibilityfortheaccuracy
andcompletenessoftheinfomnationcontainedinthisreport.
96FM0007
ACKNOWLEDGMENTS
The primary authors ofthis report are Allen Dobson, Ph.D., Project Director, with
assistanceofKevinColeman,ProjectCoordinatorof TheLewinGroup;andJonathanWeiner,
Dr. P.H., Principal Investigator; Gerard Anderson, Ph.D., and Barbara Starfield, M.D., Co-
InvestigatorsandStephanieMaxwell,ProjectManageroftheJohnsHopkinsUniversity.
The authors woulo especially I.;,_ to recognize the energy and effort of the
programming staff of Johns Hopkins University: Yifei Hu, Programming Analysis; Chad
Abrams;andAndrewBaker.
TheauthorsgratefullyacknowledgethesupportandassistanceofMel ingber, Ph.D.,
the ProjectOfficerforthis contract. His insights andfeedbackthroughoutthe studywere
particularlyhelpfulinidentifyingandcorrectingtechnicalconcernsandintheimportantlessons
andpolicyimplicationsofourwork.
96FM0007 TheLewinGroup
TableofContents
EXECUTIVESUMMARY ES-1
A. BACKGROUNDANDOVERVIEW ES-1
1. RiskAssessment,RiskAdjustment,andRiskAdjustedCapitated
PaymentSystems ES-1
2. TheAAPCC ES-2
B. MODELDEVELOPMENT ES-3
1. ModelComponents ES-3
2. DatabaseDevelopment ES-4
3. DesigningtheJHURiskAdjusterModels ES-5
a. RateCellsvsRegressionAnalysis ES-5
b. Expendituresin'/ear2:TheDependentVariable ES-6
c. RiskAssessors:TheIndependentVariables ES-6
C. EVALUATINGTHEJHURISKADJUSTERMODELS ES-8
1. EvaluationMeasures ES-9
2. RandomGroups ES-9
3. Non-RandomGroups ES-10
4. GamingandAdministrativeFeasibility ES-15
D. CONCLUSIONS ES-17
AC.HAPTEBRACI.KIGNRTROOUDNUDCTION 11--11
21.. CRuirsrkeAndtjMuesdtimceanrteDHefMinOedRiskAdjustedPaymentMethod:AAPCC 11--11
a. CriticismsofAAPCC 1-2
3. CurrentMedicarePolicyContext 1-3
B. PROJECTGOALSANDOBJECTIVES 1-3
C. CONCEPTUALBASISOFMODELS:ADGSANDPACS 1-4
1. ADGsMorbidityClusters 1-4
2. PACS 1-5
96FM0007 i TheLewinGroup
TableofContents(Continued)
CHAPTERII.METHODS ||-1
A. DATAFILEDEVELOPMENT 11-1
1. BasePopulation 11-1
2. PopulationExclusions 11-1
3. Split-halfMethod 11-2
4. DependentVariable 11-2
a. DependentVariableConstruction 11-2
b. AdjustnnenttoYear-twoDecedents 11-3
c. OutlierTruncation 11-4
d. OtherDependentVariables 11-4
B. RISKADJUSTERMODELDEVELOPMENT 11-4
1. OverviewandPhilosophy 11-4
2. Socio-DemographicData 11-5
3. AmbulatoryDiagnosesandADGs 11-5
4. InpatientDiagnosesandMDCs 11-6
5. Ambulatory/lnpatient(ADG/MDC)Integration 11-7
6. AnAlternativeIntegratedModel:"HospitalDominant"Diagnoses 11-8
7. ClaimsSourcesforMDCandADGAssignments 11-9
a. MDCAssignmentCriteria 11-9
b. ADGAssignmentCriteria 11-9
c. Face-to-FaceProcedureDiagnoses 11-10
CHAPTERIII.RESULTS:ADG-MDCANDADG-HOSDOMMODELS IIM
A. EXPLANATIONOFMULTIVARIATEREGRESSIONRISKADJUSTERMODELC 111-1
B. INTERPRETATIONOFVARIABLESINJHUMODELS III-2
1. DependentVariable 111-2
2. IndependentVariables III-2
a. TheADG-MDCModel III-2
b. TheADG-HosdomModel 111-4
c. TheAAPCC III-5
d. BaseExpectedPayment III-6
C. ILLUSTRATIONOFCAPITATIONRATEDETERMINATIONUSINGJHUMODELS III-6
D. SIMULATIONOFSTOP-LOSSREINSURANCEMECHANISMS III-9
E. DATAREQUIREMENTSFORJHUMODELS 111-10
96FM0007 ii TheLewinGroup
TableofContents(Continued)
CHAPTERIV.EVALUATINGANDTESTINGTHEJHURISKADJUSTERMODELS IV-1
A. MEASURESANDTESTSUSEDTOEVALUATETHEPREDICTIVEACCURACY
OFRISKADJUSTERMODELS IV-1
1. TheTypesofTest IV-1
a. AdjustedRSquareStatistic IV-1
b. ThePredictiveRatio iV-2
2. ThePopulationsUsedtoTestaGivenModel IV-3
a. RepeatedRandomSubsamples IV-3
b. Non-RandomGroups IV-3
B. RESULTS IV-4
1. RandomSamples IV-4
a. RiskAdjustersAlone IV-4
b. RiskAdjusterswithReinsurance IV-9
2. Non-RandomGroups IV-11
a. Age,Gender,andRace IV-11
b. TheUseofMedicalServices IV-16
c. MedicalConditions IV-16
d. ExpenditureGroups IV-18
e. GeographicRegion IV-22
C. CONCLUSIONS IV-24
CHAPTERV.GAMINGANDADMINISTRATIVEFEASIBILITY V-1
A. GAMING V-l
1. OpportunityforUpcoding V-1
2. OffsetstoUpcoding V-4
3. PredictiveAccuracyversesSusceptibilitytoGaming V-5
4. TheLocusofCare V-5
B. ADMINISTRATIVEFEASIBILITY V-6
1. DataCollectionRequirements V-7
2. CreatingaCapitatedRiskAdjustmentPaymentSystem V-7
3. PotentialPerceptionsOfRiskAdjusterMethodsByPlans V-8
96FM0007 iii TheLewinGroup
TableofContents(Continued)
CHAPTERVI.DISCUSSION VI-1
A. MODELSTRENGTHS VI-1
1. PredictiveAccuracy VI-1
2. ClinicalAcceptanceandCogency Vi-2
3. Gaming VI-2
4. IVlonitoring VI-3
B. MODELLIMITATIONS VI-4
1. DiscretionaryAdmissions VI-4
2. NoncontinuousEnrolleesandDecedants VI-4
C. SUGGESTEDFURTHERRESEARCH VI-5
1. EnhanceHospitalDominantMarker VI-5
2. IntegrateModelswithReinsuranceandCan/e-OutPlans VI-5
3. HCFADemonstrations VI-6
REFERENCES
LISTOFTABLES
LISTOFAPPENDICES
LISTOFTABLES
ES-1 JHURiskAdjusterAnnualScores:1992 ES-7
ES-2 DistributionofPredictiveRatiosfor100RandomSamplesofDifferentSizes ES-10
ES-3 AdjustedRSquareStatisticsandPredictiveRatiocbyYearOne(1991)
DiagnosticCategories ES-14
11-1. ComparisonofDevelopmentandEvaluationData 11-2
111-1. ADG-MDCRiskAdjusterModel 1-3
III-2. ADG-HosdomRiskAdjusterModel 1-5
III-3. Demographic("AAPCC")Model 1-5
III-4. DeterminingCapitationRatesforSevenHealthPlanEnrollees 1-7
111-5. PercentofVariationExplainedin1992TotalIndividualExpenditures, 1-9
IV-1 DistributionofPredictiveRatiosforRepeatedRandomSamples IV-5
IV-2 DistributionofPredictiveRatiosforRepeatedRandomSamplesof
50,000withReinsurance IV-10
IV-3 PredictiveRatiosandAdjustedRSquareStatisticsbyAge/GenderandRace IV-12
IV-4 PredictiveRatiosandAdjustedRSquareStatisticsbyUseofMedicalSen/ices lV-17
IV-5 PredictiveRatiosandAdjustedRSquareStatisticsbyClinicalCategoriesin1991 IV-19
IV-6 PredictiveRatiosandAdjustedRSquareStatisticsbyExpenditureGroupsin1991 IV-20
IV-7 PredictiveRatiosandAdjustedRSquareStatisticsbyRegion IV-23
V-1 PossibleNationalAnnualPaymentsAmountsin1992and1996 V-3
96FM0007 iv TheLewinGroup
....
LISTOFAPPENDICES
1-1. OriginalADGs
1-2. OriginalACGs
1-3. OriginalPACScomponents
11-1 AssessmentofCompletenessofAmbulaton;&InpatientDiagnosticCodesforUse
inJHURiskAdjustmentModel
11-2. DataFileDevelopment
11-3. PhysicianPaymentEstimationMethod
II-4. DecedentAdjustmentMethod
11-5. PercentofVariationinSubtotalExpendituresExplained
11-6. AnalyticalIssuesinModelDevelopment
11-7. ExampleofSeveralRejectedModels
11-8. PercentofVariationinExpendituresExplainedbyCell-BasedACGsModels
11-9. ICD-9-CMDiagnosisCodetoADGMaps
11-10. AltemativeDiagnosticSourcesandPercentofTotalVariationExplained
11-11 HospitalDominant(Hosdom)ICD-9-CMCodes
III-1 EstimatesofModelsUsingTruncatedData
III-2. RetrospectiveResultsofModels
IIVV--12. PIrCeDd-ic9t-iCvMeRCaotdieossfaonrdSeDrivsiecaeseanGdroEuxppsenUdsiteudreinGPrroeudipcstiivne1R9a9t2ioEvaluation
96FM0007 V TfiBLewifiGroup
