GacSanit.2019; :283–288
Special
article
Assessing
progress
of
the
Pan
American
Health
Organization’s
Policy
Research
for
Health
in
member
states
Claudia
Frankfurter
a,b,∗,
Jimmy
Le
c,
Luis
Gabriel
Cuervo
aaPanAmericanHealthOrganization/WorldHealthOrganization,Washington,DC,USA bFacultyofMedicine,UniversityofToronto,Toronto,Canada
cJohnsHopkinsBloombergSchoolofPublicHealth,Baltimore,MD,USA
a
r
t
i
c
l
e
i
n
f
o
Articlehistory: Received24August2017 Accepted5April2018
Availableonline12November2018
Keywords:
PanAmericanHealthOrganization Healthpolicy
Evaluationstudies
a
b
s
t
r
a
c
t
Theimprovementofhealthinthetwenty-firstcenturyisinextricablylinkedtoresearchforhealth.In responsetogrowinginternationalappealtoaddressregionalhealthneeds,thePanAmericanHealth Orga-nization(PAHO)anditsMemberStatesapprovedthePolicyonResearchforHealth(CD49/10)in2009. Thisdocumentrepresentstheflagshipregionalpolicyonresearchforhealthandoutlineshowhealth systemsandservicesintheregioncanbestrengthenedthroughresearch.Ithasbeenimplementedby thetwocomponentsofPAHO—theMemberStatesandthePanAmericanSanitaryBureau.Thepolicy containedaspecificdirectivemandatingPAHOtoreportonitsimplementation,developmentof subse-quentstrategies,andactionplanstargetingitsgoverningbodies.TheAmericasarethefirstWorldHealth Organization(WHO)regiontoissuearegionalPolicyonResearchforHealth,whichwasharmonizedwith WHO’sStrategyonResearchforHealth,approvedin2010.Attendingtotherecommendationsissuedby PAHO’sAdvisoryCommitteeonHealthResearchandWHO’sAdvisoryCommitteeonHealthResearch,the PAHODepartmentofKnowledgeManagement,BioethicsandResearchsetouttoadvancetheassessment oftheimplementationofthePolicyonResearchforHealththroughthecreationofamonitoringand eval-uationScorecard.IndicatorsrelevanttothePolicyonResearchforHealthobjectivesweremappedfrom theCompendiumofImpactandOutcomeIndicators,withnewindicatorscreated.Apracticalframework basedonavailableindicatordatawasproposedtogenerateabaselinepolicyassessmentandincorporate ameansofincrementallyenhancingthemeasurements.Inthiscasestudy,weoutlinetheiterationsof thePAHOPolicyonResearchforHealthScorecard,aswellasthelessonslearnedthroughoutthe devel-opmentprocessthatmaybeavaluableguideforhealthresearchentitiesmonitoringandevaluatingthe progressoftheirownpolicies.
©2018PublishedbyElsevierEspa ˜na,S.L.U.onbehalfofSESPAS.Thisisanopenaccessarticleunder theCCBY-NC-NDlicense(http://creativecommons.org/licenses/by-nc-nd/4.0/).
Evaluación
del
progreso
de
la
Política
sobre
Investigación
Sanitaria
de
la
Organización
Panamericana
de
la
Salud
en
sus
Estados
miembros
Palabrasclave:
OrganizaciónPanamericanadelaSalud Políticasanitaria
Estudiosdeevaluación
r
e
s
u
m
e
n
LamejoradelasaludenelsigloXXIestáinextricablementeligadaalainvestigaciónsanitaria.Enrespuesta alallamadainternacionalcreciente,decaraaabordarlasnecesidadessanitariasregionales,la Orga-nizaciónPanamericanadelaSalud(OPS)ysusEstadosmiembrosaprobaronen2009laPolíticasobre InvestigaciónSanitaria(CD49/10).Estedocumentoconstituyelapolíticaregionalinsigniasobre inves-tigaciónsanitaria,ydestacacómopuedenreforzarselossistemasyserviciossanitariosenlaregióna travésdelainvestigación.HasidoimplementadoporpartedelosdoscomponentesdelaOPS:los Esta-dosmiembrosylaAgenciaSanitariaPanamericana.Lapolíticaconteníaunadirectivaespecífica,que encomendabaalaOPSlarealizacióndeuninformesobresuimplementación,desarrolloypolíticas subsiguientes,aligualquelosplanesdeaccióndirigidosasusórganosdirectivos.Elcontinente amer-icanoeslaprimeraregióndelaOrganizaciónMundialdelaSalud(OMS)queestableceunapolítica regionalsobreinvestigaciónsanitaria,armonizadaconlaEstrategiasobreinvestigaciónsanitariadela OMS,aprobadaen2010.AtendiendoalasrecomendacionesemitidasporelComitéAsesorsobre inves-tigaciónsanitariadelaOPSyelComitéAsesorsobreinvestigaciónsanitariadelaOMS,elDepartamento deGestióndelConocimiento,BioéticaeInvestigacióndelaOPSpropusoavanzarenlaevaluacióndela implementacióndelapolíticadeinvestigaciónsanitariaatravésdelacreacióndeuncuadrodemando desupervisiónyevaluación.Losindicadoresrelevantesdelosobjetivosdelapolíticasobreinvestigación sanitariafueronpareadosapartirdelosindicadoresdeCompendiodelImpactoyResultados,conlos nuevosindicadores.Sepropusounmarcoprácticobasadoenlosdatosindicadoresdisponibles,para generarunaevaluacióndelapolíticabasaleincorporarunmediodemejorarlasmedidasgradualmente.
∗ Correspondingauthor.
E-mailaddress:c.frankfurter@mail.utoronto.ca(C.Frankfurter).
https://doi.org/10.1016/j.gaceta.2018.04.015
0213-9111/©2018PublishedbyElsevierEspa ˜na,S.L.U.onbehalfofSESPAS.ThisisanopenaccessarticleundertheCCBY-NC-NDlicense(http://creativecommons.org/ licenses/by-nc-nd/4.0/).
Enesteestudiodecasodestacamoslasiteracionesdelcuadrodemandodelapolíticasobreinvestigación sanitariadelaOPS,asícomolasleccionesaprendidasalolargodelprocesodedesarrollo,quepodrían constituirunaguíavaliosaparalasentidadesdeinvestigaciónsanitariadecaraasupervisaryevaluarel progresodesuspropiaspolíticas.
©2018PublicadoporElsevierEspa ˜na,S.L.U.ennombredeSESPAS.Esteesunart´ıculoOpenAccessbajo lalicenciaCCBY-NC-ND(http://creativecommons.org/licenses/by-nc-nd/4.0/).
Background
“Researchforhealth”hasbeenacoreactivityofthePan Amer-icanHealthOrganization(PAHO)sinceitsestablishmentin1902. “Research”,definedasasystematic processofevidence genera-tioninresponsetoaspecifiedhypothesisorchallenge,iscritical tofulfillingPAHO’s role in improvinghealth and health equity intheAmericas.1,2“Forhealth”encapsulatesthephilosophythat
thegenerationofevidencedrivinghealth decision-makingisan inter-sectoral,multi-disciplinaryactivity.2,3High-qualityresearch
forhealthproducesinformationthatcanguidePAHOandits Mem-berStatesinmakingevidence-informeddecisionandpromoting healthequityintheAmericas.2
PAHO’sPolicyonResearchforHealth(Policy),adoptedbyits MemberStatesatthe49thDirectingCouncilin2009,represents aninternationalrecognitionthatresearchforhealthiscrucialin achievinghealthequityandrepresentsacorefunctionofthe Orga-nization.ThePolicyisaresultofiterativeconsultationswiththe WorldHealthOrganization(WHO),PAHOAdvisoryCommitteeon HealthResearch(ACHR),aswellasexternalandinternal stakehold-ers,includingWHO CollaboratingCenters,researchinstitutions, andconsumerrepresentativesandotherswhoparticipatedin pub-licconsultationsduringitsdevelopmentprocess.ThePolicyalso outlineshowhigh-qualityresearchcanstrengthenhealthsystems andservicesinthePAHOregionthroughsixobjectives:(1) promo-tionofresearchgeneration,(2)researchgovernancestrengthening, (3)humanresourcecompetencydevelopment,(4)establishment ofeffectiveandstrategicalliances,(5)adoptionofbestresearch practices,and(6)researchfindingdisseminationandutilization.4
ThePolicyfeaturesaspecificdirectivethatmandatesPAHOto reporttotheGoverningBodiesonitsimplementationalongside thedevelopmentofsubsequentstrategiesandactionplans.PAHO reinforcedthismandateatits44thACHRmeetingin2012byissuing arecommendationtodevelopaScorecardtoassess implementa-tionprogressofthepolicyacrosstheAmericas.5TheDepartmentof
KnowledgeManagement,BioethicsandResearchofPAHOtookit uponitselftodevisesuchatool.Inthiscommentary,wepresentour experiencesandlessonslearnedinthedevelopmentofthe“Policy onResearchforHealthScorecard”(Scorecard)asatoolto moni-tortheimplementationofPAHO’sPolicyonResearchforHealthCD 49/10.
DevelopmentoftheScorecard
A Scorecard leveraging PAHO’sexisting administrative eval-uation tools was designed by the Department of Knowledge Management,BioethicsandResearchofthePanAmericanHealth Organization.Thistoolwasderivedfromatotalofthreeiterations. Weinitiallyconductedasurveyof45managersofthePan Ameri-canSanitaryBureau(PASB),theSecretariatofPAHO,toassesshow managersperceiveandsupporttheimplementationofthePolicy. Inthissurvey,44percent(20/45)ofthePASBmanagers partici-patedandfromtheirresponses,weidentifiedtwopriorityareasof concern:adherencetoresearchregistrationatPASBandtheneed forhighercompliancewithstandardsandprocessesforguideline formulationandthereviewofresearchprotocols.Thefullresults
ofthissurveyarereportedelsewhere.6Giventhelimitedresponse
rateofthesurveyandtheneedtoassessPolicyprogressin Mem-berStates,weoptedtopursueamorecomprehensiveandobjective methodologyofassessingthePolicyimplementation.
Inthefirstiterationweidentifiedindicatorsforeachofthe rec-ommendedactionsinthePolicy,whichresultedinalonglistofover 60questionsthatwouldrequireofworkshopsineachcountrytobe completed.Recognizingthelimitedfeasibilityofthe implementa-tionofsuchworkshopsandtheneedforannualmaintenance,this optionwasshelved.
Intheseconditeration,wereviewedthePolicy,theReportofthe 45thACHRofthePanAmericanHealthOrganization,theStrategic PlanofthePanAmericanHealthOrganization2014-2019,the Com-pendiumofImpactandOutcomeIndicators,andtheWHOStrategy onResearch forHealth.Weidentified36 potentialindicatorsto assessadherenceandcompliancetothePolicy.4,7–10This
method-ologyrequiredworkshopsinMemberStatestoenableindicator measurementand wasdeemedhighlyresource-intensiveoutof keepingwithlong-termsustainability.
InpreparationforthethirdandfinaliterationoftheScorecard, weconductedascanofPAHO’s2014-2015CompendiumofImpact andOutcomeIndicators,aswellastheStrategicPlanofthePan AmericanHealthOrganization2014-2019andfoundnine indica-torsthatmetpolicyobjectivesthatwerebeingroutinelyassessed throughPAHO’sPerformanceMonitoringandAssessment semi-annualassessmentsandend-of-bienniumreviews.8,9Areviewof
otherrepositoriesofhealthpolicyevaluationindicatorssuchasthe EuropeanCoreHealthIndicatorswasconducted,butnofurther pol-icyindicatorsthat wererelevantforCD49/10wereidentified.11
Byextractingexistingindicatorsinthecurrentinternal monitor-ingsystemutilizedbyPAHO,theneedforanewdatacollection methodologywaseliminatedasproxyindicatorsinthebiannual periodic evaluations were already being conducted within the countries.We accessedthedatabehindtheindicatorsto popu-lateapreliminaryScorecardbasedontheexistingindicatorsand proposednovelindicatorstoaddressgapsinthecoverageofthe policy.Inleveragingroutinelycollecteddata,implementationof theScorecarddidthusnotrequiresubstantialinvestmentoftime orpersonnel.
Scorecardimplementation
ToassessoftheimplementationofPAHO’sPolicyonResearchfor Health,wedesignedaninitialScorecardcomprisingof29indicators (seeAppendix1online).AsummaryofthemajorScorecard com-ponentsisfeaturedinTable1.Wedividedtheindicatorsintosix maincategories(quality,governance,humanresources, partner-ships/alliances,standards,dissemination/impact),reflectingthesix coreobjectivesofthePolicy.Forexample,the“Governance” pol-icy objective is captured by indicators assessing countries and territorieswithfunctionalmechanismsforgovernanceofhealth research,implementationofregionalpolicyonResearchforHealth, andimplementationofregionalknowledgemanagementstrategy, amongstothers.
Ofthe45policysub-objectives,wewereabletofind29data sourceswithintheCompendiumofImpactandOutcomeIndicators
Table1
PAHOScorecardtoAssessProgressofPolicyCD49/10inMemberStates.
Objective1:Promotethegenerationofrelevant,ethical,high-qualityresearch
-NumberofregionalinitiativesoractionplansoftheInter-AmericanandUnitedNationssystemsdealingwithhealthanddevelopmentdesignedor implementedwithPAHOsupporttoadvancethehealthprioritiesoftheRegion
-Numberofcountriesandterritoriesimplementingtheregionalknowledgemanagementstrategyforhealth -Numberofcountriesandterritorieswithfunctionalmechanismsforgovernanceofhealthresearch
-Numberofstudiesconductedtoinformthedesignofneworimprovedinterventionsforreproductive,maternal,newborn,child,adolescent,andadulthealth -NumberofregionalinitiativesoractionplansoftheInter-AmericanandUnitedNationssystemsdealingwithhealthanddevelopmentdesignedor
implementedwithPAHOsupporttoadvancethehealthprioritiesoftheRegion
Objective2:Strengthenresearchgovernanceandpromotethedefinitionofresearchagendas
-Strengthenitscapacitytoguideandsuperviseitsresearchactivitiesandtoassesspastexperienceinsupportingthedevelopmentofnationalhealthresearch systems,beforegeneratingnewsolutionsforpresentandfuturechallenges
-AssistMemberStatesindevelopingappropriateresearchgovernancestructuresandinstrengtheningandmaintainingsustainedpublictrustand engagementwithresearch;
-SupportMemberStatesindevelopingstrategiesandactionplanstoimplementandarticulatepoliciesforresearchforhealthandinnovation,aswellasin developingstrategiesandactionplanstoimplementPAHO’sPolicyonResearchforHealth
-Fosteranappreciation,atthepoliticallevel,ofthevalueofresearchinacceleratinghealthimprovementsanddevelopment,andseekpoliticalcommitment tonationalhealthresearchaimingfortheallocationofatleast2%ofthebudgetsofministriesofhealthtoresearchandresearchcapacitystrengthening,in ordertoreachfundinglevelsproposedinWHAresolutions,expertcommittees,ministerialforumsandstrategicplans
-Helpgovernmentsincreasetheircapacitytoadapt,disseminate,anduseknowledgetranslationtoolsthatfacilitatelinkingresearchtohealthcarepolicyand practiceandtotheassessmentandselectionofhealthtechnologies,essentialmedicinesanddevices
-WorkwithMemberStatestodefineresearchagendasattheRegional,sub-regional,andnationallevels,fomentingcountryownershipofresearchagendas -WorkwithMemberStatestostrengthenresearchasapublichealthfunction,developingagendasforresearchforhealth,institutionalresearchcapacity,and
technicalassistance,andsupportresearchinpublichealthatthesubnationallevel
-AssistMemberStatesinmonitoringfundingflowsforresearchforhealthinrelationtotheneedsandexpendituresrequiredand,whennecessary,advocate forresourcestoberedirectedtopriorityareas,monitoringprogressinreachingrecommendedmilestones
Objective 3: Improve competencies of and support for human resources involved in research
-PromotethemainstreamingofhumanresourcesworkinginresearchforhealthandtheintegrationofglobalandRegionalpolicies,strategies,andplansof actionforhumanresourcesinhealth
-Strengthenthecapabilityofitsstafftousescientificknowledgeandsystematicreviewsoftheliteraturewhentheydeveloptechnicalcooperationand addressuncertaintiesinthefaceofinsufficientresearchevidence
-Workwithpartners,includingbutnotlimitedto,health,scienceandtechnology,education,development,andlegalsectors,andresearchinstitutions,to enrichthehealthsciencescurricula;improvecompetenciesinresearch,monitoring,andevaluation;andengageincapacitybuildingactivitiestoincrease healthprofessionals’capabilitytounderstandanduseresearchresultsandtoengageothersectorsthatinfluencehealthcare,healthsystems,andhealth governance
-AssistMemberStatestoevaluatetheircurrentandfuturehumanresourceneedstoconductresearchforhealth,tohelpthemdevelopnationalpoliciesand long-termplanstoeducateandretainthenecessarynumberofhealthresearcherswiththerequiredskillsandcapacities,andfindconstructiveapproaches thatengageexpatriateresearchers
-HelpMemberStatesaddress,throughappropriateresearchanddevelopmentofstrategicincentives,thefactorsthatdeterminemigrationandalienationof researcherstopromotethedevelopment,retentionandthrivingofproductiveresearchgroups
-CooperatewithMemberStatestopromotegenderequityinthecompositionofresearchgroupsandresearchmanagementstructures,andtodevelopways tosupportincreasingthenumberofresearchersfromunder-representedethnicgroups
-Supportthedevelopmentofthestructures,methodsanddirectivesthatpromoteandmaintainsystematicevidence-informedapproachesintheevaluation andselectionofhealthtechnologies
Objective4:Seekefficienciesandenhancedimpactandappropriationofresearchthrougheffectiveandstrategicalliances,collaboration,andthe buildingofpublictrustandengagementinresearch
-Engagetheprivatesectorwithanemphasisonachievinglong-termgoalsandcommitmentsandfosteringmulti-centercollaborations,innovation,andthe sharingofideasandappropriatetechnology
-FacilitaterelevantcollaborationwiththeUnitedNationssystem,theinter-Americansystem,civilsocietyorganizations,developmentagencies,andother stakeholders
-Workwithopinionleaders,strategicpartners,andgovernmentstomobilizesupportandresourcesforresearchforhealth -MakemoreefficientandeffectiveuseofitsownspecializedcentersandofWHOcollaborativecenters
-Facilitatecommunicationandcoordinationbetweenthepublichealthandtheindustrialsectorstoencouragethedevelopmentofnewproductsand proceduresthataddressrelevantpriorities
-Workincoordinationwiththeeducationsector,thescienceandtechnologysector,independentresearchcenters(non-profitandfor-profit),andnetworksin ordertohaveresearchgroupsinMemberStateshavecriticalskillsandsufficientnumberstodevelop,grow,regenerate,andachievesustainableprogress; -Promoteexchangeandcollaborationwithinandbetweencountriesandsub-regions,withtheparticipationofvariouscomplementarydisciplines -Promotetheidentificationandimplementationofstrategicapproachestoaddressthehealthdeterminantsefficientlyandeffectively,andfacilitatingan
equitableinvestmentofresourcescommensuratetothesusceptibilitytoimprovethehealthandwell-beingofpopulationsbyaddressingparticular determinants
Objective5:Fosterbestpracticesandenhancedstandardsforresearch
-PromotenormsandstandardsthatareinlinewithWHO’sStrategyonResearchforHealthandfostertheirimplementationandcompliancewithexisting researchstandards
-Advocateforresearchproposalstoincludeplansforthedissemination,translation,implementationofthenewknowledgetheymightgenerate -Promotecivilsociety’senhancedparticipationandownershipinresearchasatruepartnerinresearchforhealth,contributingtothedevelopmentof
researchpolicies,thedefinitionofresearchagendas,andthedevelopmentanduseofresearchforhealth
-HelpMemberStatescreateoraccessresearchforhealthinventoriesandregistersthatarecomparableandintegratedwithWHO’sInternationalClinical TrialsRegistryPlatformprimaryregisters,andadoptstandardidentifiersanddatasetcollectionsthatcontributetointernationalregistrationeffortsand internationalethicsandpublicationsstandards
-Promoteaccessanduseandfurtherdevelopmentofhelpfulorganizedcollectionsandregistriesofresearchsynthesis,includingsystematicreviews, evidencesummaries,andpolicybriefs
-SupportMemberStatesinthedevelopmentofstrategiesandactionplans,regulations,andincentivestostrengthenadherencetoresearchregistration -Workincollaborationwithrelevantgovernmentsectors,theUnitedNationssystem,theinter-Americansystem,centersofexcellence,collaboratingcenters,
civilsocietyorganizations,andotherstakeholderstopromotetheethicalregulationofresearchforhealthinhumansandthestrengtheningofethical reviewcommitteesandcommissionsinMemberStates
-HelpMemberStatesbuildeffectiveandefficienttoolsfordeterminingandassessingtheextenttowhichtheresearchtheyconductadherestointernational goodpracticestandards,includingethics,safety,andresearchmanagementstandards
-PromotethedevelopmentofvalidatedindicatorstoassessandmonitortheeffectsofinvestmentinresearchandscientificproductionintheAmericas(9) andthealignmentbetweenresearchactivitiesandresearchpriorities
Table1(Continued)
Objective6:PromotethedisseminationandutilizationofresearchFindings
-Continuetoencourageopenaccesstoscientificliteratureandtofosternovelapproachestocopyrightandintellectualpropertythatwillallowknowledge essentialforhealth,equity,anddevelopmenttobesharedandmadewidelyavailable
-Continuetoparticipateintheinternationaldebateonhowlegalframeworksforintellectualropertyaffectresearchforhealth,especiallytheimpacton developmentandequitableaccesstothebenefitsresearch
-Promoteknowledgesharingamongresearchers,policymakers,andotherusersandfosterthedevelopmentandevaluationofnewknowledgetranslation initiativesandtoolsintheRegion
-Seektheempowermentandparticipationofcivilsocietyorganizationsinsettingpriorities,generatingknowledge,andharnessingresearchevidence -publishrelevantresearchfindings,recommendations,andguidelinesthatemergefromresearchforhealthinformatsthataremostappropriateforthe
targetaudience
-Workwiththemediatoimprovepublicunderstandingofthebenefitsofresearchforhealthandtoimprovescientificliteracyofpolicymakers,health providers,andthepublic
-Workincooperationwithitsspecializedcenters,suchastheLatin-AmericanandCaribbeanCenteronHealthSciencesInformation(BIREME),toindexand organizeresearchevidenceinhelpfulwaysandpromotetheVirtualHealthLibrarymodelandtheindexing,organization,access,andsharingofrelevant healthinformation
-Promoteaccesstoanduseofresearchevidencesummariesthatintegrateresultsthroughvalidmethods,thusfacilitatingabetterunderstandingofthe relevanceandeffectsofinterventionsandpromotingefficienciesinthesearchandanalysisofqualitativeand/orquantitativescientificresearch
thatmappedontoourindicators.Somepolicysub-objectivesdid nothaveanyapplicableindicatorsavailable.Atotalofthreenew indicators/outputswerethereforeproposedtobeaddedintothe CompendiumtofulfillthecoreofthePolicyassessmentinfuture assessments: delivery of research reports by the PASB, budget allocationto researchfor health, andmonitoring ofscholars in health research. Our approach was informed by scanning the literaturefor implementation of other health researchpolicies. The World Health Organization’sMonitoring and Evaluation of Mental Health Policies and Plan advisedthat a combination of quantitativeandqualitativedatabeemployedandrecommended useofdatathatmaybealreadyavailable.12
The Research Promotion and Development team of PAHO’s OfficeofKnowledgeManagement,Bioethics,andResearch,piloted theScorecardin2015.Thepurposeofthispilottestwastocollect datatoevaluateeachoftheproposedindicatorsbasedonthree parameters:feasibility,relevance,andconvenience.Weevaluated feasibilityintermsofhoweasilyaccessibledatawascollectedfor specificindicators.Relevancewasassessedbasedonwhetherthe objectivesbeingcapturedwerereflectiveofthekeycomponents ofthePolicy.Conveniencewasdeterminedfromtheeaseandtime requiredfordataaccess.
WepopulatedtheScorecardusingpreviouslygathered informa-tionfromtheinternalPAHOmonitoringsystemfrom2014-2015. Wecalculatedadherencetoeachobjectivebasedonwhetherdata wasreportedornot.AMemberStatewaslabelledtobeadherent toanobjectiveifatleast50percentofsub-objectivesweremet. Basedonourinitialpopulationoftheframework, wegenerated percentagesofcompletedreportingpermember,aswellasaheat mapoutliningadherencerates.TheresultsoftheScorecardwill beusedbyPAHOinitially,andmaybesubsequentlypublishedin thefuturefollowingcollectionoffeedbackandassociated Score-card revisions. Once institutionalized and operationalized, data collectedbienniallybasedontheselectedindicatorsmayprovide relevantstakeholderswithanassessmentofPolicy implementa-tionprogress,aswellasthefactorshamperingprogressinatimely systemicfashionpost-policyadoption.
AccordingtothedatabasemaintainedonHealthResearchWeb, activepoliciesonresearchforhealthhavebeenlistedin16 coun-triesandaregion:AntiguaandBarbuda(2016),Bahamas(2010), Belize (2014), Bermuda (2014), Brazil (2015), Cayman Islands (2012), Chile (2011), Dominican Republic (2014), El Salvador (2017),Jamaica (2015),Mexico(2013),Panama (2016),Paraguay (2016,StatePolicy),SanKittsandNevis(2011),Suriname(2011), UnitedStatesofAmerica(2013)and18CARICOMcountries.Active agendasonresearchhavebeenlistedby8countriesandtwo sub-regions:Chile(2009),Guatemala(2014),Guyana(2013),Honduras (2015),Mexico(2017)Panama(2016),Peru(2016),andUruguay (2017).TheCARICOMcountriesshareaCaribbeanResearchAgenda
(2011),andtheCOMISCACountriessharetheHealthPlanfor Cen-tralAmerica andtheDominican Republic,and havereactivated theirCommissiononHealthResearchin2017.13Engagementand
commitmentinhealthresearchisevidentlytakingplacearoundthe regionoftheAmericas,anditisthusimportanttocapturethe activ-itytakingplaceandmonitoritsdevelopmentinconcordancewith PolicyonResearchforHealth.TowardstheendoftheScorecard developmentprocess,theWHOreleasedaseriesofindicatorsand datadevelopedinlimitedconsultationwithregionalofficesaspart oftheGlobalObservatoryonHealthResearchandDevelopment.14
Theseindicators arevaluable additionstotheliterature,and in collaborationwiththeWHO,PAHOisexploringwaystopromote theiruseandappropriationbyMemberStatesintotheScorecard, inkeepingwiththeproposalofthe46thSessionoftheACHRtouse suchcomplementarydataforpolicyevaluations.15
Limitations
TherewerelimitationstothedevelopmentofthisScorecard. Firstly,thereisaneedforstandardizationinreportingmannerisms acrossMemberStates.Datacollectionmayvaryamongst Mem-bers,but weproceededwiththis approachdue toitsabilityto leverageexistingsystemsanddata.Furthermore,somepolicy sub-objectivesarechallengingtoassessgiventhedifficultyofattaining certaindata.We have begun toremedythis byproposingnew indicatorstoaddtotheCompendiumasaninitialstep,withthe intentionof eventually buildingmore indicators into the Com-pendiuminordertobeabletosystematicallyandroutinelycollect datafrommembers.Lastly,giventhelackofavailableindicators forallsub-objectives,someareasofpolicyCD49/10arenotableto beassessed.Wenonethelessbelievethedevelopmentofthis pre-liminaryScorecardisanimportantstepintheassessmentofthe Policythatwillpavethepathforwardforfuturesystematicand comprehensivepolicyevaluationandassessment.
Recommendations
Overall,PAHO,MemberStates,ministriesof health,and aca-demiccenterswilllikelybenefitfromrenewingtheircommitment totherealizationofthePolicyonResearchforHealthand reviv-ingtheireffortstowardsbringingaboutprogressintheAmericas. Lookingtothefuture,atwo-foldapproachwillbeundertakenby theACHRSecretariatandentitiesinvolvedinpromotingresearch forhealthatPAHO.IncreasedawarenessonthePolicyonHealth forResearchanditsobjectiveswillbepromoted.Itisverylikely thatinaveryfast-pacedpoliticalenvironmentsubjecttopersonnel turnover,thepolicyhasfallenofftheradarofmanyPAHO lead-ersandMemberStatedelegates.Havingacontinuedassessmentof progresswiththePolicywillprovideactionabledatatoguideits
implementation.TheACHRSecretariatatPAHOwillsendoutan updateonthepolicytoallgoverningbodies,aswellassuggestions onhowtoadvanceandupdatethepolicyasthelandscapeevolves. ThisSecretariatwillalsoaskcountryofficestocontactactorscentral totherealizationofpolicyobjectivessuchasacademicinstitutional leaders,healthdepartmentpersonnel,aswellasscienceand tech-nologydepartmentstoaskthemtobolstertheirinitiativestowards strengtheningresearchforhealth.
Secondly,theentitiesresponsibleforpromotinganddeveloping researchwithinthe PanAmerican SanitaryBureaushould sup-portcountryofficestoprovidedetailsontheindicatorswithinthe Scorecard.IncreasingtherobustnessoftheScorecardwillinclude validationofthetoolwithprimarydatafromthememberstates. Effortswillbeprincipallytargetedtowardsattainingfull report-ingacrossallmembers,andtowardsfine-tuningtheinstruments. InitiativesfulfillingobjectivesofthePolicymayverywellbe under-wayinmanycountries,butwithoutthecomprehensiveandtimely reportingofcountryoffices,PAHOwillnotbeabletoassesswhat areasunderitsjurisdictionitshouldcontinuetosupportandwhere significantattentionandresourcesoughttobedirectedtofulfillthe policymandates.Reinforcementofreportingmandates, consulta-tionwiththeGoverningBodiesoverevolvingindicators,aswell asdedicationofappropriatehumanresourcestowardsthe com-pletionofthemonitoringandevaluationsystemislikelytohelp improvethenetquantityandqualityofindicatorreporting.Uptake fromtheorganizationalleadershipwillberequiredtochampion theseendeavors.
InordertoinstitutionalizetheScorecard,supportfromPAHO’s executiveadministrationwillbesoughtout.Wewillcontinueto refinetheexistingindicatorsandprogressivelyproposethe intro-ductionofnewindicatorsthatwillmaketheassessmentofthe policyacorporateproductbuiltaroundtheexisting administra-tivetoolsandsystematicacrossallrelevantentities.TheScorecard willbecompletedonabiennialbasis,withmilestonesdocumented onabiannualschedule,usingdataprovidedbythemembers.The ACHRwillthenprovidehigh-levelcommentaryontheresultseach timethescorecardfilledout.TheScorecardwillbepopulatedby professionalsassessingthePolicyimplementationwithinthe Sec-retariatforPASBandtheACHR,andthensubsequentlycirculated forreview.FeedbackfromtheACHRmemberswillbeaggregated andthenre-directedtotherespectivemembersworkingin coor-dination with theunit of Program Budget and of Country and Sub-RegionalCoordination.
Itisimportanttoseekmechanismstopromoteharmonization and coordinationamong WHOregions (e.g.global andregional AdvisoryCommitteesonHealthResearch)thatidentifystrategies seekingtheappropriationandparticipationofMemberStatesand WHOregionalofficesintheWHOGlobalObservatoryonResearch forHealth.
Conclusions
ThePAHODepartmentofKnowledge Management,Bioethics andResearchdesignedapracticalwaytouseavailableinstitutional indicatorsasameansofassessingtheimplementationofthe Pol-icyonResearchforHealth(CD49/10).Ourexperiencesarelikely toberelevanttootherWHOregionalofficesandresearch enti-tiesthat aredealingwithhealthandengagedin measuringthe progressoftheirownmandates,policiesorstrategiesandlikely toencountersimilarjourneysinstrivingforfeasiblepolicy mon-itoringandevaluation.Thetaskofmonitoringandevaluatingthe implementationofapolicyisacomplexphenomenonyetcriticalto thepolicyprocess.Thereisadelicatebalancebetween comprehen-siveness,accuracy,andfeasibilitythroughoutthepolicyevaluation processthatistobestrivenfor.
Editorincharge
ClaraBermúdez-Tamayo.
Authorshipcontributions
L.G.Cuervo,C. FrankfurterandJ.Lecontributedtothe con-ceptionanddesign ofthework.C. FrankfurterandL.G.Cuervo contributedtothedatacollectionandtheanalysisand interpreta-tionofthedataandthewritingofthearticle.C.Frankfurter,J.Le andL.G.Cuervocontributedtothecriticalreviewwithimportant intellectualcontributions.Allauthorsapprovedthefinalversion foritspublicationandagreetobeaccountableforallaspectsofthe workinensuringthatquestionsrelatedtotheaccuracyorintegrity ofanypartoftheworkareappropriatelyinvestigatedandresolved.
Funding
None.
Conflictsofinterest
None.Anauthor(LGC)isastaffmemberofthePanAmerican HealthOrganization.Theauthoraloneisresponsiblefortheviews expressedinthispublication,andtheydonotnecessarilyrepresent thedecisionsorpoliciesofthePanAmericanHealthOrganization.
Acknowledgements
WewouldliketothankmembersoftheAdvisoryCommitteeon HealthResearchwhoinvaluablyparticipatedintheconsultation processontheScorecard.WewouldliketothankDr.JohnLavis forhisguidanceandsupervision,Ms.RachelHutchinsonforher contributionstotheearlierversionsoftheScorecard,aswellasMr. RonyMazaatthePanAmericanHealthOrganizationforhissupport withtheCompendiumofImpactandOutcomeIndicators.
AppendixA. Supplementarydata
Supplementarydataassociatedwiththisarticlecanbefound,in theonlineversion,atdoi:10.1016/j.gaceta.2018.04.015.
References
1.CommissiononHealthResearchDevelopment.HealthResearch.NewYork,NY: OxfordUniversityPress;1990.
2.PanAmerican HealthOrganization49th DirectingCouncil 61stSessionof the Regional Committee of WHO for the Americas. Policy on research for health: document CD49/10. Washington, DC; 2009. Available from: http://www2.paho.org/hq/dmdocuments/2009/CD49-10-e.pdf
3.WorldHealthOrganization.TheBamakocalltoaction:researchforhealth. World Health Organization. Geneva, Switzerland; 2008. Available from: http://www.who.int/rpc/news/BAMAKOCALLTOACTIONFinalNov24.pdf 4.Pan American Health Organization. Resolution CD49/10.
Washing-ton, DC; 2009. Available from: http://new.paho.org/hq/pdmdocuments/ 2009/CD49-R10-Eng.pdf
5.Advisory Committee onHealth Research. Advisory Committee on Health Research44thMeeting.Barcelona,Spain;2011.
6.CuervoL,SoellingS,HutchinsonR.Policyonresearchforhealthimplementation survey(CD49/10).Washington,DC;2014.
7.AdvisoryCommitteeonHealthResearch.45thsessionofPAHO’sAdvisory Com-mitteeonHealthResearch.Washington,DC;2012.
8.Organization.PAH.StrategicplanofthePanAmericanHealthOrganization 2014-2019.Washington,DC;2014.
9.Pan American Health Organization. Compendium of impact & outcome indicators. Washington, DC; 2014. Available from: http://www.paho.org/ hq/index.php?option=com docman&tas=docdownload&gid=23129&Itemid =270&lang=en.
10.World Health Organization. The WHO Strategy on Research for Health. Geneva, Switzerland; 2012. Available from: http://www.who.int/phi/ WHOStrategyonresearchforhealth.pdf
11.PublicHealthEvaluationandImpactAssessmentConsortium.Evaluationofthe useandimpactoftheEuropeancommunityhealthindicators.2013.Available from:http://ec.europa.eu/health/indicators/docs/echireportv20131031.pdf. 12.WorldHealthOrganization.Monitoringandevaluationofmentalhealth
poli-cies and plan. Geneva; 2007. Available from: http://www.who.int/mental health/policy/services/14-monitoringevaluationHKprinter.pdf
13.PanAmerican HealthOrganization.HealthResearch Web.Availablefrom: https://www.healthresearchweb.org
14.WorldHealthOrganization.GlobalHealthObservatoryData.2018.Available from:http://www.who.int/gho/en/
15.PanAmericanHealthOrganization.46thSessionoftheAdvisoryCommitteeon HealthResearch.Washington,DC;2017.