A POMDP approach to age-dependent primary screening policies for cervical cancer in Colombia

A POMDP Approach to Age-dependent Primary

Screening Policies for Cervical Cancer in Colombia

Martha Isabel Namen Leon

Universidad de los Andes, [email protected]

Raha Akhavan Tabatabaei

Universidad de los Andes, [email protected]

Esma Gel

Arizona State University, [email protected]

Cervical cancer is the second leading cause of cancer-related deaths in Colombian women. Different screening policies are practiced in order to prevent this disease in the medical community, including the common cytology. One of the newest and most popular is the HPV-DNA test as primary screening, since it performs better than cytology regarding potential risks of low sensitivity and specificity. However, this test is not common in Colombia due to its high cost and inaccessibility in many communities. Our goal is to define since what age and with what frequency should a patient be tested with either HPV-DNA test or cytology. Medical guidelines have already been established to address this problem. However, they do not consider the imperfect information of these tests. We formulate a finite-horizon Partially Observable Markov Decision Process (POMDP) model that incorporates primary screening (cytology or HPV-DNA test) as a method of detection. Age-specific non-observable disease progression and regression are also included, and type I and II errors of these tests are considered as the proportion of false positive and false negative patients diagnosed annually. We optimally solve this POMDP model using Monahan’s algorithm with Eagle’s reduction. Our results show that our proposed screening schedules outperform the existing guidelines with respect to the total expected quality-adjusted life years (QALYs).

Key words: partially observable Markov decision processes, dynamic programming, decision analysis, medical decision making, operations research applications in healthcare, cancer screening, cervical cancer, cytology, HPV-DNA test, primary screening.

1. Introduction

Cervical cancer is the second most common cancer in women worldwide, with an estimated 530,000 new cases diagnosed and 270,000 deaths per year. More than 85% of these deaths are in low and middle-income countries (WHO 2013). In Colombia, there are approximately nine daily deaths, 6,800 newly detected cases and 3,300 deaths per year (Profamilia 2014). Although Colombia is one of the countries with the highest mortality rate of cervical cancer in the region, its incidence and mortality rates have decreased over the last 40 years. By improving socio-economic conditions,

executing effective screening and treatment programs, and training the population with preventive procedures, the death rate has decreased from 14 deaths per 100,000 women in 1987 to 7,08 deaths per 100,000 in 2014, setting a goal of 5,5 deaths per 100,000 by 2021 (Min.Salud 2013).

Cervical cancer is defined as an uncontrollable growth of cells that occurs in the female reproductive system, specifically in the lower part of the uterus (womb). Squamous and Glandular cells are the two type of cells that cover the cervix. These cells do not suddenly turn into cancer. Instead, normal cells of the cervix first gradually develop pre-cancerous changes that may turn into cancer. This growth is almost always caused by the Human Papilloma Virus (HPV). There are more than 100 strands of HPV, classified into two categories: low risk HPVs and high risk HPVs. Both low risk and high risk strands cause changes in the cervical cells, but low risk strands only cause non-cancerous skin warts. Terms used to describe these pre-cancerous changes include

cervical intraepithelial neoplasia (CIN), squamous intraepithelial lesion (SIL), anddysplasia (ACS 2014). The high risk strands, also called oncogenic HPVs are: HPV 16 and 18; these two strands cause virtually 70% of all the cervical cancer cases (Walboomers et al. 1999).

Any non-infected woman can acquire an HPV infection through sexual transmission. These infections can spontaneously clear or progress to low-grade lesions. Low-grade lesions can clear, progress to high-grade lesions or regress to benign abnormal cells. A high-grade lesion is considered in-situ cancer that can progress to invasive cancer (NCI 2014).

“Cervical cancer screening is the systematic application of a test to identify cervical abnormal-ities in an asymptomatic population” (WHO 2013). There are primary and secondary tests for screening. Cytology and HPV-DNA tests are considered for primary screening. Cytology is the collection of cells to detect potentially pre-cancerous and cancerous processes in the endocervical canal of the female reproductive system. HPV-DNA test is an inspection of the genetic material (DNA) of HPV on a sample of cells collected from the cervix (NCI 2014). The importance of doing an effective primary screening lies in the fact that pre-cancerous cervical lesions progress very slowly, can also regress and are generally asymptomatic. Hence, cervical cancer can be prevented with an appropriate screening.

The most common primary screening test in Colombia is the cytology, which classifies patients as “Normal” with a non-positive result or “Abnormal” with a positive result, depending on the cause of the abnormality. However, cytology is not a perfect test and has a 48% risk of false-negative results and 23.4% of false-positive results (Munoz et al. 2004). The risk of producing

false-negatives decreases to 8% and of false positives to 13%, when an HPV-DNA test is performed (Gamboa et al. 2008). However, the price of performing this test is twice the price of cytology in Colombia (Min.Salud 2013), between $7 to $15 US Dollars for cytology, versus $16 to $26 US Dollars for HPV-DNA test approximately (Profamilia 2014), (UAE-CRES 2011), (UAE-CRES 2011a).

When performing a cytology, the possible outcomes are: negative, atypical squamous cells of undetermined significance (ASCUS), low-grade squamous intraepithelial lesion (LSIL), high-grade squamous intraepithelial lesion (HSIL), squamous cell carcinoma (SCC) and adenocarcinoma (AC), where negative (C−) and ASCUS (C1) are considerednon-positive results, and LSIL (C2), HSIL (C3), SCC (C4) and AC (C5) are consideredpositive results. . As it can be seen in Figure 1, according to the current guidelines if C−is observed, the following cytology should be performed in 3 years. IfC1 is observed another cytology should be performed in 1 year (ASCCP 2013).

When an abnormality is found with cytology, a positive outcome is observed, and a secondary screening test such as colposcopy or biopsy is performed, where inspection and sampling of cervical tissue for problematic areas.

In case of getting a negative result with the secondary screening test, a cotesting (cytology and HPV-DNA test) is recommended after one year. Otherwise, the treatment for the identified cause (conization, in-situ treatment, invasive treatment) should be performed.

Figure 2 illustrates the case of performing an HPV-DNA test. The observations can be negative (H−) or positive (H+) for HPV strands 16 or 18. If H−is observed the test should be repeated in 5 years. Otherwise, a secondary screening test is performed, and another HPV-DNA should be repeated in one and a half years.

Figure 2 HPV-DNA test as primary screening test

Several models for cervical cancer prevention and improvement of screening policies have been proposed in the Operations Research (OR) and Medical Decision Making (MDM) literature. Ley-Chavez (2012) makes an extensive literature review focusing on Markov models, simulation, influence diagrams and other decision analysis tools. Her review is based on screening and treatment decisions of various diseases, and applications of OR to cervical cancer prevention. Goldie et al. (1999) use a Markov model to evaluate the cost-effectiveness of several cervical cancer screening strategies in HIV-infected women. Myers et al. (2000) also construct a Markov model that simulates the natural history of HPV and cervical cancer in a hypothetical cohort of women. Kim et al. (2002) propose a cost-effectiveness analysis of different ASCUS triage strategies. Eggington et al. (2006) propose a Markov model for cost-effectiveness analysis of different policies of referral to colposcopy. Kulasingam et al. (2006) extend the work in (Myers et al. 2000) and evaluate the cost-effectiveness of extending cervical cancer screening intervals among women with prior normal cytology results. Kulasingam et al. (2011) present a decision analysis for the U.S. Preventive Services Task Force to determine how many colposcopies per life-year gained are associated with each of the different ages for beginning screening for cervical cancer and how many HPV DNA tests in conjunction with cytology are used. Goldie et al. (2004) propose

a simulation model for the natural history of HPV infection and cervical cancer to evaluate the cost-effectiveness of using the HPV-DNA test as primary screening in combination with cytology for women over 30. McLay et al. (2010) present a simulation-optimization model to determine at which ages screening should be performed. However, to the best of our knowledge, no previous model has been proposed to approach cervical cancer prevention, including imperfect test results. The purpose of this study is to define since what age and with what frequency should a patient be tested, given the inaccessibility and relatively high cost of primary screening tests in Colombia. In order to address this need, we formulate a partially observable Markov decision process (POMDP) model that incorporates the two primary screening tests, age-specific non-observable disease progression and regression and test characteristics.

A POMDP is a generalization of a Markov decision process (MDP) that allows sequential decision making when the information regarding the true state of the system is incomplete. POMDPs capturing partial observation of the disease progression and imperfect test results, making them ideally suited to healthcare problems. However, the application of POMDPs to medical decision making has been limited to only a few studies, primarily due to their high computational complexity (Schaeffer et al. 2014).

This POMDP approach has been used before in the literature for breast cancer, (Ayer et al. 2012) and colorectal cancer (Safa et al. 2014). However, in cervical cancer only other methods such as cost-effectiveness analysis, Markov models, simulation and optimization have been implemented (Ley-Chavez 2012).

The remainder of this paper is organized as follows. In Section 2 we present our proposed POMDP model for this problem. In Section 3 we describe the solution algorithm implemented. In Section 4 we present the model inputs and we show and discuss some computational results. Finally, we summarize, conclude and suggest some future work in Section 5.

2. POMDP Model Formulation

We formulate a discrete-time, finite-horizon POMDP model with the objective of maximizing the total expected Quality-Adjusted Life Years (QALYs) of the patient under consideration. We assume that the decision maker is risk neutral. Like most of the existing cervical cancer screening guidelines (ACS 2014), (ASCCP 2013), (WHO 2013), (NCI 2014), we do not consider the financial costs associated with cervical cancer screening in this study, because we take the patient’s

perspective, and assume that these costs are covered by insurance plans (Rathore et al. 2000).

We start with the assumption that at the beginning of every year, a woman either undergoes a cytology, an HPV-DNA test or is recommended to wait for another year. This decision is made based on the woman’s current risk of cervical cancer, which we refer to as her belief state. Our model keeps the information gained from previous screening results by updating the belief state at every decision epoch based on the screening observations. If the woman is recommended to have a cytology or an HPV-DNA test and if the test turns out to be positive, it is followed by an assumed perfect secondary screening follow-up test (e.g., colposcopy, biopsy). This is a reasonable assumption because the literature reports that colposcopy and biopsy are reliable procedures with true positive rates very close to 1. If the result of the secondary test is also positive (i.e., the woman has cancer), we assume that the woman starts treatment and quits the decision process by moving to a post-cancer state with probability 1. Otherwise, the woman continues the decision process after a non-positive cytology or HPV-DNA test, a negative colposcopy or biopsy or a recommended Wait decision. The notation used in the model is as follows:

• Decision epochs: t= 30,31, ..., T; where,T <∞denotes the end of the planning horizon. We assume that screening decisions are made every year and define t as the age of the patient. In accordance with the customary application in the field of gynecology, we assume that screening decisions are made on an annual basis above the age of 30, since this is the earliest age among recommended starting ages for routine HPV-DNA screening. We end our decision horizon at age 77, consistent with the Colombian female life expectancy.

• Core state space: S. It is the set of the true health states the patient can be in at the beginning of year t. Any element of S is represented as st ∈ S =

{CIF, ICF, C1N, C2N, C3N, CV N, COT, C3T, CV T, DT H}, whereCIF represents infection and cancer free , ICF: infected but cancer free, C1N: cervical intraepithelial neoplasia 1 (mild dys-plasia, or abnormal cell growth), C2N: cervical intraepithelial neoplasia 2 (moderate dysplasia), C3N: cervical intraepithelial neoplasia 3 (severe dysplasia), also referred to as in-situ cancer,CV N: invasive cancer, COT: treatment forC2N (conization), C3T: treatment for in-situ cancer, CV T: treatment for invasive cancer, and DT H: death. For brevity, we match each state s∈S with an integer value, i∈ {1,2,3, ...,10} to represent the true health states in analytical expressions. Note that the decision maker only directly observes the true state when the patient is in states {7,8,9,10}. However, when the patient is in one of the states {1,2,3,4,5,6}, the decision maker does not know the true state of the patient with certainty.

We define SP O _{as a subset of S, which includes all the partially observable states, S}P O ₌

{1,2,3,4,5,6}. In addition, we define SP O1_{={1,2,3} as the subset of partially observable states}

that do not directly transition into treatment, and SP O2 _{={4,5,6} as the subset of partially}

observable states that may directly transition into treatment. Note that SP O_={SP O1 _∪SP O2_}.

Completely observable states will be defined overSO_={7,8,9,10}.

• Action taken at timet: at, i.e.,at∈A={W, C, H}, where W, C, H represent “Wait”,

“Cytol-ogy” and “HPV-DNA test”, respectively.

• Observation space: Θa, which includes observations seen upon taking action at. If at=W

no observation is obtained. If at=C, the decision maker can observe a normal cytology (C−),

atypical squamous cells of undetermined significance (C1), or a positive cytology (C+), where C+ ={C2, C3, C4, C5}, and C2, C3, C4, C5 stand for low-grade squamous intraepithelial lesions (LSIL), high-grade squamous intraepithelial lesions (HSIL), squamous cell carcinoma (SCC) and adenocarcinoma (AC), respectively. Then, ΘC = {C−, C1, C+}. If at = H, the decision

maker can observe negative (H−) or positive (H+) results for HPV strands 16 and 18. Then, ΘH={H−, H+}.

Figure 3 illustrates the core state transitions according to the screening results obtained. Tran-sitions marked with green arcs are independent of the screening results, since they represent tran-sitions to a death or cancer treatment state, and loops of the absorbing states. On the other hand, transitions marked with blue arcs occur when the results of the tests are non-positive. Hence, these are the natural disease progression and regression transitions. In addition, transitions marked with red arcs only occur when a positive result is observed in the states s∈SP O2 _{that may transition}

Figure 3 Health state transitions according to the screening results

• Observation Probabilities: Ka

t(o|s) represent the probability of observing ot∈Θa, given that

the patient is in state s∈SP O _{and action a}

t∈At is taken at time t. These probabilities refer

to type I and II errors of screening tests or false positive, false negative, true positive and true negative. In other words, these are likelihood probabilities of making a particular observation given the actual health state of the patient. Note that the cytology and HPV tests only provide partial information on the patient’s true health state, but help update the belief state with the additional information provided by the test results. The positive test results are typically followed by a perfect test, such as colposcopy and biopsy, as we have described above. To calculate observation probabilities, we use the test specificity (the proportion of cancer-free patients that are correctly identified as negative by the test) and sensitivity (the proportion of women with cancer that are correctly identified as positive by the test), obtained from the literature. Note that these can change with patient’s age and patient’s health state. Specifically, sens(s,a,o) refers to the sensitivity, i.e, proportion of women in state s∈SP O2 _{who are correctly identified as}

positive (ot=C+, H+) by the test (at=C, H), respectively. Similarly, spec(s,a,o) refers to the

specificity, proportion of women in state s∈SP O1 _{who are correctly identified as non-positive}

(ot=C−, C1, H−) by the test (at=C, H), respectively. The observation probabilities at time t

s C− C1 C+ H− H+

ICF(1) spec(1, C, C−) spec(1, C, C1) 1−spec(1, C, C−)−spec(1, C, C1) spec(1, H, H−) 1−spec(1, H, H−) CIF(2) spec(2, C, C−) spec(2, C, C1) 1−spec(2, C, C−)−spec(2, C, C1) spec(2, H, H−) 1−spec(2, H, H−) C1N(3) spec(3, C, C−) spec(3, C, C1) 1−spec(3, C, C−)−spec(3, C, C1) spec(3, H, H−) 1−spec(3, H, H−) C2N(4) 1−sens(4, C, C+)−sens(4, C, C1) sens(4, C, C1) sens(4, C, C+) 1−sens(4, H, H+) sens(4, H, H+) C3N(5) 1−sens(5, C, C+)−sens(5, C, C1) sens(5, C, C1) sens(5, C, C+) 1−sens(5, H, H+) sens(5, H, H+) C1N(6) 1−sens(6, C, C+)−sens(6, C, C1) sens(6, C, C1) sens(6, C, C+) 1−sens(6, H, H+) sens(6, H, H+)

Table 1 Observation probabilitiesKa t(o|s)

• Core state transition probabilities: Pt(a,o)(s0|s) represent the probability that the patient

will be in state s0_{∈S at time t+ 1, given that she was in state s∈S, took action a}

t∈A, and

observed o∈Θa at time t. When any of the performed tests (cytology or HPV-DNA test) are

non-positive (C-, C1, H-), or when no screening is performed, the patient does not undergo to secondary screening tests. Then, the core state transition probabilities refer to the natural disease progression or regression of the disease.

PW,∅_=PC,C− _=PC,C1 _=PH,H−₌

                               

(1 : CIF) (2 : ICF) (3 : CI1) (4 : CI2) (5 : CI3) (6 : CNV) (7 : COT) (8 : C3T) (9 : CVT) (10 : DTH)

(1 : CIF) p1₁₁ p1₁₂ 0 0 0 0 0 0 0 1−(p11+p12)

(2 : ICF) p1₂₁ p1₂₂ p₂₃1 p1₂₄ 0 0 0 0 0 1−(p1₂₁+p1₂₂+p1₂₃+p1₂₄) (3 : CI1) p1₃₁ p1₃₂ p1₃₃ p1₃₄ p1₃₅ 0 0 0 0 1−(p1₃₁+p1₃₂+p1₃₃+p1₃₄+p1₃₅) (4 : CI2) p1₄₁ p1₄₂ p1₄₃ p₄₄1 p1₄₅ p1₄₆ 0 0 0 1−(p1₄₁+p1₄₂+p1₄₃+p1₄₄+p1₄₅+p1₄₆) (5 : CI3) p1₅₁ p1₅₂ p1₅₃ p₅₄1 p1₅₅ p1₅₆ 0 0 0 1−(p1₅₁+p1₅₂p1₅₃+p1₅₄+p1₅₅+p1₅₆) (6 : CNV) 0 0 0 0 p1₆₅ p1₆₆ 0 0 0 1−(p1₆₅+p1₆₆)

(7 : COT) 0 0 0 0 0 0 p1₇₇ 0 0 1−p1₇₇

(8 : C3T) 0 0 0 0 0 0 0 p1₈₈ 0 1−p1₈₈

(9 : CVT) 0 0 0 0 0 0 0 0 p1₉₉ 1−p1₉₉

(10 : DCC) 0 0 0 0 0 0 0 0 0 1

                               

Matrix 1: Wait (no observations), nonpositive outcomes, no secondary screening tests:

When the result of the primary test (cytology or HPV-DNA test) is positive, a secondary screening test is performed and for any state s∈SP O2 _{immediate treatment is advised. Matrix 2 shows how}

states s∈SP O2 _{transition with probability 1 to the absorbing treatment states when a positive}

PC,C+_=PH,H+₌                                

(1 : CIF) (2 : ICF) (3 : CI1) (4 : CI2) (5 : CI3) (6 : CNV) (7 : COT) (8 : C3T) (9 : CVT) (10 : DCC) (1 : CIF) p2

11 p212 0 0 0 0 0 0 0 1−(p211+p212)

(2 : ICF) p2₂₁ p2₂₂ p₂₃2 p2₂₄ 0 0 0 0 0 1−(p2₂₁+p2₂₂+p2₂₃+p2₂₄) (3 : CI1) p2₃₁ p2₃₂ p2₃₃ p2₃₄ p2₃₅ 0 0 0 0 1−(p2₃₁+p2₃₂+p2₃₃+p2₃₄+p2₃₅)

(4 :CI2) 0 0 0 0 0 0 1 0 0 0

(5 :CI3) 0 0 0 0 0 0 0 1 0 0

(6 :CN V) 0 0 0 0 0 0 0 0 1 0

(7 :COT) 0 0 0 0 0 0 1 0 0 0

(8 :C3T) 0 0 0 0 0 0 0 1 0 0

(9 :CV T) 0 0 0 0 0 0 0 0 1 0

(10 :DCC) 0 0 0 0 0 0 0 0 0 1

                               

Matrix 2: Positive outcomes, secondary screening tests:

The probability of change in a health state in the subsequent interval is the same for women who have a non-positive or false-positive cytology or HPV-DNA test. That is Pt(W,∅)(s0|s)=P

(C,C−)

t (s0|s)=P

(C,C1)

t (s0|s)=P

(H,H−)

t (s0|s), ∀s, s0 ∈ S and

Pt(W,∅)(s0|s)=P

(C,C−)

t (s0|s)=P

(C,C1)

t (s0|s)=P

(H,H−)

t (s0|s)=P

(C,C+)

t (s0|s)=P

(H,H+)

t (s0|s), ∀s∈SP O

1 .

• Information space: Π(S) is the space of all probability distributions over the state space S. Any element of Π(S) is referred to as an information state, denoted by π, which consists of the occupation probabilities over the state space. That is, we let π(s) denote the probability of occupying states, when π= [π(s)].

• Belief space: B(SP O_{) is the set of all possible belief states, denoted by b}

t. It is a truncated

version ofπ that represents a probability distribution defined over the partially observable states. For example, if π = [0.25,0.25,0.25,0.25,0,0,0,0,0,0], then bt = [0.25,0.25,0.25,0.25,0,0]. The

belief state, bt∈B(SP O) serves as a sufficient statistic to represent the entire history of the

process, and indicates the degree to which the decision maker “believes” the true health state of the patient to be any one of the partially observable health states,s∈SP O_{. At any decision epoch,}

the belief state is updated using the new information that is obtained through the screening tests (Cassandra AR 1994).

• Updated belief state: τ[b, a, o] captures the information on how transitions occur among the belief states. We define τ[b, a, o] = [τ[b, a, o](s0)], where τ[b, a, o](s0) represents the probability of occupying state s0_∈SP O _{at time t+ 1, given that the decision maker’s belief about the patient’s}

health state was bt ∈B(SP O), action taken was at∈A, and o∈Θa was observed at time t.

observing o∈Θa.

If at=W, the updated belief state is the product of the probability that the patient will

transition at time t+ 1 to state s0_∈SP O _{given that at time tshe is in any possible state s∈S}P O_,

and actionW is taken. Then,

τ[b, a, o](s0) = X

s∈SP O bt(s)P

W,∅ t (s

0_{|s). (1)}

If at=C orat=H, the updated belief state is the conditional probabilityP(s0|s, b, a, o), which

represents the probability of being in state s0∈SP O _{at time t+ 1, given that at timet there is a}

belief bt∈B(SP O), action at∈Atis taken, o∈Θa is observed and a transition from states∈SP O

tos0_∈SP O _occurs.

The numerator in Equation 2 shows that the conditional probability is calculated by the product of the core state transition probability at time t+ 1 to state s0_∈SP O _{given that at time}

t the patient is in any possible state s∈SP O_{, action a}

t∈At is taken and o∈Θa is observed, the

likelihood of observing o∈Θa and the belief of being in that state at time t. The denominator in

Equation 2 is the product of the belief at timetand the likelihood probability of observing o∈Θa

over all states s∈SP O_.

τ[b, a, o](s0) =



   

   

P

s∈SP Obt(s)K a

t(o|s)Pta,o(s

0

|s)

P

s∈SP Obt(s)K a

t(o|s) ifat=C, o=

C−, C1 orat=H, o=H−

P

s∈SP O1bt(s)K a

t(o|s)Pta,o(s

0_|s)

P

s∈SP O1bt(s)K a

t(o|s) ifat=C, o=C+ orat=H, o=H+

(2)

When at=W;at=C,o=C−, C1; at=H, o=H−, the computation of τ[b, a, o] is

straightfor-ward. When at=C, o=C+; at=H, o=H+, if the patient is found to be in a state s∈SP O

2 , there is no update to the belief state because the treatment starts and the decision process ends. On the other hand, if the patient is found to be in a states∈SP O1_{, i.e., experiences a false-positive}

cytology or HPV-DNA test, then this patient continues to follow the decision process. This explains why the summation in Equation 2 for a positive result is only overs∈SP O1 _{and not over}

s∈SP O _{as in the non-positive result case.}

In regular Markov Decision Processes (MDPs), actions are selected based on current states. In a POMDPs some current states are not directly observable, but based on the observation, a

belief about the state the patient is in can be obtained. Since there is no certainty about the current state, all the information needed about the complete history of the patient is acquired by maintaining and updating the belief state (a probability distribution over all partially observable statess∈SP O_{) (Cassandra AR 1994). Smallwood et al. (1973) demonstrated that the belief state}

is a sufficient statistic for the past sequence of observations. Hence, if we update the belief state after each observation, we are saving all the available information from the previous period.

The order in which these events occur are illustrated in Figure 4. At the beginning of year t a decision at∈At is taken based on the current belief state bt∈B(SP O). Depending on the action

chosen,o∈Θa is observed and a health state transition occurs during the yeart. At the end of the

year, the belief state is updated to τ[b, a, o].

Figure 4 Order of events in the POMDP model.

2.1. Reward Function

Immediate rewards of our model are defined depending on the states∈SP O _{of the patient and the}

current time period. When the secondary test confirms the patient is in a states∈SP O2 _{and has}

started the corresponding treatment, we define Rt(s) to represent the total expected post-cancer

QALYs that can be accrued at timet.

In case the patient is in a states∈SP O2 _{but no secondary test is performed or in case the patient}

is in a states∈SP O1_{, we definer}

t(s, a, o) to represent the expected QALYs between timetandt+ 1

when the patient’s true health state iss∈SP O_{, the action chosen isa}

t∈A, ando∈Θa is observed.

The possible boundaries for rt(s, a, o) are: 0≤rt(s, a, o)≤1. These QALYs are defined such that

the value gained in a year can not exceed 1. The functionrt(s, a, o) incorporates the life expectancy

between t and t + 1 and the dis-utilities associated with a test that occurs in that time interval. Note that if the patient is in state s∈SP O2 _{and experiences a positive test, then she is assigned}

a lump-sum reward (QALYs) Rt(s). That is no QALYs are assigned over the next decision epoch

upon experiencing a true positive test. Then, rt(s, a, o) = 0 ∀s∈SP O

2

, a6=W, o∈ {C+, H+}.

In addition, when we are at the end of the time horizon no QALYs are assigned in the future. We definerT as the total expected remaining QALYs at timeT given that the true health state of

the patient iss∈SP O_.

2.2. Optimality Equations

The objective of this POMDP model is to maximize the total expected QALYs the patient can attain when the current information state is π∈Π(S), represented byVt∗(π).

As it can be seen in Equation 3, when we are certain that the patient is in a cancer treat-ment state, our reward function represents the total post-cancer QALYs previously defined as Rt(s) ∀s∈SP O

2

. For any other possible value different than zero in at least one state s∈SP O_,

Va

t(bt) represents the maximum total expected QALYs the patient can attain upon taking action

at∈At, when the current belief state is bt ∈S(SP O) at time t. Our goal is to find Vt∗(bt) =

max{VW

t (bt), VtC(bt), VtH(bt)}.

Vt∗(π) =

            

Rt(4) ifπ= [0000001000]

Rt(5) ifπ= [0000000100]

Rt(6) ifπ= [0000000010]

V∗

t (bt) ifbt= [π(1). . . π(6)]6= [000000]

0 otherwise (“death” state)

(3)

Ifat=W, the value function is computed as shown in Equation 4. The maximum total expected

QALYs when taking action W are the expected QALYs between time t and t+ 1 given that the patient is in state s∈SP O_{, plus the future expected rewards multiplied by the belief of being in}

that state. The term 1−PW

t (10|s) is the probability that the patient does not die between t and

t+ 1. Note that this term will be close to 1 for states s∈SP O1_{, but it will get smaller for states}

s∈SP O2_{, because states= 10 is a death state for any cause including cervical cancer. Then, when}

a patient is in a state s∈SP O2_{, i.e, a cancer state or high lesion state, the probability of dying}

due to cervical cancer is higher than in a states∈SP O1_{, i.e, a non-cancer diagnosed state. Hence,}

1−PW

t (10|s)∀s∈S

P O2_>1−PW

t (10|s) ∀s∈S P O1_.

VtW(bt) =

X

s∈SP O bt(s)

rt(s, W) + (1−PtW(10|s))V ∗

t+1(τ[b, W,∅])

Ifat=C orat=H, the value function is computed as shown in Equations 5 and 6, respectively.

When there is a non-positive observation, the logic is the same as in Equation 4. In the other hand, when there is a positive observation, since a secondary test is suggested, our value function will differ for states s∈SP O2_{, because treatment will be performed and the value function will be}

reduced toRt(s) ∀s∈SP O

2 .

VC t (bt) =

X

s∈SP O bt(s)

h

KC

t (C− |s)

rt(s, C, C−) + (1−P C,C−

t (10|s))V ∗

t+1(τ[b, C, C−])

i

(5)

+ X

s∈SP O bt(s)

h

KtC(C1|s)

rt(s, C, C1) + (1−P C,C1

t (10|s))V ∗

t+1(τ[b, C, C1])

i

+ X

s∈SP O1

bt(s)KtC(C+|s)

rt(s, C, C+) + (1−P C,C+

t (10|s))Vt∗+1(τ[b, C, C+])

+ X

s∈SP O2

bt(s)KC(C+|s)Rt(s) , t= 30,31, . . . , T−1

VtH(b) =

X

s∈SP O bt(s)

h

KtH(H− |s)

rt(s, H, H−) + (1−P H,H−

t (10|s))V ∗

t+1(τ[b, H, H−])

i

(6)

+ X

s∈SP O1

bt(s)KtH(H+|s)

rt(s, H, H+) + (1−P H,H+

t (10|s))Vt∗+1(τ(b, H, H+))

+ X

s∈SP O2

bt(s)KtH(H+|s)Rt(s) , t= 30,31, . . . , T−1

At the end of the time horizon, the value function will be computed as in Equation 7. No future rewards will be added, but the total expected remaining QALYs (rT(s) ∀s∈SP O) are considered.

VT∗(bt) =

X

s∈SP O

bT(s)rT(s) (7)

In a regular MDP in order to solve these equations and find the optimal policy, the state is mapped to actions. On the contrary, in POMDPs the mapping is done from the belief space to actions. This renders backward induction as an inapplicable algorithm to solve the model, since the belief space is a continuous space, and it is impossible to count all possible belief states (Cassandra AR 1994). Hence, two different approaches can be used to solve the model, either discretize it or use some special properties of the value function to apply a solution algorithm.

Smallwood and Sondik (1973) show that the optimal value function of a POMDP, V∗ t(bt) is

piecewise linear and convex (PW&C)∀t≤T. Hence, at the belief pointbt∈BS

P O

these functions can be expressed as the upper surface of a finite number of hyper-planes.

Namen, Akhavan-Tabatabaei and Gel: A POMDP Approach to Age-dependent Primary Screening Policies for Cervical Cancer in Colombia

15

0  

_b  

0   1  

V(b)  

b  

α1  

α3  

αK  

α4  

Figure 5 Linear representation of the value function

Figure 5 shows how the value function can be represented as a set of hyper-planes through the belief space and moreover, in terms of the coefficients of those linear functions, which are known asalpha-vectors.

2.3. Optimality Equations in Terms of The Alpha-vectors

Equation 8 shows the equivalency of the value function in terms of the alpha-vectors. The value function of a belief point will be the dot product of the alpha-vector and the belief vector.

Vt∗(bt) =maxk

n _X

s∈SP O

b(s)αkt(s)

o

(8)

Equation 9 shows how the alpha-vectors are computed whenat=W. In this case, the maximum

future expected QALYs are added to the immediate rewards. Equation 10 is used in order to know what the maximum future expected QALYs can be.

αlt(b,W)(s) =rt(s, W,∅) +

X

s0∈SP O

PtW,∅(s0|s)α i(b,W,∅)

t+1 (s

0_{) (9)}

where,

i(b, a, o) = arg maxk

n X

s∈SP O bt(s)

X

s0_∈SP O Pta,o(s

0_|s)αk t+1(s

0₎o ₍₁₀₎

k={l(b, W), l(b, C), l(b, H)}

Forat=Corat=H Equation 10 is also used, and as it was explained in the previous subsection,

the value functions forat=C orat=H depend on the observations, the alpha-vectors also depend

αlt(b,C)(s) =                                                          KC

t (C− |s)

h

rt(s, C, C−) +

P

s0_∈SP OP C,C−

t (s

0_|s)αi(b,C,C−) t+1 (s

0₎i

+KC t (C1|s)

h

rt(s, C, C1) +

P

s0∈SP OP C,C1 t (s

0_|s)αi(b,C,C1) t+1 (s

0₎i

+KC

t (C+|s)

h

rt(s, C, C+) + maxk

P

s0_∈SP OP C,C+ t (s

0_|s)αk t+1(s

0₎i _ifs∈SP O1_,

KC

t (C− |s)

h

rt(s, C, C−) +

P

s0∈SP OP C,C−

t (s

0_|s)αi(b,C,C−) t+1 (s

0₎i

+KC t (C1|s)

h

rt(s, C, C1) +

P

s0∈SP OP C,C1 t (s0|s)α

i(b,C,C1) t+1 (s0)

i

+KC

t (C+|s)Rt(s) ifs∈SP O2

(11)

αlt(b,H)(s) =

                                   KH

t (H− |s)

h

rt(s, H, H−) +

P

s0_∈SP OP H,H−

t (s

0_|s)αi(b,H,H−) t+1 (s

0₎i

+KH

t (H+|s)

h

rt(s, H, H+) + maxk

P

s0∈SP OP H,H+

t (s0|s)αkt+1(s

0₎i _ifs∈SP O1_,

KH

t (H− |s)

h

rt(s, H, H−) +

P

s0_∈SP OP H,H−

t (s0|s)α

i(b,H,H−) t+1 (s

0₎i

+KH

t (H+|s)Rt(s) ifs∈SP O2

(12)

and,

l∗(bt) =arg maxk

n X

s∈SP O

bt(s)αkt(s) o

(13)

=arg max{l(b,W),l(b,C),l(b,H)}

n X

s∈SP O

bt(s)αlt(b,W)(s), X

s∈SP O

bt(s)αlt(b,C)(s), X

s∈SP O

bt(s)αlt(b,H)(s) o

For our problem, αlt∗(b)(s) can be intuitively interpreted as the maximum QALYs that a patient

3. Solution Algorithm

We solve our POMDP model optimally using Monahan’s algorithm (Monahan GE 1982), with Eagle’s reduction (Eagle JN 1984), (Cassandra AR 1994).

The solution algorithm proceeds as follows: at each decision epoch, we first generate all possi-ble α−vectors by using Equations 9, 11 and 12, and then, we eliminate the α−vectors that are dominated at all possible information states. We do this elimination process in two phases. In the first phase, we eliminate allα−vectors whose components are completely dominated by other α−vectors. That is, if∃ j such thatαjt(s)≥αkt(s) ∀s∈S, then we eliminateα

k

t. These completely

dominated alpha-vectors are represented with a red color in Figure 6.

Algorithm 1: Monahan’s Enumeration with Eagle’s Reduction Algorithm

Initialize

αi_T(b,a,o)(s) =rT(s) ∀bt∈B(SP O)at∈A, o∈Θa

Generate

Generate all possibleα−vectors. Using Equations 9, 11, 12.

Mark each of the generated α−vectors and add them to a list.α={α1_{, α}2_{, ..., α}K_}

Eagle’s Reduction

Choose a markedα−vector from the list, unmark it and delete it from the list if its components are completely dominated by any otherα−vector. Do this until the list is empty.

b   0   1  

V(b)  

b  

0   1  

V(b)  

b  

α1  

α2  

α3  

αK  

α4  

α5  

0   1  

V(b)  

b  

a1  

0   1  

V(b)  

a2   a2   a3  

Figure 6 Linear representation of the value function and dominated alpha-vectors

After the first elimination phase, there are still some alpha-vectors that are not completely dominated at all belief points, but that may be dominated at some belief points. In order to know if there is any information vector at which αk is uniquely the optimal, the linear program (LP) in

Algorithm 2: Monahan’s Elimination Algorithm

Mark

Mark all of the remaining α−vectors in the list

Select

Choose a marked α−vector from the list. If none exists, then terminate. Otherwise,

Construct LP

Unmark the selected α-vector and construct the LP in Equation 14 for that vector

Eliminate

If the LP in Equation 14 yields a solution σ≤0, then remove this α−vector from the list.

max σ (14)

s.t. X

s∈S

π(s)αk

t(s)−α j t(s)

−σ≥0 ∀j∈α

X

s∈S

π(s) = 1

π(s)≥0 ∀s∈S

If this LP yields a solution σ >0, then there exists at least one information state π such that αk

t is uniquely optimal. Otherwise, we can eliminate α k

t from the possible set ofα−vectors (α).

b   0   1  

V(b)  

b  

0   1  

V(b)  

b  

α1  

α2  

α3  

αK  

α4  

α5  

0   1  

V(b)  

b  

a1  

0   1  

V(b)  

a2   a2   a3  

Figure 7 Linear representation of the value function after applying Monahan’s algorithm with Eagle’s reduction

Figure 7 shows how the second elimination phase works. By using the LP in Equation 14 the rest of the alpha-vectors are eliminated if there is at least one information vector at which αk is

uniquely the optimal.

4. Computational Experiments

In this section, we present some computational examples on the generated policies at different ages and show how the decision changes after an observation is obtained. The input data come from different sources listed in Table 2. Core state transition probabilities represent the natural history

of cervical cancer. Rewards correspond to total expected gained QALYs or postcancer QALYs. Sensitivity is the probability that a patient that is in a state s∈SP O2 _{is correctly diagnosed with}

a positive result, and specificity is the probability that a patient that is in a state s∈SP O1 _is

correctly diagnosed with a non-positive result.

Parameter Data source

Core state transition probabilities Ley-Chavez et al. 2012, Kulasingam et al. 2006 Bergeron et al. 2008, CDC 2012, Simulated data.

Rewards Ayer et al. 2012, Simulated data

Sensitivity & specificity of cytology Ley-Chavez et al. 2012, Munoz et al. 2004 Simulated data

Sensitivity & specificity of HPV-DNA test Gamboa et al. 2008, Simulated data Table 2 Sources of model inputs

We will first see the optimal screening strategies as a function of in-situ and invasive cancer for 30, 40, 50 and 60 year old patients. Then, we will show how the optimal screening policy is generated for a 40-year old patient with 70% chance of being in s=C2N and 15% chance of being in both, in-situ and invasive cancer. In addition, other cases are presented by defining the initial belief of a patient after applying some risk estimation models. At the end of the section, a comparison between the current screening policies and the optimal screening strategy is also included.

4.1. Optimal Screening Strategy

Figure 8 depicts the optimal screening strategy as a function of in-situ and invasive cancer risks for various ages. The horizontal and vertical axes represent the probability (risk) of in-situ and invasive cancer, respectively; therefore, the probability of being in any other state is equal to 1 minus these probabilities.

We can see that as the age increases it takes more time to a patient to reach a belief state outside of the Wait area. On the other hand, it is likely that a 30-year old patient needs to take an HPV-DNA test for at least half of all the possible combinations of in-situ and invasive cancer risks. In addition, the HPV-DNA and the cytology screening threshold risk for invasive cancer is lower than for in-situ cancer. This can be interpreted as the risk of invasive cancer is more significant than the risk of in-situ cancer in determining the screening decision.

Figure 8 Optimal primary screening decisions by age

4.2. Optimal Screening Policy Generation

We now illustrate in Figure 9, how the optimal preventive policy is being generated for a 40 year-old patient that has a 15% chance of being in in-situ or invasive cancer and 70% chance of being in C2N. Based on her age and risk profile she is in the Wait region. After one year her belief state is updated and the risk of being in both in-situ and invasive cancer increases, locating her in the Cytology region. Assuming C1 is observed, the belief state is updated again but the patient remains in the same region. At age 42 C+ is observed and after updating the belief state, the patient moves to the HPV-DNA region. Assuming H− is observed, at age 44 after updating the belief state the patient moves to theWait region, and at age 45 another cytology is suggested.

Figure 9 Optimal screening policy generation.

Figure 10 illustrates how from a same belief state the optimal preventive policy varies based on the outcome observed. In this example, we continue with a 40 year-old patient, but with 20% chance of being in in-situ cancer, 70% chance of being in invasive cancer and 10% chance of being inC2N. According to her age and risk profile she is in the HPV region. If the result of the exam isH−one year period of Wait is recommended. On the other hand, if the result isH+ a cytology is recommended. In case of waiting, after updating the belief vector a cytology is recommended and if the result is C1 another cytology is recommended. In the scenario of the HPV-DNA test followed by cytology, if C− is observed, one year period of Wait is recommended and then a cytology is also recommended.

Now consider a 40-year old patient with 70% risk of invasive cancer, 20% risk of in-situ cancer and 10% risk of CIN2. Figure 10 shows how the optimal policy changes according to the screening results obtained.

Figure 10 Optimal screening policy generation for different observations.

By comparing these two scenarios we can see that after a negative result with the HPV-DNA test a cytology is recommended in 2 years. However, if a positive result is obtained a follow-up cytology in one year is recommended. Thus, the frequency and the exam for screening are highly dependent of the screening results obtained.

The policy graphs presented in Figure 8, 9 and 10 can also be represented as a function of any other state s∈SP O_{. This can be useful in cases where there is not clarity about the belief of}

4.3. Risk Estimation Models

The previous subsections showed what the optimal preventive policy should be based on an initial risk of in-situ and invasive cancer and by updating the belief states using Equations 1 and 2. In order to know this initial belief states, some risk estimation models have been proposed in the literature.

The Siteman Cancer Center (SCC) has a cervical cancer disease risk estimator based on the sex, age, previous cancer presence, previous hysterectomies, if the patient smokes or not, number of sexual partners in her lifetime, number of births, previous sexually transmitted infection (STI) and if she has taken a Pap test (cytology) in the last 3 years. The result is a scale between low, average and high risk of cervical cancer, and some recommendations and explanations about what raises the risk and how to lower the risk to the lowest possible are also given (Harvard Cancer Risk Index 2000).

The ATHENA Cervical Cancer Risk Assessment Module (ACCRAM) is another risk estimator based on the Athena Study Results of the cytology and HPV-DNA test and the age of the patient. It gives a percentage estimation of CIN2+ (CIN2,CIN3 (in-situ) or invasive cervical cancer) risk. We will now illustrate some case examples for different patient profiles and their possible belief state by using the risk estimators previously mentioned.

Case 1. A 35-year old patient with a previous cytology with C− result and non HPV-DNA test record. Using the ATHENA Cervical Cancer Risk Assessment Module (ATHENA 2014) her risk of CIN2+ (chance of being in any state s∈SP O2_{) is 0.082. Hence, her chance of being in any}

state s∈SP O1 _{is 1-0.082=0.918. Based on the optimal screening policy presented in Figure 11,}

the optimal action for this patient is to wait at age 35.

Case 2. Same patient as in Case 1 but with a negative (H-) HPV-DNA test record. Her risk of CIN2+ reduces to 0.01. Hence, her chance of being in any states∈SP O1 _{is 1-0.01=0.99. Based on}

the optimal screening policy presented in Figure 11, the optimal action for this patient is also to wait at age 35.

Case 3. Same patient as in Case 1 but with a positive (H+) HPV-DNA test record for strand 16. Her risk of CIN2+ increases to 0.144 Hence her chance of being in any state s∈SP O1 _is

for this patient is also to wait at age 35.

Case 4. Same patient as in Case 1 but with a positive (H+) HPV-DNA test record for strand 18. Using the ATHENA Cervical Cancer Risk Assesment Module (ATHENA 2014) her risk of CIN2+ (chance of being in any state s∈SP O2_{) increases to 0.109. Hence her chance of being in}

any states∈SP O1 _{is 1-0.109=0.891. Based on the optimal screening policy presented in Figure 11,}

the optimal action for this patient is also to wait at age 35.

Figure 11 Optimal primary screening at age 35 as a funtion of CIN2+ risk

Notice that the policies presented in Figure 11 may vary from the ones in Figure 8, since the policy in Figure 11 is a function of the minimum risk needed for any state sinSP O2 _{in order}

to define the optimal action. This means that as more information is obtained, such as the distribution of the belief over all the states a higher risk than the one presented in Figure 11 may be needed to undergo that action. For example, if the only information available is that the patient has 30% risk of being in CIN2+ but not the distribution of the belief is available, based on the optimal policy in Figure 11, the patient should undergo cytology at age 35. However, if we know that 15% of the risk is in situ cancer and 15% of the risk is in invasive cancer, based on the optimal policy in Figure 8 the patient should Wait at age 35.

As we can see, the optimal action for all patients from Case 1 to Case 4 is to Wait even if their risk profile is different (different records in the screening tests), since the risk of CIN2+ was low, between 0 and 0.22 and no distribution over all the states of the belief state was obtained due to the current risk estimation models available in the literature. In the next section, the creation of a better risk estimation model for cervical cancer is suggested as future work.

In addition to the action that should be taken based on a risk profile, our model can also be used to find the optimal screening frequency that should be done for any of the primary screening

tests.

Case 5. A 30 year-old patient with a previous cytology with C1 result and a positive (H+) HPV-DNA test record for strand 16. Using the ATHENA Cervical Cancer Risk Assesment Module (ATHENA 2014) her risk of CIN2+ (chance of being in any state s∈SP O2_{) is 0.316. Hence}

her chance of being in any state s∈SP O1 _{is 1-0.316=0.684. We will assume that these risks}

are uniformly distributed over all the partially observable states. Thus, her belief state will be b30= [0.228,0.228,0.228,0.106,0.105,0.105]. Based on the optimal screening policy presented in Figure 11, the optimal action for this patient is to wait at age 30. Assuming that the outcomes of the following primary screening tests are all negative, this patient should undergo cytology exams at ages 33, 35, 39, 45, 47, 48, 53, 55, 58 and 59. The updated risks and the corresponding optimal actions for this patient are presented in Table 3.

For this patient, no more screening is suggested after 59 years-old, and no HPV-DNA tests are suggested over her lifetime. This case illustrates the effect of an initial low risk for both in-situ and invasive cancer and negative results in all the exams. Compared to the policies showed in Figure 9 and 10 where the HPV-DNA test is recommended, it can be seen that the HPV-DNA test is always recommended after a positive result in the cytology, or a starting high risk in the initial belief state, and none of these events occur in Case 5.

All the previous cases illustrate the optimal action and optimal screening policy for a patient based on an estimated risk of CIN2+, that was calculated with previous information of a cytology or a HPV-DNA test result. However, in some cases no previous history exists and then the decision maker is not able to have an estimate of the initial belief state. For this case, we will assume that her risk of being in any state is uniformly distributed over all the partially observable states. Case 6 illustrate the optimal action for this patient profile.

Case 6. A 35-year old patient with no previous history. Since there is no previous information we will assume that her risk of being in any state is uniformly distributed over all the partially observable states. Thus, her belief state will beb35= [0.167,0.167,0.167,0.167,0.166,0.166]. Based on the optimal screening policy presented in Figure 8, the optimal action for this patient is to wait at age 35. A different approach can be a 50% chance of CIN2+ risk and 50% chance of non CIN2+ risk, and based on the optimal screening policy presented in Figure 11 even a Cytology or HPV-DNA test is recommended. However, as it was shown in the previous cases, the initial risk profile has a high impact in the following screening decision, and without previous information,

the initial risk estimation may be a false positive or false negative risk patient profile. Hence, it is recommended to perform a primary screening test when no previous history is available, in order to have a better estimate on the risk profile of a patient.

4.4. Current Screening Policies vs. the Optimal Screening Strategy

The estimation of invasive cervical cancer risk changes under different screening scenarios. We compare three scenarios: no screening, screening every three years with cytology after age 30 assuming that each screening result was negative, screening every five years with HPV-DNA test after age 30 assuming that each screening result was negative. The two last scenarios are the current screening policies recommended in the cervical cancer medical guidelines. To illustrate the effect of a woman’s prior screening history in her estimated risk of invasive cervical cancer, consider three different patients who are all 40 years old with the same risk characteristics, but different screening histories: patient 1 has never had a primary screening test before, patient 2 had prior cytologies at ages 30, 33, 36 and 39, all of which were negative; patient 3 had prior HPV-DNA tests at ages 30 and 35, both with negative results.

Figure 12 Invasive cancer risk as a function of personal history of screening

As it is shown in Figure 12, patient 1 has a 19.19% risk of invasive cancer, patient 2 has a 7.52% and patient 3 just 2.54% at the age of 40. The optimal action for patient 1 is toWait, the optimal action for patient 2 is to Wait, and the optimal action for patient 3 is also to Wait. However, based on the current screening guidelines patient 2 is recommended to have an HPV-DNA test. Hence, under-screening increases the risk of invasive cancer, while over-screening is unnecessary and instead incorporating personal history may be useful to define optimal screening strategies.

Age Risk of in-situ cancer Risk of invasive cancer Optimal action

30 0.105 0.105 W

31 0.118 0.115 W

32 0.135 0.126 W

33 0.137 0.140 C

34 0.118 0.123 W

35 0.153 0.133 C

36 0.114 0.111 W

37 0.126 0.112 W

38 0.138 0.115 W

39 0.142 0.130 C

40 0.137 0.121 W

41 0.146 0.126 W

42 0.157 0.144 W

43 0.173 0.172 W

44 0.201 0.191 W

45 0.217 0.210 C

46 0.197 0.202 W

47 0.215 0.221 C

48 0.200 0.217 C

49 0.179 0.198 W

50 0.193 0.206 W

51 0.213 0.210 W

52 0.227 0.231 W

53 0.229 0.242 C

54 0.206 0.218 W

55 0.219 0.237 C

56 0.205 0.219 W

57 0.216 0.235 W

58 0.224 0.252 C

59 0.207 0.247 C

60 0.191 0.232 W

61 0.208 0.235 W

62 0.211 0.239 W

63 0.223 0.239 W

64 0.228 0.246 W

65 0.232 0.256 W

66 0.234 0.261 W

67 0.239 0.264 W

68 0.239 0.268 W

69 0.242 0.269 W

70 0.249 0.273 W

71 0.254 0.274 W

72 0.257 0.275 W

73 0.259 0.277 W

74 0.261 0.278 W

75 0.263 0.278 W

76 0.268 0.282 W

77 0.273 0.283 W

5. Conclusions and future work

We addressed the problem of screening frequency for cervical cancer in Colombia, considering disease regression, progression and test characteristics. We formulate and optimally solve a POMDP model for age-based optimal screening strategies, showing that age is a significant factor in determining optimal screening decisions. Through some illustrative example we demonstrate the performance of this model by comparing how the policy changes for patients with different risk profiles. One important contribution of our work is that we include the HPV-DNA test in the possible decisions than can be taken and since it is a relatively new test used in Colombia, it gives our model a long-term validity.

An important follow-up to this work is to extend the model for each particular observation of cytology and HPV-DNA test. Some modifications can also be included, such as adding cotesting as a possible primary screening action. In order to have more accurate results, a natural history simulation model for cervical cancer and HPV infection may be helpful to have input data parameters of the target populations. A cervical cancer risk estimation model is needed to define a better initial belief state based on a patient risk profile. Some sensitivity analysis should be done for false positive and false negative analysis. Additionally, some work can be done to reduce the computational time of the model and to improve its performance.

Future research directions include the comparison of policies between vaccinated and non-vaccinated patients and between different socio-economic populations.

References

Harvard Report on Cancer Prevention. Volume 4: Harvard Cancer Risk Index, Cancer Causes and Control

11:477-488,2000. Kluwer Academic Publishers

http://yourdiseaserisk.wustl.edu/Commonpdf/CancerPrevention.pdf.

ATHENA Cervical Cancer Risk Assessment Module.

https://www.hpv16and18.com/assets/risk-calculator/calculator.html.

Siteman Cancer Center, A National Cancer Institute Comprehensive Cancer Center. Barnes-Jewish Hospital

- Washington University School of Medicine. Retrieved Dec 17, 2014.

http://yourdiseaserisk.wustl.edu/YDRDefault.aspx?ScreenControl=

YDRGeneral&ScreenName=YDRcervical.

American Society for Colposcopy and Cervical Pathology (ASCCP), Updated Consensus Guidelines for

Managing Abnormal Cervical Cancer Screening Tests and Cancer Precursors (2013). Retrieved Nov

21, 2014.

http://www.asccp.org/portals/9/docs/algorithms%207.30.13.pdf.

World Health Organization (WHO) Guidance note (2013), Comprehensive cervical cancer prevention and

control: a healthier future for girls and women. Retrieved Nov 16, 2014.

http://apps.who.int/iris/bitstream/10665/78128/3/9789241505147_eng.pdf?ua=1.

National Cancer Institute (NCI) HPV and Cancer Fact Sheet (2014). Retrieved Nov 16, 2014.

http://www.cancer.gov/cancertopics/factsheet/Risk/HPV.

American Cancer Society (ACS) Cervical Cancer (2014). Retrieved Nov 16, 2014.

http://www.cancer.org/acs/groups/cid/documents/webcontent/003094-pdf.pdf.

Profamilia (2014). Retrieved Nov 16, 2014.

http://www.profamilia.org.co/cancerdecuellouterino/.

UAE-CRES (2011). Informe pruebas ADN-VPH an´alisis de impacto financiero sobre la UPC para los

reg´ımenes subsidiado y contributivo. Bogota.

UAE-CRES (2011A). Pruebas ADN-VPH para la tamizaci´on de c´ancer de cuello uterino. Bogota.

Incidence and mortality rates for cervical cancer have decreased in Colombia, Ministerio de Salud (2013).

Retrieved Nov 16, 2014.

http://www.minsalud.gov.co/Paginas/Bajan-tasas-incidencia-mortalidad-cancer-cuello.

aspx.

Walboomers, JM and Jacobs, MV and Manos, MM and Bosch, FX and Kummer, JA and Shan, KV and

Snijders, PJ and Peto, J and Meijer, CJ (1999) Human papillomavirus is a necessary cause of invasive

cervical cancer worlwide. 189(1)12:21. Journal of Pathology. Wiley Online Libray.

A.J. Schaefer, M.D. Bailey, S.M. Schechter, and M.S. Roberts. Modeling medical treatment using Markov

and Healthcare: A Handbook of Methods and Applications, pages 593-612. Kluwer Academic

Publish-ers, Bostos, MA, 2004.

Ayer T, Alagoz O, Stout NK (2012) OR forum: A POMDP approach to personalize mammography screening

decisions. Oper. Res. 60(5):1019-1034.

Safa Erenay F, Alagoz O, Said A (2014) Optimizing Colonoscopy Screening for Colorectal Cancer Prevention

and Surveillance. Articles in Advance, Manufacturing & Service Operations Management. pp 1-20.

Ley-Chavez A (2012) Quantitative Models to Design and Evaluate Risk-Specific Screening Strategies for

Cervical Cancer Prevention. Dissertation, Ohio State University.

Rathore S, McGreevey JD, Schulman k, Atkins D (2000) Mandated coverage for cancer-screening services:

whose guidelines do states follow? Amer. J. Preventive Medicine 19(2)71-78.

Munoz, et al. (2004) Incidence, Duration, and Determinants of Cervical Human Papillomavirus Infection in

a Cohort of Colombian Women with Normal Cytological Results.

Gamboa, et al. (2008) Cost-effectiveness of conventional cytology and HPV-DNA testing for cervical cancer

screening in Colombia.

Optimal control of Markov decision processes with incomplete state estimation. J. Math. Anal. Appl.

10(1)174-205.

Smallwood RD, Sondik EJ (1973) The optimal control of partially observable Markov processes over a finite

horizon. Oper. Res. 21(5) 1071-1088.

Monahan GE (1982) A survey of partially observable Markov decision processes: Theory, models, and

algo-rithms. Management Sci. 28(1) 1-16.

Eagle JN (1984) The optimal search for a moving target when the search path is constrained. Oper. Res.

32(5) 1107-1115.

Cassandra AR (1994) Optimal policies for partially observable Markov decision processes. Technical report,

Brown University, Providence, RI.

Kulasingam SL, Havrilesky L, Ghebre R, Myers ER. Screening for Cervical Cancer: A Decision Analysis

for the U.S. Preventive Services Task Force. AHRQ Publication No. 11-05157-EF-1. Rockville, MD:

Agency for Healthcare Research and Quality; May 2011.

Goldie, S. J., M. C. Weinstein, K. M. Kuntz, and K. A. Freedberg 1999. The Costs, Clinical Benefits,

and Cost-Effectiveness of Screening for Cervical Cancer in HIV-Infected Women. Annals of Internal

Medicine 130, no. 2: 97-107.

Myers, E. R., D. C. McCrory, K. Nanda, L. Bastian, and D. B. Matchar 2000. Mathematical Model for the

Natural History of Human Papillomavirus Infection and Cervical Carcinogenesis. American Journal of

Epidemiology 151, no. 12: 1158.

McLay, L. A., C. Foufoulides, J. R. Merrick. 2010. Using simulation-optimization to construct screening

Kim, J. J., T. C. Wright, and S. J. Goldie. 2002. Cost-effectiveness of alternative triage strategies for atypical

squamous cells of undetermined significance. Journal of the American Medical Association 287, no. 18:

2382-2390.

Goldie, S. J., J. J. Kim, and T. C. Wright. 2004. Cost-Effectiveness of Human Papillomavirus DNA Testing

for Cervical Cancer Screening in Women Aged 30 Years or More. Obstetrics & Gynecology 103, no. 4:

619-631.

Eggington, S., R. Hadwin, A. Brennan, and P. Walker. 2006. Modelling the impact of referral guideline

changes for mild dyskaryosis on colposcopy services in England. Guidelines for the NHS Cervical

Screen-ing Programme. NHSCSP Publication 24.

Kulasingam, S. L., E. R. Myers, H. W. Lawson, K. J. McConnell, K. Kerlikowske, J. Melnikow, A. E.

Wash-ington, and G. F. Sawaya. 2006. Cost-effectiveness of Extending Cervical Cancer Screening Intervals