Part 2: Characteristics for Success: Cervical Cancer Screening
Part 3: Implementation of Clinical Quality Measure: Cervical Cancer Screening
Part 4: Improvement Strategies: Cervical Cancer Screening
Part 5: Holding the Gains and Spreading Improvement
Part 6: Supporting Information
Part 3: Implementation of Clinical Quality Measure: Cervical Cancer Screening
Before following the steps in Part 3, an organization should first make a commitment to increase the rate of cervical cancer screening with Pap testing, and complete the initial steps outlined in the previous section that include:
Performance on this measure indicates how effectively all the steps of the processes used to deliver care work together so that cervical cancer screening with Pap testing is optimized. Because there are so many factors that can have an impact on whether patients receive screening Pap tests, it helps to visualize how these steps are mapped. The next section defines a Critical Pathway and illustrates the application of this concept for testing improvements to improve cervical cancer screening in female patients.
A critical pathway, also known as a clinical pathway, is a visual depiction of the process steps that result in a particular service or care. The sequence and relationship among the steps are displayed, which reveals a map of the care process. Additional information, including tools and resources regarding the mapping of care processes, can be found in the Redesigning a System of Care to Promote QI module. In an ideal world, the care process is reflective of evidence-based medical guidelines. Evidence-based medicine aims to apply the best available evidence gained from the scientific method for medical decision making that leads to improved outcomes. There are a number of evidence-based recommendations for Cervical Cancer Screening, including those referenced earlier in this module.
A map of the care process steps, which incorporates all of the known evidence and follows respected evidence-based medical guidelines, can be considered the idealized critical pathway. While the guidelines for Cervical Cancer Screening do not completely align, there are recommended steps that include shared decision making that incorporate individual risk, including age, known family history, individual risk factors, and patient preferences. The purpose of listing these steps is to reflect current best practices for cervical cancer screening and form a systematic method to consider the systems of care that underpin appropriate screening. It is important to emphasize that clinical evidence and guidelines will evolve as knowledge progresses; therefore, the idealized critical pathway may evolve over time and not meet the needs of every individual.

Walkthrough of the Idealized Critical Pathway
The steps illustrated in the schematic reflect a system for cervical cancer screening that is working well. This pathway extends beyond the boundaries of what is assessed with the Cervical Cancer Screening clinical quality measure as important aspects of care for women of high risk precede the age range targeted in the measure. These steps are pertinent to effective cervical cancer screening in general and encompass cervical cancer screening for patients of average risk:
1. All female patients should have readily-available information about cervical cancer and other age-appropriate screening. Data on natural history of HPV infection and the incidence of high-grade lesions and cervical cancer suggest that screening can safely be delayed until age 21. (7) Family and personal medical histories should be obtained for all patients and used to guide screenings. (24)
2. Prescreening and education using the recommended clinical guidelines that are tailored toward the patient's risk serve as opportunities for prevention. Female patients who are 21 years or older should be assessed specifically for risk factors for cervical disease and offered a cytologic screening test at least every three years until age 65. Discontinuation of cervical cancer screening in older women is appropriate, provided women have had adequate recent screening with normal Pap results. The optimal age to discontinue screening is not clear, but risk of cervical cancer and yield of screening decline steadily through middle age. The U.S. Preventive Services Task Force (USPSTF) found evidence that yield-of-screening was low in previously-screened women after age 65 who had previous normal test results. (25) The American Cancer Society (ACS) guidelines recommend that older women, who have three or more consecutive normal/negative cervical cytology tests that are technically satisfactory and documented and had no abnormal/positive cytology tests within the last 10 years, can safely stop screening.(26)
3. A patient-provider partnership is needed to ensure that decisions respect a patient's wants, needs, and preferences, and she has the required education and support to make informed decisions and participate in her own care. (27) Patients who are aged 21 years or older should be strongly encouraged to complete cytologic screening.
4. Through the screening and risk assessment step, the provider and patient together determine that cytologic screening should be ordered.
4a. If a patient does not meet screening guidelines, she is not screened; however, it is an opportunity to educate the patient on the importance of screening if her risk status changes or when it is age appropriate. Interim and follow-up care is then discussed to ensure that the patient has what is needed to prevent cervical cancer. Guidelines are emphasized so the patient understands the benefits of cervical cancer screening and its risk factors. Appropriate follow-up screening occurs in a timely manner and the cycle repeats. In addition, a patient may choose to decline screening even if strongly encouraged by the health care team. A patient should be periodically re-assessed and supported to complete screenings as per current guidelines.
4b. Ensuring that cervical cancer screening has been completed is essential for preventive care. Care teams should invite a conversation about any barriers – real or perceived – to completing the cervical cancer screening and work together with a patient to mitigate those barriers.
5. Establishing a process to retrieve and review cytology results is important for tracking the number of completed screenings and a patient's adherence to recommended guidelines. Internal systems should clearly define who reviews the results of both positive and negative screenings.
6. Patient notification of the results provides an opportunity to involve the patient in her care plan and educate her about healthy behaviors to minimize cervical cancer risks. Appropriate follow-up to the screening is important and, regardless of the result, should include timely notification to the patient.
6a. Negative screening results should prompt interval screening recommendations per the adopted guidelines.
6b. Positive results should be communicated to the patient in a culturally-sensitive manner. Treatment information and advice should be provided to the patient with an appropriate appointment or referral for additional diagnostic testing or treatment.
The cycle repeats with appropriate interval screening, including outreach to patients as needed.
Note: With the recent recommendations from the American College of Obstetricians and Gynecologists (ACOG), national authorities are looking at aligning measures with the new guidelines. Decisions on a clinical level and informing patients on the benefits and risks of screening remain essential in determining who should be screened for cervical cancer.
A quality improvement team benefits from mapping out how care is actually provided. Once it is able to evaluate where there are potential opportunities for improvement, it can use some of the improvement ideas that have worked for others, as outlined in Table 4.2: Sample Changes That Work.
A couple of important notes:
In addition to understanding the steps for Cervical Cancer Screening, factors that interfere with optimal care should be understood. As there may be several of these factors, a QI team may find it helpful to focus its attention on factors that interfere with ideal outcomes. This becomes especially useful as plans are developed to mitigate these factors.
Factors that have an impact on Cervical Cancer Screening can be organized into those that are patient-related, related to the care team, and a result of the health system. Overlaps exist in these categorizations, but it is useful to consider factors that have an impact on care processes from each perspective to avoid overlooking important ones.
Patient factors are characteristics that patients possess, or have control over, that have an impact on care. Examples of patient factors are age, race, diet, and lifestyle choices. Common patient factors may need to be addressed more systematically, such as, a targeted approach to address low health literacy, or a systematic approach to educate staff on the cultural norms of a new refugee population. Examples of how patient factors may influence cervical cancer screening include:
Care team factors are controlled by the care team. These types may include care processes, workflows, how staff follows procedures, and how effectively the team works together. Care team factors that may influence Cervical Cancer Screening include:
Health system factors are controlled at the high level of an organization and often involve financial and operational issues. Health system factors that may influence Cervical Cancer Screening include:
These factors, when added to the critical pathway, create another dimension to the map as shown in Figure 3.2:


| Factor Category | Factors pertinent to our organization - Steps |
|---|---|
| Patient | Some patients prefer to go to a private provider, health department, or family planning clinic for their Pap test. One small site of the organization is located in the same building as the health department and sends women there for gynecological examinations. Due to the close proximity, patients expect WCHG to have results on file. |
| Care Team | When a patient reports that she had a Pap test elsewhere, the staff requests that the patient provide information about the location and sign a records release. This is not routinely done for the site that refers to the health department. |
| Health Systems | Compliance with the measure requires documentation of Pap results; copies of reports are not consistently received from outside providers. |
| After the team thought through some of the challenges, it is able to focus improvement efforts on this part of the care system. | |
The team continues to look at different parts of the pathway to identify relevant impacts for each part. Once it is able to evaluate where there are potential opportunities for improvement, it can use this information to target its efforts. Additional examples of strategies to improve care for the measure, Cervical Cancer screening, are described in the Part 4: Improvement Strategies section of this module.
Once the team visualizes the pathway and identifies opportunities for improved care processes, the next step is to collect and track data to test and document them. First, a QI team needs to determine how to collect data to support its improvement work. This step is essential for understanding the performance of its current care processes, before improvements are applied, and then monitoring its performance over time.
There are three major purposes for maintaining a data infrastructure for quality improvement work:
The first step to creating a data infrastructure for monitoring the performance measure is to determine the baseline. A baseline is the calculation of a measure before a quality improvement project is initiated. It is later used as the basis for comparison as changes are made throughout the improvement process. For the Cervical Cancer Screening measure, an organization can determine the percentage of patients aged 21to 64 years who had one or more Pap tests during the measurement year or the previous two years. Performance reflects the current organizational infrastructure and the patient's interactions with existing care processes and the care team.
Baseline data is compared to subsequent data calculated similarly to monitor the impact of quality improvement efforts. The details of how to calculate the data must be determined to ensure that the calculation is accurate and reproducible. The difference between how an organization provides care now (baseline) and how it wants to provide care (aim) is the gap that must be closed by the improvement work.
The next step of data infrastructure development involves a process in place to calculate the measure over time as improvements are tested. A QI team's work is to make changes, and it is prudent to monitor that those changes result in achieving the stated aim. This involves deciding how often to calculate the measure and adhering to the calculation methodology.
Finally, an organization's data infrastructure must include systematic processes that allow analysis, interpretation, and action on the data collected. Knowledge of performance is insufficient for improvement. It is important for an organization to understand why performance is measured and to predict which changes will increase cervical cancer screening rates with Pap testing based on an organization's specific situation. Collecting data related to specific changes and overall progress related to achieving an organization's specified aim are important to improvement work. The next section describes in more detail how to develop a data infrastructure to support improvement.
This section explores each step to create the data infrastructure used to improve performance on the measure, Cervical Cancer Screening.
Source: NQF/NCQA
Description: The percentage of women 21to 64 years of age who received one or more Pap tests to screen for cervical cancer.
Rationale/Purpose: Most cervical cancer can be prevented, and when found and treated early, most can be cured. Despite effective screening techniques, it was estimated that in 2009 there would be 11,270 new cases and 4,070 deaths from cervical cancer. The goal for this measure is to ensure adequate screening of women for cervical cancer using the Papanicolaou or Pap smear.
Numerator/Denominator:
Numerator: Women in the denominator with one or more Pap tests during the measurement year or two years prior to the measurement year.
Denominator: All women patients 24 to 64 years of age who received one or more Pap tests during the measurement year or 2 years prior to the measurement year.
Denominator Exclusions/Inclusions/Notes/Comments:
Denominator Exclusions: Women who had a hysterectomy, with no residual cervix, and the administrative data does not indicate a Pap test was performed. Note: Look for evidence of a hysterectomy as far back as possible in the patient's history through administrative or chart data. The hysterectomy must have occurred by December 31 of the measurement year.
Denominator Inclusions: Given the measurement look-back period, women aged 21to 64 years will be captured in this measurement. Identify patients who had at least one office visit in the prior 12 months.
Numerator/Exclusions/Notes/Comments:
Numerator Exclusions: Do not count lab results that explicitly state the sample was inadequate or that “no cervical cells were present;” this is not considered appropriate screening. Do not count biopsies because they are diagnostic and therapeutic only and are not valid for primary cervical cancer screening.
Numerator Inclusions: Women from the denominator with one or more Pap tests during the measurement year or two years prior to the measurement year. Count any cervical cancer screening method that includes collection and microscopic analysis of cervical cells. Documentation in the medical record must include one of the following: a note or billing code indicating date the test was performed and its result; copy of a lab test performed by another provider, or a note documenting the name, date, and results of a test performed by another provider.
Step 1 - Determine and Evaluate the Baseline
As previously discussed, a baseline for improvement is a calculation that provides a snapshot of the performance of the systems of care for a measure before improvements are applied. The baseline is determined by calculating the measure and collecting the information for the numerator and denominator.
Determination of a baseline is accomplished by actually calculating the measure and requires that the information for the numerator and denominator be collected. There are several methods to collect this information. While electronic methods are more efficient once established, manual chart audits using random sampling techniques are equally valid. random sampling techniques are equally valid.
Consistent data collection sources and methodologies are critical to ensure reliable data. Please note that the tables referenced in this section are from the NQF-Endorsed National Voluntary Consensus Standards for Physician-Focused Ambulatory Care Appendix A- NCQA Measure Technical Specifications. 
The following tables depict a decision algorithm for the measure, Cervical Cancer Screening. The algorithm outlines the steps that an organization follows to determine its baseline and monitor improvements for Cervical Cancer Screening:
| Identify the Denominator | |
|---|---|
| The denominator for this measure is all women patients 24 to 64 years of age during the measurement year or 2 years prior to the measurement year. | |
| a. Use a three-year date range: the measurement year and the two years prior to the measurement year. A date range to audit. | |
| b. Choose a selection method | Use date of birth or age from EHR or Practice Management System to identify women 24 to 64 years of age as of December 31 of the measurement year. Note: Given the measurement look-back period, women aged 21 to 64 years will be captured in this measure. Identify patients who have had at least one office visit in the prior 12 months. |
| c. Exclude women who had a hysterectomy and with no residual cervix. Look for evidence of a hysterectomy as far back as possible in the patient's history, through administrative or chart data. The hysterectomy must have occurred by December 31 of the measurement year. | Use these codes or operative reports to verify hysterectomy:
|
| Identify the Numerator |
|---|
| Based on an organization's systems, evaluate all of the individuals who remain in the denominator and choose a method to determine those who should be included in the numerator--women in the denominator who received one or more Pap tests during the measurement year or the two years prior to the measurement year. |
a. Administrative Method: Audit all submitted claims or encounters for patients in the denominator and include those with the following codes:
|
b. Medical Record Audit: Audit all patients in the denominator or use valid sampling methodology. The records audited may be electronic or paper. Include the patient in the numerator if the documentation in the medical record includes:
|
| Calculate the Measure |
|---|
| Divide the numerator by the denominator and multiply by 100 to get the percentage of women who received one or more Pap tests between the ages of 24 to 64 years. |
Compare an organization's performance to national benchmarks and other available data. The NCQA Web site
updates national and State performance on this measure annually. Note that there is considerable variation among practices reporting. Other opportunities for comparison data are from payers, State cancer control programs, State and regional quality improvement organizations, and aggregate reports for specific HRSA-funded programs.
Decide if the performance is satisfactory based on available data from reliable sources. It is important to consider the organizational capacity and constraints, but it is recommended that an organization's aim is high. An organization with a low performance may want to allow a longer time to achieve excellence, but striving to reach a screening rate greater than 75 percent is feasible for most. If the performance is satisfactory, an organization may wish to choose another measure and focus on other systems of care.
If the performance is unsatisfactory, consider adopting the measure and using it to monitor improvements to the care delivery system. An organization should understand that if a measure is adopted for improvement, ongoing and regular measurement is necessary to reach and sustain its organizational goals. Advanced discussions can be found in the Managing Data for Performance Improvement module.
Evaluate the baseline. Initially, a team compares its baseline to the performance it hopes to achieve. It is important to remember this gap in performance is defined as the difference between how the care processes work now (baseline) and how an organization wants them to work (aim). An organization may often modify its aim or timeline after analyzing its baseline measurement and considering the patient population and organizational constraints.
As an organization moves forward, the baseline is used to monitor and compare improvements in care over time. While it is important for an organization to stay focused on its aim, it is equally significant to periodically celebrate the interim successes.
Step 2 - Create a reliable way to monitor performance over time as improvements are tested.
An organization should standardize its processes and workflows to ensure the team collects and calculates performance data the same way over time. An organization should:
Note: The frequency of team meetings is not necessarily prescribed for success. Many successful teams meet once a week while others may meet bi-weekly when focusing their improvement efforts on any given measure. Success of these meetings is rather the output of the team members' active engagement in the meeting and being prepared to report on recent improvement findings. More information, including resources and tools for developing and implementing effective team meetings can be found in the Improvement Teams module.
Step 3 - Create systematic processes that allow an organization to analyze, interpret, and act on the data collected.
Having the data is not enough. Improvement work involves thinking about the data and deciding what to do based on that analysis. A QI team needs to put processes in place – team meetings, scheduled reports, and periodic meetings with senior leaders, to use the data tracked. This section describes how a QI team may accomplish the work of creating actionable plans based on the data collected. In Example 3.2: QI at White Cloud Health Group, the scenario illustrates how a team may use these concepts to act on its data.

Act: Make decisions based on data. Once a QI team has a better understanding of what the data means, efforts should be targeted to further advance the performance toward the aim. Often the decisions are made at the team level about what to tackle first. Then small tests of change can be accomplished to determine what improvements could be implemented to enhance performance. The practice of using small tests of change actually allows multiple changes to be tested simultaneously.
Note: An advanced discussion on how to use the data collected to advance an organization's improvement, including resources and tools to support improvement, can be found in the Managing Data for Performance Improvement module.
Example 3.2: QI Team at White Cloud Health Group (WCHG)
The Quality Improvement (QI) Team at WCHG worked diligently to improve cervical cancer screening over the past several months. The team focused some of its efforts on patient education and outreach, and internal tracking systems. But during the last three months, the performance remained the same at 50 percent, which was below its aim of having greater than 80 percent of women aged 21 to 64 years screened for cervical cancer.
Analysis: The team noted improvement initially. Newly implemented care processes and patient volumes seemed to be stable but performance was flat for the last three months.
The team leader asked for a list of those patients who had been seen in the past three months and were not up to date on screening–outliers for the measure. Further study of these specific cases found that two thirds of those patients were seen for other complaints and chart records do not indicate any discussion of or attempt to schedule cervical cancer screening.
Interpretation: Because there was initial improvement followed by several months of flat performance, the team leader looked for obvious changes in processes that would have an impact on performance, but found none. The team leader interpreted the data to mean that initial changes provided some improvement, but not enough to achieve its aim and have the desired impact. More work was needed. The team leader employed a common strategy to find additional opportunities; i.e., she looked at the population not in compliance (the outliers) for a common cause to be addressed. In this case, a common thread was that patients were coming in for care but were not offered testing.
This information allowed the team to consider ways to remind staff to assess women's need for screening. It looked at Sample Changes that Work (Table 4.2) for ideas then added suggestions based on its own patient population. The team decided to increase focus on the medical record. The electronic health record's (EHR) key user created an automatic reminder system in the EHR to appear for any woman aged 21 to 64 years.
Act: The information gathered from the analysis and interpretation of the data allowed the team to focus its next efforts. Since numerous patients were not being offered testing, the team targeted its efforts on improving its risk assessment processes. This enabled the team to focus on PDSAs to test changes specific to these areas and monitor its progress.
You will need Adobe Acrobat® Reader™ to view PDF files located on this site.
If you do not already have Adobe Acrobat® Reader™,
you can download here for free. 