At each part of the process, it is important to engage key stakeholders including staff, funders, patient advocates, and anyone else who is affected by the services you provide. You should ensure that the process is transparent to them and driven by their needs as well as external requirements and management goals.
Step 1: Evaluate organizational priorities
The purpose of performance measurement is to make progress toward specific objectives that support an organization’s overarching goals. Before selecting measures, you should take time to consider what your organization’s primary goals are.
In its report, Crossing the Quality Chasm: A New Health System for the 21st Century , the Institute of Medicine describes six aims for health care improvement, which you can remember with the acronym STEEEP. Health care should be:
Consider these and other goals that are relevant to your organization and then what action-oriented steps you should take to achieve them.
Step 2: Choose performance measures
Here are some tips for the process of selecting performance measures for your organization:
Include staff in the measure selection process
The measures should be meaningful to staff, because they may be involved in collecting the data and the data will be a reflection of the work they do. They should be able to clearly see how these measures can support their work.
Choose performance measures with the following characteristics:
Align with your organization’s goals;
Demonstrate a relationship to positive health outcomes;
Are under the control of the health care system (e.g. something that the health care system can effectively change)
Are reliable, valid, and standardized.
For more information on standardizing data processes, see Module 4: Managing Data for Performance Improvement.
Use established measures: NQF, HEDIS
To ensure that the measures you use meet the criteria described above, you can use established measures that have already been developed by other organizations.
Use available data sources
At least initially, measures should use data that your organization already collects or could collect using existing resources. Once the measurement process is more advanced, you could consider what additional resources would be necessary to gather the data that best captures what you want to measure. If you determine that a certain measure would be burdensome to implement, consider whether it can be justified by the potential for improvement.
Use a variety of measures
Using a mix of structural, process, and outcome measures will provide a comprehensive picture of your organization’s health care quality. Outcome measures are the most desirable, because they show direct impact on patient health. Structural and process measures can be used in cases where outcome measures are not available or feasible. These measures can also be used when trying to identify where your organization is falling short in achieving a patient outcome.
Step 3: Determine a Baseline
Once you’ve chosen your performance measures, you should use existing data to determine the baseline for that measure, which provides a basis for comparison in the future. If you do not have existing data, you should begin collecting the necessary data before making any system changes to improve quality. That way, you will establish baseline measures that can serve as a basis for comparison when you do begin to make changes to improve quality. This will help you determine whether a change you make is actually contributing to an improvement in quality.
Step 4: Evaluate Performance
Ways to Measure Quality. The Measurement of Health Care Performance by the Council of Medical Specialty Societies (PDF - 20 pages) describes three ways of measuring and presenting provider quality:
The simplest performance measure is a numerator/denominator equation that measures compliance:
The resulting percentage indicates that level of compliance.
Example: Numerator/Denominator Equation that Measures Compliance
Measure: Cervical Cancer Screening
This is well suited to measurement of processes. For this type of measure, it’s important that the process being measured has been proven to be of benefit to the patient. Also, the definition of the denominator is important because not all treatments are appropriate for all patients, so you do not want to count those patients in the group of those who “should NOT” have received a particular health care intervention. Finally, obtaining an accurate denominator has the capability to be challenging as there is not always perfect documentation on opportunities for care to providers. For example, in the cervical cancer screening measure shown above, the denominator includes women from 21 to 64 years of age. Another cervical cancer measure might include women age 24 to 64, reflecting a different view of the at-risk population.
Actual vs. Expected Performance
One challenge of using outcome measures is that providers only have control over some of the factors that influence a person’s health. While you can educate and monitor a patient, you cannot control whether that person follows the recommended regimen, such as eating healthy food, exercising, or taking medication consistently.
One way to account for this is to compare outcomes for a particular patient with the outcome that would be expected for a group of patients with similar underlying conditions and health status by using regional or national benchmarks.
For example, New York and several other State governments report provider performance in coronary artery bypass graft surgery by comparing the observed mortality rate for a hospital to the expected mortality rate for the mix of patients treated at that hospital. These reports also present a risk-adjusted mortality rate that compares a provider’s performance to the performance of all providers in the state as a whole. In order to calculate expected and risk-adjusted mortality rates, you need to account for the risk factors for each patient. New York State collects about 40 specific clinical characteristics per patient. The accuracy of the resulting reports is dependent on the accuracy of the information about these clinical conditions. These reports also depend on having effective, robust, tested statistical models that account for these patient characteristics in predicting mortality rates.
Performance Against a Benchmark
Under this approach, your organization would compare your performance to a regional or national benchmark, ideally one that represents optimal care, rather than average performance.
Once you have chosen measure(s), it’s important to define success at the beginning of the implementation process. The Turning Point Guidebook for Performance Measurement (PDF - 81 pages) provides three possible definitions of success for measuring performance in public health practice:
You should be explicit about which level you have chosen as your target for success. For example, if you set a challenge level target and that target is not reached, staff may become discouraged about the process if they thought it was a minimum level that had to be reached. If you set and reach a minimum level target, you may want to consider raising the target in the future.
Step 5: Report Results
When presenting results, you may want to tailor the information to different audiences to ensure that the information is presented in a way that is easy to understand. It is important to report performance measurement results internally so staff can be proud of where targets have been met and motivated to make changes where the data indicates a need for improvement. Also, it may be desirable or necessary to report results externally to stakeholders. When an organization reports results, it’s possible that consumers of the information will misinterpret the results or challenge your work. It is important that you provide context surrounding the measures. The context could include why certain measures were chosen or factors that influence the measures you’ve chosen. For example, budget cuts have an impact on whether you are able to meet the targets you’ve set.
Step 6: Develop a plan and make changes to improve performance
Step 7: Monitor performance over time
After you begin monitoring performance and making changes to improve performance in certain areas, it’s important to continue to measure the effects of those changes, to see whether they had the intended impact and ensure that they don’t result in unintended consequences in the targeted area or other areas of the care you provide. Module 3: Developing and Implementing a Quality Improvement Plan.
The Institute for Healthcare Improvement has a tool on its website that you can use to graph the data you’ve generated. There are also opportunities to track and graph progress through use of registries and electronic health records (EHRs).
You will need Adobe Acrobat® Reader™ to view PDF files located on this site. If you do not already have Adobe Acrobat® Reader™, you can download here for free.