The javascript used on this site for creative design effects is not supported by your browser. Please note that this will not affect access to the content on this web site.
Skip Navigation
H H S Department of Health and Human Services
U.S. Department of Health and Human Services
Health Resources and Services Administration

A-Z Index  |  Questions? 

Testing for Improvement

Part 1: Introduction

Part 2: Methods for Testing Changes

Part 3: Successful Tests of Change

Part 4: Test for Improvement

Part 5: Implementation

Part 6: Supporting Information


Part 3: Successful Tests of Change 

Because all change does not necessarily result in an improvement, individuals who are developing the change should always be looking for a way to reduce the risks involved with the test while maximizing the learning experience. A few simple rules, which are described in the following subsections, can help the QI team gain the optimum learning experience with minimum risk.

Test on a Small Scale

The extent or scale of the test should be decided while considering factors related to risk. For example, a QI team should take into account the level of confidence it has that the change will result in an improvement. It should also contemplate the risks from a failed test. Very small tests are needed when the repercussions of a failed test could have a negative impact on an organization's finances, staff, patient care, or could even cause injuries. When risks are high, many teams seek guidance from organizational leadership.

Testing on a small scale reduces people's fear of making a change. When small scaled tests are not considered, the individuals developing the change often work to perfect it because of the potential consequence of failure. When planning a test of change, a QI team should consider building its knowledge through small scaled tests.

The Value of Multiple Test Cycles

Testing on a small scale leads to the use of multiple test cycles and builds knowledge that the change will, in fact, result in an improvement when it is implemented. Generally, people have a more difficult time committing to a change when an organization moves to a large scaled change instead of a small scaled change. An example of a small scaled change is testing the refined check- in process in a health care organization to improve patient flow at one facility as opposed to an organization-wide change. It is also suggested that volunteers can be used to test the cycle and reduce the burden on overworked staff.

Test Under Various Conditions

Whether a change will sustain improvement over time is challenging to predict because unforeseen situations occur, and conditions, policies, and organizations eventually change. A team may overcome this challenge by testing a change under various conditions, such as, different times, days, shifts, conditions, materials and populations. This in itself is difficult when an organization has limited resources or little knowledge of testing across a wide range of conditions, which is common in a health care delivery site. The following subsections offer suggestions for overcoming some of these challenges.

Collect Data Over Time

Effective change requires observation before and after its implementation and documentation of any differences. The process of collecting and documenting data results in a measurement; however, the mere act of gathering data can also cause a change, such as, interruptions to workflow. Measuring and recording such occurrences moves an organization to its desired improvement results.

Collecting data over time provides insights and learning by revealing trends and improvement opportunities. A graphical display of data tracked over time is a compelling tool in an improvement campaign. Additional information on methods for displaying data is in the Managing Data for Performance Improvement module.

Data collected and analyzed over time enables an organization to make informed decisions, incorporate backup plans, and prepare for uncertainties. Observing trends and patterns in any area, such as, length of patient stay, volume, visit demand, patient satisfaction, clinical outcomes, and staff turnover, is a prerequisite for achieving continuous improvement.

There are numerous tools available for a QI team to track data when performing a test cycle. One QI tool is the PDSA Worksheet. A QI team completes a PDSA Worksheet for each test conducted. If several different changes are tested, each change may go through several PDSA cycles. The team retains electronic or paper copies of the PDSA Worksheets for all changes tested, which helps it to understand why a PDSA did not work, or it provides additional testing opportunities.

Use Comparison Studies

A QI team may use simultaneous or paired comparison studies when it observes and analyzes two or more alternatives at the same time. Comparing alternatives studies the effect of outside or unforeseen events during the test.

Implement Random Sampling

Random sampling is a method that produces an independent and equal chance of selecting a participant, which is also known as a probability sample. Random sampling provides an objective snapshot of an organization's performance on a process or measure without the burden of collecting data from the entire population. This methodology can be complex, such as, a computerized sampling technique seen here Exit Disclaimer. or a simple mathematical equation. The random sample should be representative of the targeted population, with each group having an equal chance for selection.




You will need Adobe Acrobat® Reader™ to view PDF files located on this site. If you do not already have Adobe Acrobat® Reader™, you can download here for free. Exit Disclaimer