A wide variety of measures are available to evaluate all aspects of health IT implementation: clinical outcomes measures, clinical processes measures, staff adoption and attitudes measures, patient knowledge and attitudes measures, workflow impact measures and financial impact measures.  Rural care providers should keep in mind that standard evaluation metrics may need to be adapted for use with smaller patient populations and in areas with scarcer resources.  The key is to create a useful and manageable set of measures that will be collected and evaluated at regular intervals.  

An evaluation should include both qualitative and quantitative metrics.  Frequently, evaluations of health IT implementations rely heavily on quantitative data.  While quantitative studies will provide evaluators with data on rates and percentage changes post-implementation, a single-pronged evaluation strategy may not be able to answer questions about usefulness and usability.  Qualitative studies enable evaluators to understand how well the user interacts with the new system.  Qualitative studies also speak to a larger audience because they are generally easier to understand.  Thus, a good health IT implementation evaluation should use a good balance of quantitative and qualitative measures.

Resources: 

• Evaluation of Clinical Information Systems Expectation and Experience (PPT - 156KB)
• AHRQ National Resource Center Evaluation Toolbox
• Leapfrog Computerized Physician Order Entry (CPOE) and Electronic Health Record (EHR) Evaluation Tools (PDF - 1.35MB)