Section
III: Overview of the Measurement
Framework and Identification of
Specific Measures for Cultural
Competence
I. Overview
of the Measurement Framework and Identification of Specific
Measures for Cultural Competence
Previous
sections of this report addressed the concept of cultural
competence and the spheres or domains in which it should
be manifested in health care settings and the topic areas
within each domain.
In looking towards the development of a measurement
profile for cultural competence, we now turn to a presentation
of a measurement framework, a structured way of thinking
about the types of measures relevant to health care settings.
We then use this framework to report findings from
the literature on specific indicators and measures that
might be applicable to measuring cultural competence.
A more detailed display of these findings is presented
in Attachment 3.
This section of the report:
-
Provides
a description of the methodology used to review the
measurement literature;
-
Presents
an overview of the measurement framework for cultural
competence;
-
Provides
a description of the current state of the field in identifying
measures and indicators of cultural competence by domain
and;
-
Discusses
the different levels at which indicators and measures
can be applied.
It
is important to note that the measurement framework and
specific indicators and measures reported in this
section do not represent the final stage in developing
a measurement profile of cultural competence.
Rather, this serves as a starting point that will
be further refined and informed by the input of key stakeholders
(e.g., Technical Expert Panel, conversations with other
experts, and site visits to health care delivery settings
that employ innovative approaches to delivering culturally
competent care). The description of measures used in the
field of cultural competence presented here is continually
being updated to capture a “universe” of performance measures
that will be further refined and developed.
In selecting from this “universe” of measures for
the evolving measurement profile, consideration will be
given to identifying measures that are meaningful, quantifiable,
practical, and are useful to the variety of stakeholders
interested in measuring cultural competence in health care
settings.
A. Measurement
literature review methodology
The
review of the measurement literature included a review of
assessment tools, standards and guidelines developed by
and for specific stakeholders, and articles focused on measuring
cultural competence. The literature identified either actual
performance measures, provided suggestions of potential
measurement areas, or described assessment tools. For the
purposes of this report, these disparate types of information
will all be classified as “measures.”
Sources
of measures or indicators of cultural competence were found
across a variety of literature, both federal and non-federal.
Federal agencies, including the Centers for Medicare and
Medicaid Programs (CMMP, formerly the Health Care Finance
Administration), the Department of Health and Human Services’
Office of Minority Health (OMH), and the Substance Abuse
and Mental Health Services Administration (SAMHSA), have
sponsored studies to identify measures of cultural competence
for various health settings. For example, CMMP sponsored
a report to develop a set of recommendations for measures
of cultural competence for managed care organizations that
provide care to Medicare and Medicaid beneficiaries under
contracts with CMMP or with State Medicaid agencies.
SAMHSA has also sponsored studies to develop measures by
which managed care organizations could be assessed on cultural
competence.
DHHS’ OMH sponsored a study that developed consensus and
standards regarding what constitutes cultural and linguistic
competence in health care service delivery (i.e., CLAS).
Other federal sources
include works by DHHS’ Health Resources and Services Administration
(HRSA). Among these are a set of measures applied by HRSA’s
Maternal and Child Health Bureau to review potential and
existing grantees for cultural competence.
The project team also reviewed criteria used in HRSA’s Cultural
Competence Works competition to recognize exemplary HRSA-funded
programs for their culturally competent services
and programs highlighted by HRSA’s Bureau of Primary Health
Care as innovative in delivering services that bridge the
cultural gap between providers and patients.
The innovative activities performed by these programs were
reviewed to identify indicators of cultural competence that
could serve as the basis for developing actual measures
of cultural competence. The project team also reviewed tools
developed by various authors and programs to assess the
cultural competence of personnel and to conduct organizational
assessments of cultural competence. For example, the National
Center for Cultural Competence (NCCC) has developed a series
of self-assessment tools for use by providers and organizations.
Authors such as Flores, G., Mason, J.L., Lavizzo-Mourey,
R., Mackenzie, E.R., and Tirado, M. have also developed
either tools or models for measuring cultural competence.
Refer to the Annotated Bibliography for a description of
these and other sources used to identify measures and Attachment
3 for a comprehensive listing of measures.
B. Overview
of the measurement framework
Considering
issues related to the structure, process, and outcome of
care is a well-established and useful way of thinking about
measurement in the health field, especially as related to
assessing quality of care.
Building on this formulation, we add an additional
area of assessment, “organizational viewpoint,” and use
four categories, described below, as a measurement framework
by which to describe the types of measures applicable to
cultural competence found in the literature. This framework
provides one way of addressing the questions. “How do you
know cultural competence when you see it?” You should know
it, in part, by an organization’s structures, processes,
outcomes and viewpoint.
Type of measure
-
Capacity/structure
measures: assess the organization’s
capability to support cultural competence through adequate
and appropriate settings, instrumentalities and infrastructure,
including staffing, facilities and equipment, financial
resources, information systems, governance and administrative
structures, and, other features related to organizational
context in which services are provided.
-
Process measures:
assess the content and quality of activities, procedures,
methods and interventions in the practice of culturally
competent care and in support of such care.
-
Impact/outcome
measures: assess the contribution of cultural competence
to the achievement of various levels of objectives (e.g.,
intermediate, ultimate), with respect to the provision of
care, the response to care, and the results of care.
-
Organizational
viewpoint measures: assess the values, principles, perspectives,
outlook, and organizational attitudes espoused and displayed
by an organization as these relate to cultural competence.
Vantage
point and usage
In addition,
we include in the measurement framework categories for examining
the different vantage points from which different measures
can be examined. Measures
can be categorized in terms of the stakeholders to whom
this measure would be of interest
(e.g., payers, providers, etc.), the purposes for
which these measures might be used (e.g., oversight, quality
improvement, formative evaluation), and the level of analysis
(e.g., individual, organization, health system, or societal)
for which the measure might be useful.
Exhibit III provides a graphical depiction
of the measurement framework which will be applied to identifying
indicators and measures of cultural competence by domain
and topic area. Attachment
3 includes the comprehensive, categorized list of measures.
Exhibit
III
Measurement
Framework Summary Table
|
Domain
|
Topic Area
|
Measures/ Indicators
|
Type
of measure
|
Vantage
point
|
Usage
|
Citation
|
|
Capacity/
Structure
|
Process
|
Outcome/ Impact
|
Org. Viewpoint
|
|
|
|
|
|
|
|
|
|
|
|
C. Measures
by Type
A
review of the literature found wide variation in the type
of measures currently available to assess the domains of
cultural competence, with a greater representation of process,
capacity/structure, and organizational viewpoint, than outcome/impact
measures. The
following summarizes this variation and the types of measures
are discussed according to their frequency. Refer to the
Annotated Bibliography for a description of documents used
to identify measures and Attachment
3 for a comprehensive listing of measures.
-
Process
measures: Process measures were the most prevalent
types of measures across all the domains. The communication
domain included process measures focused on the use
of interpreters and translated materials and the training
and staff development domain assessed the number of
staff trained in cultural competence. In the polices
and procedures domain, there were process measures focused
on ensuring cultural competence through ensuring proportionality
of racial/ethnicity providers to consumers and hiring
of adequate numbers of multicultural/multilingual staff.
The intervention and treatment model domain and family
and community participation domain included measures
that demonstrated the inclusion of consumer input in
the treatment development, planning and decision-making
processes. Exhibit
IV provides a sample list of measures found in the literature categorized
by domain.
Exhibit
IV
|
Sample
of Process Measures by Domain
|
|
Domain
|
Topic
Areas
|
Measures/Indicators
|
|
Communication
|
Interpreter
|
Yearly
updated directory of trained interpreters is available
within 24 hours for routine situations and within
one hour or less for urgent situations.
|
|
Communication
|
Interpreter
|
Percentage
of clients with limited English proficiency (LEP)
who have access to bilingual staff or interpretation
services.
|
|
Communication
|
Translated
materials
|
Percent
of clients who receive oral and written notices,
including translated signage at key points of
contact, in their primary language informing them
of their right to receive no-cost interpreter
services.
|
|
Communication
|
Linguistically
competent organization
|
|
|
Communication
|
Language
ability, written and oral of the consumer
|
Consumer
reading and writing levels of primary languages
and dialects is recorded.
|
|
Policies
and procedures
|
|
|
|
Policies
and procedures
|
Staff
hiring, recruitment
|
|
|
Training
and staff development
|
Training
and professional development
|
-
%
of staff with cultural competence training.
-
%
of staff attending ongoing cultural competence
training.
-
%
of ongoing cultural competence training completed.
|
|
Training
and staff development
|
Training
and professional development
|
Cultural
competence training is part of the credentialing
process for case managers.
|
|
Intervention
and treatment model features
|
Diagnosis,
care planning, referral
and treatment
|
Client
assessments are conducted in client’s primary
language.
|
|
Intervention
and treatment model features
|
|
Indicators
of culturally competent treatment plan in health
plan:
-
The
Treatment Plan reflects both consumer and
family involvement in its development and
agreement. The degree of family involvement
depends on the wishes of the consumer.
-
The
organization has a written policy and a demonstrated
practice linking families to advocacy and
education groups.
|
|
Intervention
and treatment model features
|
Input
into treatment decision and service quality
|
Indicators
of culturally competent treatment plan in health
plan:
-
The
Treatment Plan reflects both consumer and
family involvement in its development and
agreement.
-
There
is evidence in the Treatment Plan of the use
of racial/ethnic community services and resources.
-
The
Treatment Plan was developed with a culturally
competent clinician or consultation from such
a clinician
-
Consumer
and family involvement and investment in the
development of, and agreement with, the Care
Plan.
-
Culturally
defined needs addressed in the care plans
of consumers from various racial/ethnic
groups.
-
Leadership
by racial/ethnic Mental Health Specialists
in the care planning process for consumers
from various racial/ethnic groups.
|
|
Family
and community participation
|
Community
and consumer participation
|
Degree
to which families participate in key decision-making
activities.
-
Family
participation on advisory committees or task
forces
-
Hiring
of family members to serve as consultants
to providers/programs
-
Inclusion
of family members in planning, implementation
and evaluation of activities
|
-
Capacity/structure
measures: These types of measures were widely dispersed
across the various domain areas. Capacity/structure
measures found in the communication domain focused on
the amount of financial resources dedicated to interpretation
and translation services, the conduct of audits of provider
networks to measure the linguistic capacity of the provider,
and availability to mechanisms to disseminate culturally
competent information to consumers. In the policies
and procedures domain, capacity/structure measures focused
on whether organizations had governing boards or advisory
committees composed of diverse ethnic/racial /cultural
groups, used creative financing mechanisms to ensure
access to traditional healers in health plan benefit
packages, or had mechanisms in place to track consumer
grievances and complaints. Capacity/structure measures
were also found in the facility characteristics, capacity
and infrastructure domain with measures that assessed
whether organizations had adequate mechanisms in place
to maintain and track data on the ethnic/racial/cultural
composition of its service population. Capacity/structure
measures in the monitoring, evaluation and research
domain focused on whether organizations had the infrastructure
to: conduct self-assessments on cultural competence
through internal audits; conduct evaluations of health
plan decision-making based on enrollee ethnicity; or
conduct culturally competent community need assessments.
Exhibit V provides a sample
list of measures found in the literature.
Exhibit
V
|
Sample
of Capacity/Structure Measures by Domain
|
|
Domain
|
Topic
Areas
|
Measures/Indicators
|
|
Communication
|
Translated
materials
|
Allocated
resources for interpretation and translation services
for medical encounters and health education/promotion
material.
|
|
Communication
|
Linguistic
capacity of the provider
|
Ability
to conduct audit of the provider network which
includes the following components:
-
Languages
and dialects of community available at point
of first contact
-
#
trained translators and interpreters available
-
#
of clinicians and staff proficient in languages
of the community
|
|
Communication
|
Provide
information, education
|
-
Organization
has the capacity to disseminate information
on health care plan benefits in languages
of community.
-
Organization
has the capacity to disseminate information
and explanation of rights to enrollees.
|
|
Policies
and procedures
|
Grievance
and conflict resolution
|
Organization
has structures in place to address cross cultural
ethical and legal conflicts in health care delivery
and complaints or grievances by patients and staff
about unfair, culturally insensitive or discriminatory
treatment, or difficulty in accessing services
or denial of services.
|
|
Policies
and procedures
|
Grievance
and conflict resolution
|
Organization
has feedback mechanisms in place to track # of
grievances and complaints and # incidents.
|
|
Policies
and procedures
|
Planning
and governance
|
Composition
of the governing board, advisory committee, other
policy-making and influencing groups, and consumers
served reflects service area demographics.
|
|
Facility
characteristics, capacity, and infrastructure
|
Available
and accessible services
|
-
Transportation
is available from residential areas
to cultural competent provider
-
Organization
has the flexibility to conduct home visits
and community outreach
-
Cultural
competent services are available evenings
and weekends
|
|
Facility
characteristics, capacity, and infrastructure
|
Information
systems
|
Capacity
for tracking of access and utilization rates for
population of different racial/ethnic groups in
comparison to the overall service population.
|
|
Monitoring,
evaluation and research
|
Organizational
assessment
|
Ability
to conduct ongoing organizational self-assessments
of cultural and linguistic competence and integration
of measures of access, satisfaction, quality and
outcomes into other organizational internal audits
and performance improvement programs.
|
-
Organizational
viewpoint measures: Organizational viewpoint measures
were distinguished from process measures because they
reflect an organization’s attempt at changing the attitude,
values, belief, communication and culture of an organization
or understanding patient-specific values and beliefs.
These measures were found in the cultural sensitivity,
values and attitudes, and communication domains where
measures and assessment tools examined an organization’s
or provider’s ability to appreciate, value and respect,
the culture of others, how patients view their own health,
as well as how to communicate effectively with different
cultures. Organizations could demonstrate this ability
through the development of culturally competent mission
statements, the conduct of patient health beliefs inventory,
and the appreciation and use of various communication
styles. In the policies and procedures and training
and staff development domains, there were examples of
measures that assessed an organization’s commitment
to cultural competence through the incorporation of
cultural competence targets in staff incentive systems
and required demonstration by providers of respecting
and valuing diverse cultures. Exhibit
VI provides a sample list of measures found in the
literature.
Exhibit
VI
|
Sample of Organizational Viewpoint Measures by Domain
|
|
Domain
|
Topic
Areas
|
Measures/Indicators
|
|
Cultural
sensitivity
|
Culturally
sensitive encounters
|
Conduct
health beliefs inventory of patient to understand
the patient’s explanatory model for illness.
|
|
Values
and attitudes
|
Appreciate,
respect
|
Checklist
from the National Center for Cultural Competence
(NCCC) on Values and Attitudes. Indicate A= things
I do frequently, B= things I do occasionally,
C= things I do rarely or never.
-
I
recognize and accept that folk and religious
beliefs may influence a family's reaction
and approach to a child born with a disability
or later diagnosed with a disability or special
health care needs.
-
I
understand that traditional approaches to
disciplining children are influenced by culture.
-
I
understand that families from different cultures
will have different expectations of their
children for acquiring toileting, dressing,
feeding, and other self help skills.
-
I
accept and respect that customs and beliefs
about food, its value, preparation, and use
are different from culture to culture.
|
|
Values
and attitudes
|
Mission,
vision
|
Mission/vision
statement commit to the delivery of culturally
and linguistically competent services.
|
|
Values
and attitudes
|
