|
Tap 22 — TAPs <<<Documents<<<Home
This page contains links to external Web sites. The Treatment Improvement Exchange has no control over their content or availability.
CHAPTER VI
Quality Management
| Key issues in this chapter:
Framework for evaluating quality: structural, process, and outcome measures
Accreditation of MCOs and other providers
Report cards on MCOs
Measures of consumer and family satisfaction
Quality management systems for MCOs |
Although cost containment has been the dominant catalyst
behind the move to managed care systems, there is now strong
and growing interest in evaluating and improving the quality of
care. Quality of care has been defined as the degree to which
health services for individuals and populations increase the
likelihood of desired health outcomes and are consistent with
current professional knowledge (IOM, 1990a).
Driving the new interest in quality of care are improved
methods for assessing quality; increased competition for
contracts; concern about the effects of incentives in managed
care contracts for undertreatment and restricted access; and a
growing demand for accountability by Federal, State, and
county governments (IOM, 1996; Meyer, 1997; Rosenbaum et
al., 1997).
|
Quality standards can vary, but there is a consensus emerging about the aspects of service delivery
and treatment outcomes that should be measured to assess the system of care. The measures and
mechanisms chosen in different managed care initiatives to assess and improve the quality of care
and increase the accountability of managed care organizations (MCOs) will vary substantially
depending on the circumstances (IOM, 1996).
A purchaser of managed care can use a well-written contract to establish what standards of quality
it expects from an MCO and to specify how quality will be defined, monitored, and managed. One
of the most notable developments in purchaser contracting practices in recent years is the increased
reliance on establishing minimum standards of MCO performance and the development of
measures to determine whether MCOs are meeting those standards. The following language from
a contract between the State of Massachusetts and a health maintenance organization (HMO)
illustrates how a purchaser may provide an MCO with detailed contract language on both the
standard it expects the MCO to meet and the measure that the purchaser will use to determine
whether the MCO is meeting the standard (Rosenbaum et al., 1997):
Standard: "The HMO shall maintain an ongoing formal process to develop and
adopt clinical practice guidelines for conditions which have traditionally exhibited
high cost and/or high variation among Provider treatment methodologies.
Guidelines should combine the best available scientific evidence, outcomes, and
expert opinion in the specialty for which the guideline is being developed and
should be developed in conjunction with the Provider network to assure maximum
acceptance. The HMO shall further demonstrate that such protocols have been
implemented and that measurement of compliance with the guidelines is occurring.
Measure: The HMO shall select two clinical practice guidelines, including at
least one of the following: asthma management, prenatal care, substance abuse or
pregnancy, and a second guideline of the HMO's choice. For each of the two
clinical guidelines selected, the HMO shall document: (a) the process for the
development and dissemination of clinical practice guidelines to participating
Providers and members; (b) how the HMO incorporates scientific evidence, expert
opinions, and the opinions of network providers and expert clinicians outside of
the network into such guidelines; and (c) an ongoing evaluation process for the
purpose of updating and revising the clinical practice guidelines, as indicated by
current medical practice standards."
This chapter presents a framework that may help purchasers in thinking about quality. It also
discusses issues that purchasers should consider when developing provisions related to quality
management in requests for proposals (RFPs) and contracts. The chapter covers the following
topics:
- Framework for evaluating quality: structural, process, and outcome measures;
- Accreditation of MCOs and other providers;
- Report cards on MCOs, such the National Committee for Quality Assurance's
(NCQA) Health Plan Employer Data and Information Set (HEDIS 3.0);
- Consumer and family satisfaction measures; and
- Internal and external quality management systems for MCOs.
A. Framework for Evaluating Quality: Structural, Process, and
Outcome Measures
When purchasers are developing their strategies for using a contract with an MCO to ensure a
certain level of quality, they may find Donabedian's conceptual framework for evaluating the
quality of care useful (Donabedian, 1980, 1982, 1985). Donabedian identified three distinct
categories of measures used to evaluate the quality of care: (1) structural measures; (2) process
measures; and (3) outcome measures.
Process measures are currently the dominant type of quality measures in managed behavioral
health care contracts. As discussed below, however, there is growing interest in the use of
outcome measures, and MCOs are increasingly allocating resources to outcome measures to
maintain a competitive position in the industry. There is less emphasis on the use of structural
measures of quality in managed behavioral health care contracts than on the use of either process
or outcome measures. The use of each of these three types of measures of quality in managed
behavioral health care is discussed further below.
Framework for Evaluating Quality: Structural, Process, and
Outcome Measures
Structural Measures
Measures pertaining to the capacities, technologies, and infrastructure that
make up the structure of care (e.g., management information system,
number and types of staff, types of facilities, size and composition of an
MCO's provider network)
- Process Measures
Mesures pertaining to the administrative, clinical, and other processes by which care is provided
|
1. Process Measures of Quality
Process measures of quality--which in the field of behavioral health care are frequently referred
to as "performance measures"--are generally of three main types: (1) measures of administrative
processes (e.g., an MCO's customer service practices, and claims payment proficiency); (2)
measures of clinical processes (e.g., an MCO's compliance with patient placement criteria,
provision of relapse prevention training, continuity of care); and (3) measures of
financial/utilization processes (e.g., population penetration rates, utilization patterns, and claims
targets) (Oss, 1994).
Process measures typically measure activities that are believed to lead to positive outcomes. The
use of process measures is based on a belief that improving the process of care will yield improved
outcomes. Purchasers should recognize, however, that not all process measures relate to, or even
correlate with, outcome measures. Although the managed behavioral health care industry is
attempting to develop a consensus on the most valuable process indicators to use (see the April
1997 issue of Behavioral Healthcare Tomorrow), little research has been directed toward the
relationships between specific process measures and actual outcomes. Furthermore, little is known
about the efficacy of process measures to predict outcomes consistently across patient groups
(McLellan et al., 1996).
Some of the potential benefits and limitations of process measures of quality are identified in
Exhibit VI-1. To be most useful, process measures must be based on reliable, accurate, and
complete data and relate clearly to the outcome being sought. If process measures are used
properly, they can provide purchasers of managed substance abuse and mental health services,
MCOs, consumers, and others with valuable information that can facilitate efforts to monitor,
evaluate, and improve the quality of care.
Exhibit VI-1.
Process Measures: Benefits and Cautions |
Potential Benefits
- Measures provide operational and
measurable representations of
performance.
- Measures establish a base upon which to
set standards to be attained by the MCO.
- Measures increase overall accountability
of the MCO.
- Measures create opportunities to monitor
and improve performance over time.
- Measures provide a vehicle for
implementing quality improvement
initiatives or corrective action plans.
- Measures provide a structure for financial
incentives and sanctions.
- Measures establish a framework for
comparing performance in specified areas
across health care systems.
- Measures provide useful information to
individuals and businesses to aid in their
|
Cautions/Limitations
- Measures can easily be based upon data
that are not complete, accurate, reliable,
and/or valid.
- Measures based on data from an MCO's
management information system (MIS)
are subject to distortion,
misinterpretation, and, in some cases,
misrepresentation.
- The validity of many measures may be
questionable; some measures may be
imperfect representations of the issues
they attempt to address.
- Case-mix risk adjustments, critical in
accurately interpreting process and
outcomes measures, are seldom included.
|
2. Outcome Measures of Quality
Outcome measures of quality have begun to receive much attention in the behavioral health care
field. Although there is much evidence to support the effectiveness of prevention, treatment, and
rehabilitation for substance abuse and mental health disorders (Hubbard, 1989; IOM, 1990a;
McLellan et al., 1996; Simpson & Sells, 1990) evidence developed in other States or systems of
care is often not perceived by State legislatures and the public as applicable to local systems of
care. Consequently, the purchasers of behavioral health care services are often challenged to
defend the expenditures to a skeptical legislature and public and to demonstrate that such services
are resulting in positive outcomes and measurable societal benefits. Furthermore, some MCOs are
beginning to propose outcome-based reimbursement arrangements as a means to circumvent what
they perceive to be purchasers' overemphasis on price factors (Meyer, 1997).
Neither the mental health nor the substance abuse fields have achieved a high degree of consensus
about specific outcome measures to use and what indeed constitutes treatment success.(1) Thus, for
example, the addiction treatment field continues to struggle with the controversial issue of whether
treatment goals for persons with substance use disorders should be directed toward abstinence
from all substance use or whether reduced use can be a viable goal for some populations.
In the mental health field, a similar struggle is sometimes noted over whether symptom reduction
or symptom recovery ("cure") should be the ultimate goal for those with mental illnesses. For an
adult with severe mental illness, individually determined goals might include securing a job,
regular housing, enough income to live on, and a social network. For a child with a serious
emotional disturbance, goals might include an improved level of functioning in school, at home,
and with peers. For an individual with severe substance dependence, goals may include decreased
quantity and frequency of substance use, reduced symptoms, reduced involvement with the
criminal justice system, lower medical costs, and improved vocational or employment status (as
currently measured in the States of Minnesota, California, and Oregon) (McLellan et al., 1996).
Exhibit VI-2 lists some research-based behavioral health care outcome measures from an Institute
of Medicine (IOM) report Managing Managed Care (IOM, 1997). (Appendix G lists examples
of other potential outcome measures in behavioral health care, along with several potential process
measures.)
|
Exhibit VI-2.
Research-Based Behavioral Health Care Outcome Measures |
| 1. Percentage of individuals who show reductions in symptoms;
2. Percentage of patients who show improved functioning;
3. Number of patients who return to work;
4. After return to work, average number of consecutive days worked without absences;
5. For children and adolescents, number who return to school;
6. For children and adolescents, average number of consecutive days in attendance at school;
7. Number of clients returning to earlier levels of treatment;
8. Number of clients who are able to live independently in the community;
9. Number of clients whose substance-free status is validated through regular breath and urine
testing; and
10. Number of clients who increase participation in community activities.
SOURCE: IOM, Managing Managed Care: Quality Improvement in Behavioral Health, 1997. |
The purchasers of managed behavioral health care should be prepared to include a well-defined
plan for outcome measurement in the contract. Given that the development and use of outcome
measures in managed behavioral health care systems are still in the early stages, a purchaser should
use caution in selecting the outcome measures. Furthermore, the cost of obtaining outcome data
must be weighed against the value of the data. Factors to take into consideration include the
validity of the data and the reliability and accuracy of the information they provide, costs of data
collection, its utility for making corrective decisions, and consistency with confidentiality
requirements. In addition, outcomes must be selected that can be precisely measured, such as
psychiatric hospitalization readmission rates, or the percentage of increase in school attendance
among children, or a decrease in criminal justice system involvement. If available resources are
very limited, the purchaser may wish to consider implementing the outcome measurement system
in stages; building on it as resources become available and experience is acquired.
|
A Resource for Information on Outcomes Monitoring
Purchasers interested in developing outcome-measurement systems for substance abuse and
mental health treatment services may wish to review the Treatment Improvement Protocol
(TIP) developed by the Center for Substance Abuse Treatment (CSAT) titled Developing
State Outcomes-Monitoring Systems for Alcohol and Other Drug Treatment (CSAT, 1995a).
This TIP describes useful principles and techniques for designing outcomes-monitoring
systems, establishing policies and viable infrastructures for monitoring and evaluating the
outcome measures designed, and selecting a feasible number of outcome measures that best
reflect the goals of the contract. Examples of outcomes-monitoring systems developed in
various States are described in the TIP. |
In selecting outcome measures, the purchaser should be aware that different stakeholders--for
example, the legislature, the courts, the health care system, consumer and family groups, various
social and public health agencies--may take a substantial interest in the types of outcome measures
used. Their goals and expectations, which often vary dramatically and may conflict, can result in
strong pressures on the purchaser of managed behavioral health care services to devote finite
resources to an expansive, all-inclusive set of measures that does not include a measure of whether
individuals receiving care actually get better.
Clinical outcomes for individual patients (e.g., improvements in cognitive functioning, reduction
in cocaine use) may not be of primary interest to some stakeholders. In an analysis of outcomes
desired by various stakeholders, McLellan and Weisner (1996) found that few stakeholders sought
"direct" treatment outcomes; rather, they called for broader societal benefits such as "reduced
crime, improved health status, prevention of human immunodeficiency virus infection, reduction
of unsafe sexual practices, improved employment, and improved family functioning." Because
measurement of outcomes is a complex undertaking and can require significant financial and staff
resources, only a limited number of measures may be feasible. Alternatively, purchasers may want
to limit the tracking of certain outcomes to specific population groups rather than all enrollees.
3. Structural Measures of Quality
Structural measures of quality refer to a wide range of "tangibles" that are part of the health care
system and are believed to be positively related to outcome, or at least increase the chances that
outcome will be positive. Structural measures of quality include capacities, technologies, and
infrastructure that, when present, would be perceived by most to be related to higher quality
processes and outcomes of treatment. Thus, for example, better facilities, telecommunications,
office equipment, a management information system (MIS), and staffing in a hospital would seem
to enhance the institution's capacity to provide high-quality services, all things being equal.
Licensing of facilities usually relies heavily on structural measures of quality. Requirements
concerning the size of rooms and hallways, sanitation, medical records, fire detection, staffing, etc.
ensure that the physical facilities and clinical support systems are adequate to support the program.
Although meeting these requirements does not ensure a good treatment outcome, not meeting them
could potentially create conditions that would ensure a poor outcome, for example, an undetected
fire. Accreditation standards (discussed later in this chapter) also consider structural features, such
as staff training, staff size, medical record organization and content, infection control, and quality
assurance/improvement systems. These standards are extremely important in encouraging the
development of systems that will actually improve outcome.
Facilities and programs are increasingly understanding the value of nicely landscaped grounds,
good lighting, and decorative art. These structural features help create the perception, or an
expectation, that the program has the potential to provide quality outcomes. These "placebo"
effects may be quite effective at improving the potential for good outcomes.
Purchasers should specify what structural measures will be required as part of the managed care
system under development. For example, having comparable facilities for serving private- and
public-pay clients, living in different areas, might be required.
4. Quality Measures for Substance Abuse and Mental Health
Services: Similarities and Differences
It is important for purchasers of managed behavioral health care services to distinguish between
quality measures that can be applied effectively to both substance abuse and mental health
populations and those that are appropriate for one population or the other (as well as those that are
appropriate for children). While many process and outcome measures can be applied both to
mental health and to substance abuse treatment, an optimal set of measures for individuals with
severe mental illness will differ in many ways from a set of measures for individuals with
addictive illness. Similarly, the most appropriate measures for children and adolescents may be
quite different from those used for adults.
An IOM committee that studied managed behavioral health care identified the following general
outcome areas that can be applied to a broad cross-section of consumers of both substance abuse
and mental health treatment services (IOM, 1996): improvement in employment or vocational
status, medical status, family and social functioning, legal problem status, cognitive functioning,
and quality of life.
Expected outcomes and the processes necessary to achieve given outcomes may differ depending
on an individual's diagnosis, the severity of the problem, the drug(s) of abuse, age, and the stage
of illness or recovery. Thus, for example, the purchasers should not expect all process and
outcome measures to be equally applicable for severely mentally ill adults, individuals with long-term heroin addiction, adults with major depression, children with severe emotional disturbances,
and adolescents with comorbid mental and addictive disorders.
Structural, Process, and Outcome Measures of Quality. Purchasers may
wish to address the following in RFPs and contracts:
Include the input of consumers when establishing measures that are
relevant to them; patient satisfaction data alone are insufficient and easily
"gamed" by surveyors.
Ensure that outcome measures are measurable and limited enough so they
can be tracked and used to improve treatment.
Identify, precisely define, and/or provide all performance measures that the
MCO is required to develop and/or implement (e.g., claims paid, clinical
records, satisfaction, and per capita cost analysis).
Specify performance measures and standards to which incentives or
sanctions are attached.
Clarify purchaser quality monitoring activities, including reviews of
performance measures, clinical records, grievance and appeal data,
enrollment/disenrollment data, terminations, utilization and financial data,
and management systems and procedures.
Require that the MCO have or develop the technological capacity to fulfill
all contractually stipulated responsibilities related to the collection and
measurement of performance indicators and outcome measures.
Define the process by which disagreements, misinterpretations, or
ambiguities about measures will be resolved.
Delineate the MCO's responsibilities about the integrity of the data and
reporting requirements.
Establish standards for validation and verification of reported performance
data.
Require compliance with State and/or Federal reporting requirements.
Specify the processes and mechanisms by which the quality management
staff and structures of the MCO and purchaser will communicate.
Require that funds be set aside by the MCO to conduct outcome studies.
Specify the required capabilities, flexibility, and resource intensity of the
MCO's outcomes measurement system.
Require the MCO to collect, analyze, and develop periodic reports with
data on specified outcomes for specific subpopulations.
Specify appropriate outcome measures for different populations (e.g.,
persons with long-term mental illness, individuals who are homeless and
abuse substances, children with severe emotional disturbances).
Require that the MCO collaborate with the State, county, and/or local
mental health and/or substance abuse authorities in outcome-related
evaluation initiatives.
Require that the MCO make staff, data, and other relevant resources
available to the evaluators representing the purchaser.
Establish guidelines for phasing in outcome measurement, using
established baseline measures wherever feasible.
B. Accreditation of MCOs and Other Providers
The purchaser of managed behavioral health care services must decide whether to require an MCO
to have specific accreditation to be eligible to bid for the contract and whether to require
accreditation for all or some types of network providers. A number of organizations systematically
assess the clinical and administrative operations of individual providers, MCOs, and other health
care delivery systems and institutions and accredit them if they comply with predetermined
standards. Many of the standards used in accreditation are structural measures of quality (see
earlier discussion in this chapter). Public and private payment programs often require accreditation
as a condition of payment for covered services.
The activities of four of the most well-known national accrediting organizations that review
behavioral health care organizations--the Joint Commission on Accreditation of Healthcare
Organizations (JCAHO), the National Committee for Quality Assurance (NCQA), the Commission
on Accreditation of Rehabilitation (formerly the Commission on Accreditation of Rehabilitation
Facilities, or CARF), and the Council on Accreditation of Services for Families and Children
(COA)--are described in Exhibit VI-3.(2) In many cases, accreditation by these organizations has
become a necessary, but not sufficient, requirement to be competitive in a managed care
environment. However, very few MCOs serving persons with substance abuse and mental health
disorders have been accredited by these organizations.
Exhibit VI-3.
National Accreditation Organizations |
Joint Commission on Accreditation of
Healthcare Organizations (JCAHO):
Accreditation Programs and Services:
Behavioral Health Care Accreditation
Program
One Renaissance Boulevard
Oakbrook Terrace, IL 60181
Phone: (630) 792-5791
Fax: (630) 792-5644
Website: www.jcaho.org
|
- Accredits staff model delivery systems in
organizations providing mental health,
substance abuse, and mental
retardation/developmental disabilities
services.
- Developed accreditation guidelines for
wide variety of network-based services
systems, such as MCOs, health
maintenance organizations (HMOs),
preferred provider organizations, and
provider-sponsored networks.
- Currently evaluates and accredits more
than 15,000 health care organizations.
|
National Committee for Quality Assurance
(NCQA): Behavioral Health Accreditation
Program
2000 L Street, NW, Suite 500
Washington, DC 20036
Phone: (202) 955-3500
Fax: (202) 955-3599
Website: www.ncqa.org
|
-
Accreditation focuses primarily on the
highest organizational levels of an MCO's
structure.
- Develops quality standards and promotes
improvement in quality of care provided in
managed care plans and has recently
issued accreditation standards for
behavioral health care.
|
Exhibit VI-3.
National Accreditation Organizations (cont'd.) |
Commission on Accreditation of
Rehabilitation (CARF) (Formerly the
Commission on Accreditation of Rehabilitation
Facilities)
4891 East Grant Road
Tucson, AZ 85712
Phone: (520) 325-1044
Fax: (520) 318-1129
Website: www.carf.org
|
- Accredits individual substance abuse
programs, mental health programs, and
community-based rehabilitation programs
treating individuals with chronic mental and
addictive disorders.
- Built on strong consumer-centered
philosophy that encourages active consumer
involvement in all activities, including
collaborating in the development of
individual treatment plans.
- Developed program standards regarding
access issues.
- As of June 1997, over 13,000 programs
accredited.
|
Council on Accreditation of Services for
Families and Children (COA)
120 Wall Street, 11th Floor
New York, NY 10005
Phone: (212) 797-3000
Fax: (212) 797-1428
Website: www.nn4youth.org |
- Created quality standards for more than 50
types of services more closely related to
behavioral health care and social services
than to the medical model of delivery.
- Services include individual outpatient
programs, day treatment programs,
developmental disabilities services, day
care/foster care for children, and numerous
others.
- Accredits about 3,000 social service
programs and 1,000 behavioral health
programs
. |
Apart from national accrediting organizations, States accredit and license organizations, such as
those based on the PACT (Program for Assertive Community Treatment) model, consumer-run
clubhouses, and those providing partial hospitalization, intensive outpatient, and intensive case
management services for persons with severe mental illnesses and substance abuse disorders.
Accreditation of these organizations generally involves systematic assessments of clinical and
administrative structures. In some cases, jurisdictions have moved to or are experimenting with
national accreditation as a replacement for State or local licensing (e.g., Michigan, North Dakota
pilot program, and the Veteran's Affairs mandate for CARF accreditation for all behavioral health
and vocation programs under their Prescription for Change initiative).
Mandating the accreditation of MCOs and/or service providers may have both benefits and
drawbacks for a purchaser of managed substance abuse and mental health services. The obvious
advantage of requiring MCOs and/or service providers to obtain this "seal of approval" is that it
provides reasonable assurance that the clinical and administrative operations of the accredited
organization have met basic standards of quality in terms of process and outcomes. However,
several problems related to current accreditation practices have been noted (Horvath and Kaye,
1995; IOM, 1996; Bazelon Center for Mental Health Law, 1997):
The costs of going through an accreditation process can be prohibitive, especially
when personnel costs and time are considered and requiring accreditation could
eliminate viable MCOs or programs from bidding on the contract.
There is redundancy among accreditation standards from different accrediting
agencies. National MCOs, managed behavioral health organizations (MBHOs),
and other health care delivery systems often obtain accreditation from several
different agencies to meet the requirements of various purchasers, which results in
costly duplication of effort.
Private accreditation standards are frequently vague and do not take into account
many issues particularly important in the public sector (e.g., access, grievance
procedures, and enrollee rights); additional standards may need to be developed.
Accreditation standards often duplicate State licensing requirements.
Current accreditation standards for behavioral health care are generally less refined
than general health care measures and often emphasize structural and
administrative process issues that do not adequately address the quality of clinical
services.
In response to these and other issues, the Institute of Medicine (IOM, 1996) has questioned the
utility and validity of accreditation in the current health care environment and has encouraged
accreditation agencies to focus their standards on the most relevant issues, to examine the use of
"deeming" (accepting another entity's standards and/or review process in place of one's own in
some or all areas), and to consolidate the multitude of accreditation standards from various
organizations to reduce overlap and redundancy.
Accreditation. Purchasers may wish to address the following in RFPs and contracts:
Specify whether the MCO, network providers, and/or subsets of network
providers must be accredited.
Specify the type or types of accreditation required.
Establish any timeframes within which an organization(s) must obtain
accreditation, if not already accredited.
Specify the consequences of losing accreditation, having the accreditation
downgraded, or not achieving it within a specified period.
Ensure compliance with relevant State licensing requirements and other
applicable standards for MCOs, HMOs, and substance abuse and mental
health treatment providers.
Specify policies regarding "deeming," and clarify how providers can
request that an existing accreditation be deemed acceptable.
C. Report Cards on MCOs
Purchasers are increasingly using "report cards" to monitor the performance of MCOs. Report
cards present systematically organized data on standardized sets of measures, often with
associated minimum standards, about MCOs and/or health care providers. Purchasers, MCOs,
consumers, and others can then examine and compare objective information about the clinical and
administrative processes of different MCOs.
Widespread use of report cards on MCOs promises to increase standardization of measures and
data-gathering procedures across a variety of behavioral health care systems. It should be noted,
however, that most report cards are designed for adult services rather than services for children or
adolescents.
Report cards can be developed by any purchaser or organization. Four nationally recognized report
cards that are increasingly being used by purchasers to assess the quality and performance of those
MCOs managing behavioral health care services are these (see Exhibit VI-4):
- The Health Plan Employer Data Information Set (HEDIS 3.0) prepared by the
National Committee for Quality Assurance (NCQA);
- Performance-Based Measures for Managed Behavioral Health Care Programs
(PERMS) prepared by the American Managed Behavioral Healthcare Association
(AMBHA);
- County Behavioral Healthcare Measures prepared by the National Association of
County Behavioral Health Directors (NACBHD); and
- The Mental Health Statistics Improvement Program (MHSIP) prepared by the
Center for Mental Health Services (CMHS).
Industry norms such as HEDIS and PERMS are developed on the basis of data from the existing
administrative database (e.g., telephone answering rates) that does not necessarily include data on
certain types of outcomes that may be important to purchasers. Improvements in social
functioning or employment, for example, are not part of an administrative database. Should a
purchaser want to obtain data on these types of outcome measures, it must be willing to invest in
both infrastructure development and the collection of data. The acquisition of standardized and
reliable information about these outcomes would require a significant commitment and investment
on the part of the purchaser.
Exhibit VI-4.
Four Nationally Recognized Report Cards |
Health Plan Employer Data Information Set
(HEDIS 3.0)
National Committee for Quality Assurance
(NCQA)
2000 L Street, NW, Suite 500
Washington, DC 20036
Phone: (202) 955-3500
Fax: (202) 955-3599
Website: www.ncqa.org
|
- The most widely used set of general health
care performance standards.
- Incorporates measures designed for
Medicaid and other populations.
- Includes 71 measures in the reporting set
of measures (to be reported in 1997), plus
33 measures in a "testing set".
- Categorized into eight domains:
accessibility, treatment effectiveness,
stability, satisfaction, cost, degree of
informed choices, use of services, and
descriptive information.
- A handful of behavioral health measures
are in the reporting set and more are in the
testing set, including:
- Outpatient followup after
hospitalization,
- Number of providers accepting new
patients,
- Utilization of different specified levels
of care,
- 90-day inpatient readmission rates,
- Several related items in a health plan
member satisfaction survey.
- Precisely defined measures and standards.
|
Performance-Based Measures for Managed
Behavioral Healthcare Programs (PERMS)
American Managed Behavioral Healthcare
Association (AMBHA)
700 13th Street, NW, Suite 950
Washington, DC 20005
Phone: (202) 434-4565
Fax: (202) 434-4564
Website: www.ambha.org
|
- Includes 23 industry-standardized
measures designed specifically for
substance abuse and mental health
treatment.
- Measures access, consumer satisfaction,
and quality.
- Quality measures include systems
effectiveness for substance abuse
treatment and continuity of care for
patients in treatment and detoxification.
- Precisely defined measures.
|
Exhibit VI-4. (cont.)
Four Nationally Recognized Report Cards |
County Behavioral Healthcare Measures
National Association of County Behavioral
Health Directors (NACBHD)
6000 Lamar Street, Suite 130
Mission, KS 66202
Phone: (913) 384-3535
Fax: (913) 591-5653
Website: www.naco.org/affils/afflpres.htm#16
|
- Developed to monitor and improve the
quality of substance abuse and mental
health services.
- Designed to be particularly relevant to
service delivery at the county level.
- Categorized into five domains of three to
five measures each, with recommended
tools to obtain those measures:
- Access,
- Consumer satisfaction,
- Consumer outcomes,
- Intersystem outcomes, and
- Utilization. |
Mental Health Statistics Improvement
Program (MHSIP)
Center for Mental Health Services (CMHS)
Parklawn Building, Room 15-105
5600 Fishers Lane
Rockville, MD 20852
Phone: (301) 443-0001
Fax: (301) 443-1563
Website: www.mentalhealth.org |
- Measures for comparing and evaluating
access, appropriateness of treatment,
outcomes, and prevention in the delivery
of mental health services to persons with
severe mental illness.
- In testing and development in 20 States.
- Instrument reflects significant consumer
input into design and content.
|
D. Measures of Consumer and Family Satisfaction
Measurement of consumer and family satisfaction with behavioral health care services is
increasingly becoming an integral component of quality monitoring efforts. Purchasers are
therefore including these measurements as a quality-monitoring tool in contracts with MCOs.
Indeed, the voices of consumers, their families, and others who support consumer recovery can
provide valuable information to the purchaser about the strengths and weaknesses of an MCO and
its network of providers. Consequently, a growing number of purchasers are attaching financial
incentives or sanctions to MCO performance in this area (Petrila, 1996; Ruggeri, 1994).
Assessment of consumer satisfaction can empower consumers and send a strong message to
providers that such a focus is valued. Yet despite the value of consumer satisfaction measures for
assessing quality, the purchaser should be aware that many challenges are associated with such
measures and that findings related to satisfaction may need to be interpreted with caution. Some
evidence suggests that even when MCOs primarily serving Medicaid beneficiaries have weaker
ratings on performance measures for quality and access, enrollees are still as likely to indicate
satisfaction with these plans as are individuals enrolled in private insurance plans that have
significantly higher performance ratings (Rosenbaum et al., 1997).
A recent literature review of the measurement of consumer satisfaction summarized the challenges
currently associated with measurement of consumer satisfaction. These challenges are identified
in the box below.
Challenges in Measuring Consumer Satisfaction
- There is no widespread consensus about the definition and appropriate measurement of
satisfaction.
- Very complex relationships exist between consumer satisfaction and quality of care, consumer
demographics, provider profiles, and treatment outcomes.
- Design and implementation of surveys for special populations (such as those with substance
abuse and mental health problems treated in the public sector) are far from exact sciences.
- Tools used to measure satisfaction with behavioral health treatment have been less rigorously
evaluated for reliability and validity than tools for general health care.
- Satisfaction surveys often yield high and undifferentiated levels of satisfaction.
- Consumer responses are sensitive to the method of administration and can be affected by
social desirability (the tendency of respondents to answer in a way that would please the
person administering or issuing the survey).
- Survey questions and results are often more a reflection of basic access (e.g., answering
phones) and administrative efficiency of the MCO than of the quality of clinical services.
- Surveys require sufficient brevity to generate adequate response rates, yet they must be
specific enough to suggest actions that may be taken to improve the particular situation.
- Some researchers have found no relationship between consumer satisfaction and quality of
care and some have even found an inverse relationship.
|
Given the challenges of conducting consumer satisfaction surveys, many purchasers are now
including alternative mechanisms to assess the experience and satisfaction of enrollees with the
MCO and its network providers. These include focus groups, in-depth interviews from samples
of enrollees, and face-to-face surveys using consumers or family members as interviewers.
Consumer satisfaction teams can also be used. These teams, which are consumer-run programs,
serve as a monitoring and feedback mechanism in the system and can be given the authority to
provide feedback to upper management of the MCO and/or purchaser. In addition, there is a
current effort to adapt for behavioral health care the Consumer Assessment of Health Plan Study
of the Federal Agency for Health Care Policy and Research (AHCPR).
The purchaser may also wish to consider periodically assessing, or having the MCO assess, the
satisfaction and opinions of key providers in the MCO's network as well as other organizations
with which the MCO has an ongoing relationship. In the private sector, such mandated provider
surveys are generally conducted by external parties. Surveyed organizations may include network
providers; State and county mental health, substance abuse, social service, and other government
agencies; consumer and family advocacy groups; and any other organization whose perspective
would help illuminate the strengths and weaknesses of the MCO's performance. If the purchaser
wants the MCO to conduct the surveys, the purchaser should define its role in approving the
surveys and controlling how survey results can be used in marketing initiatives.
Measuring Consumers' Satisfaction. Purchasers may wish to address the
following in RFPs and contracts:
Require the MCO to include input from providers, consumers, families, and
other stakeholders in the development, implementation, evaluation, and
refinement of satisfaction measurement systems and tools.
Require that all satisfaction measurement tools and processes have the
approval of the purchaser before implementation.
Ensure that any surveys used, and processes followed, are methodologically
sound (e.g., use adequate sampling and weighting techniques).
Determine the frequency of surveys based on the intended use of the
information.
Specify that a standardized base of questions, data elements, and
measurement processes (including guidance on sample size, timeframes,
and response rate) must be used on surveys and other tools to facilitate
comparisons across all levels of care (additional questions and data
elements for different levels can be added to this base to allow programs
and health care systems to individualize their tools).
Require the MCO to supplement any surveys it conducts with additional,
possibly more meaningful, mechanisms for understanding consumer views,
such as focus groups or in-depth interviews conducted by family members
or consumers.
Require the use of nationally standardized report cards (e.g., HEDIS,
PERMS) or the use of selected consumer-oriented measures, to allow
comparison to other health care systems.
Require periodic assessment of complaints, disenrollments, and requests to
change facilities and providers.
Require systematic followup of survey findings, including identification
and investigation of sources of dissatisfaction, development and
implementation of a corrective action plan, dissemination of findings to
providers, enrollees, and legislative oversight committees, and a
reevaluation of the survey process.
Require regular reviews of the MCO's performance by conducting unbiased
surveys of providers and other stakeholders.
Ensure that providers and enrollees who provide negative feedback about
the MCO are not threatened or penalized in any manner for their action.
E. Quality Management Systems for MCOs
A promising strategy for establishing and maintaining a managed care system based on quality
principles is to use the RFP and contract to ensure that the MCO has a well-supported internal
quality management (QM) system. The resources supporting and sophistication of an MCO's
internal QM efforts vary tremendously among MCOs. However, MCOs are increasingly adhering
to NCQA's comprehensive standards for QM. NCQA's accreditation summary reports provide
detailed guidance to MCOs who seek accreditation.
1. Internal Quality Management
All Medicaid managed care contracts require the MCO to have an internal QM system.
Rosenbaum et al. (1997) found that most State Medicaid programs set out extensive specifications
for the structure and performance of such systems, not leaving it to the MCO's discretion.(3)
Purchasers of managed behavioral health care should make sure that the RFP and contract clearly
state the expectations, capabilities, and responsibilities of the QM system, even in cases where the
details for developing and/or refining the MCO's internal QM system are primarily the
responsibility of the MCO. Provisions requiring mandatory compliance with NCQA's standards
will give the purchaser leverage to require changes in the MCO's internal QM system if needed.
Other contract provisions may require the establishment of an MCO QM team; require an
independent quality review council composed of providers, consumers, and other stakeholders; or
require the MCO to develop and maintain a continuous quality improvement program.
Internal Quality Management. Purchasers may wish to address the following in
RFPs and contracts:
Require the MCO to create a quality management team, accountable to the
governing body of the organization.
Develop, implement, and systematically refine a comprehensive quality
management plan that is consistent with the RFP and all applicable State
and Federal requirements.
Incorporate a process for continuous improvement of quality across QM
activities.
Establish a system to monitor the completeness, accuracy, and
appropriateness of service authorization decisions and ensure compliance
with the utilization control requirements of the U.S. C.F.R. 456.
Develop a comprehensive set of procedures for network providers and
specify the MCO's responsibilities for management and reporting of serious
incidents (i.e., deaths, suicide attempts, injurious assaults on provider
premises, use of seclusion or restraints, medication errors, felony arrests,
and convictions).
Develop and implement systematic procedures for monitoring, managing,
and improving the quality of individual network providers and of the
provider network as a whole.
Require the MCO to establish, facilitate, and empower a community-based
monitoring council that includes providers, consumers, and family
members.
Require the MCO to actively cooperate with any external quality
monitoring team as it develops, implements, and monitors quality
improvement goals, objectives, and activities for the provider network and
the MCO.
2. External Quality Management
Some purchasers may opt to supplement the MCO's QM program with an external agency that
specializes in monitoring and auditing quality in health care settings to increase credibility and
accountability (Huskamp, 1996).(4)
HCFA requires that all States administering Medicaid managed care waiver programs under
Section 1915 or Section 1115 of the Social Security Act provide for external quality assurance
oversight.
1. In September of 1997, the American College of Mental Health Administration (ACMHA), concerned
that the rapid proliferation of process/performance measures and strategies is counterproductive for the
behavioral health care field, held a summit in Santa Fe, New Mexico, to address outcome measurement.
Representatives attended from such organizations as the Substance Abuse and Mental Health Services
Administration (SAMHSA), the Institute of Medicine (IOM), the National Alliance for the Mentally Ill
(NAMI), the Commission on Accreditation of Rehabilitation (CARF), the National Committee for
Quality Assurance (NCQA), the American Managed Behavioral Healthcare Association (AMBHA), and
the National Mental Health Association (NMHA). Participants were divided into working groups
reflecting the following domains: prevention, access, process/performance, outcomes, and structure.
Over the course of 2 days, each group produced a set of core values for each domain, with
recommendations for next steps. Participants are actively collaborating with key agencies to refine the
work initiated at the summit.
2. For more information, see the Institute of Medicine report (1996).
3. For a detailed analysis of this issue, see Table 5.1, Rosenbaum et al. (1997), which presents State
Medicaid managed contract provisions addressing quality assurance.
4. The Health Care Financing Administration (HCFA) mandates that all State Medicaid waiver programs
provide for external quality assurance programs to assist in managing, collaborating with, and/or
monitoring the MCO's quality improvement systems without any vested interest in the outcome.
Previous |
Table of Contents | Next Top of Page
Last Updated 11-7-02
|