TIP 14 — TIPs — Documents — Home

Chapter 2—Political Considerations in Planning a Successful OMS

As noted in the introduction to this Treatment Improvement Protocol (TIP), current efforts by single State agencies (SSAs) to address the issue of outcomes accountability are occurring in the context of healthcare reforms that may produce profound changes in the delivery and reimbursement of substance abuse treatment services. While uncertainty about changes in the healthcare system has the potential to complicate the process of planning and implementing a State outcomes monitoring system (OMS), it need not—and should not—deter SSAs from developing of such a system. Indeed, various State reforms are already placing new requirements on SSAs to conduct outcomes monitoring.

Given the inevitability of such requirements, there is a clear advantage to the SSA in taking the initiative to develop an OMS that will meet its own identified information needs as well as satisfy externally imposed demands for accountability. In doing this the SSA will not only obtain data essential to improving treatment outcomes, but will also strengthen its position as the lead agency for monitoring substance abuse treatment services within the State. In leading the development of an OMS, the SSA will help ensure that the data collected and the uses to which they are put meet the needs both of society as a whole and of the vulnerable, high-risk groups that are primary users of publicly supported alcohol and other drug (AOD) treatment services. The purpose of this chapter is to assist the SSA director and staff in understanding the process for gaining support for a State OMS that will meet these dual needs.

Outcomes monitoring systems must be relevant and acceptable to a broad spectrum of stakeholders, including the following:

• Consumers of AOD treatment services and their family members
• Patient advocacy groups
• Program administrators and clinical staff
• Organizations of counselors and other professional providers
• Other State service systems, including human services such as public health and criminal justice
• Employers
• Third-party payers
• Taxpayers
• Legislators
• State governors and other representatives of the executive branch of State government.

Support from all of these groups is needed to ensure 1) development of the OMS, 2) appropriate and effective use of the resulting data, and 3) achievement of desired policy changes. The interests that these stakeholders represent and how those interests can be addressed in building support for the OMS are discussed later in this chapter.

The principles and values of these diverse stakeholders influence the nature of AOD treatment services provided through the SSAs and have important implications for how an OMS is developed and the kinds of data that will be gathered. To develop an OMS that responds to data needs within the State and to secure support for its development, it is essential to identify and assess these principles and values in the early planning stages.

For example, one important rationale for undertaking outcomes monitoring is to achieve good treatment outcomes. A commitment to protect vulnerable and at-risk groups as part of government's social contract with the people is another. Ensuring the most effective use of limited resources is yet another value driving the development of an OMS in many States.

Historically, value judgments about addiction have often led to derailment of efforts to address alcohol and other drug abuse problems in the political arena. Clarity at the State level about the belief system and values of key policymakers such as the governor or legislative committee chairs will allow the SSA to promote outcomes monitoring effectively.

The link between substance abuse and a host of social and public health problems—family disintegration, crime, injuries, lost productivity, premature morbidity and mortality—demands that government have a role in the prevention and treatment of AOD addiction. This argument constitutes a powerful basis for garnering broad-based support for AOD treatment programs from public policymakers and their constituents.

Given the public policy importance of this link between substance abuse and other social and health problems, perhaps the most compelling measures of AOD treatment success will be demonstrated by decreases in these problems. While treatment programs cannot be held accountable for achieving such improvements singlehandedly, they clearly have an important role to play in conjunction with other health and social service agencies. SSAs have a responsibility to forge partnerships with these other agencies and develop with them a collective vision for how common programmatic goals and objectives can be met and measured.

Outcomes to be measured should include those that will justify, in terms of societal outcomes, the commitment of resources to treatment. These outcomes include potential cost savings from AOD treatment that will result from decreased demand for other services such as incarceration or foster care. Data exist that can demonstrate these cost advantages of treatment to society, and this type of cost-effectiveness data must be emphasized in a State OMS. At the same time, an agency must set realistic expectations and emphasize the interdependence of treatment success with improvements in other aspects of social welfare.

A number of factors may influence not only the form that outcomes monitoring should take, but also the process by which the OMS will be developed. While each State will have its own unique factors that must be taken into account, some factors are present in virtually all States. However, variations in the States preclude their using universal strategies for dealing with these common factors. The intent of this section is to ensure that these factors are considered and their potential for influencing the success of outcomes monitoring is assessed.

Healthcare reform. Healthcare reform may produce a number of results that will affect the delivery of AOD treatment services. In a climate of limited resources and aggressive marketplace competition, the SSAs may be asked to justify the retention of specific services for some client populations. By demonstrating the importance of these services to the general public welfare, OMS data may help these programs establish themselves as valuable "niche" providers in the reformed healthcare delivery systems of their States.

Trend toward merging AOD and mental health services. Expectations of treatment outcomes for the severely mentally ill are quite different from those for the recovering substance abuser. In cases where services are integrated or merged, care must be taken to ensure that distinctions among patients are retained in data collected for an OMS. Accurate information about patient status at intake, for example, is critical to establishing realistic expectations for outcomes and appropriate measures of success.

Budgetary resources. Within a State's AOD treatment system, development of an OMS will probably compete with existing State and Federal priorities for allocation of limited funds. It will be necessary to convince policymakers of the value of shifting funds from existing budget priorities to outcomes monitoring, a shift that may be made more difficult by the tendency of many at the policymaking level to view issues only in terms of short-term political expediency or payoff.

It will also be essential for the SSAs to convince programs that the diversion of treatment funds to support outcomes monitoring, if necessary, will ultimately pay off by demonstrating the value of AOD treatment and improving outcomes. Building in performance and funding incentives to programs that encourage outcomes monitoring may be one way of accomplishing this. The possibility of collaborative funding efforts with other provider agencies may be another.

Technological resources. Implementation of an OMS will require substantial technological resources, and limitations on those resources are among the pragmatic considerations that will affect the design of an OMS. While ideally all providers would use compatible computer hardware and software, in reality, multiple hardware and software systems already exist within most provider networks. Moreover, the startup costs associated with a fully integrated system would be prohibitive for most agencies.

From a practical standpoint, existing systems cannot be abandoned but must be factored into operational plans for the OMS. Thus, while SSAs are encouraged to develop long-range plans for such an integrated OMS—to be implemented in phases based on availability of resources to providers—they must be aware that insisting on specific hardware and software requirements for providers at the outset will only undermine the likelihood of implementing the system.

Training and technical assistance. Substantial amounts of training and technical assistance may be necessary for providers to undertake desired data collection. Both staff and budgetary resources for this effort must be identified.

State political environment. The importance of identifying key players within the State's political environment and developing a strategy for obtaining their support has already been suggested. The identity of these players may vary considerably from State to State and may be a function of personalities as well as governmental organization. The SSA's relationship to these key political players will also vary. In some instances, the SSA director serves at the pleasure of the governor. In all cases, the SSA's cultivation of political allies and effective performance as an advocate on behalf of AOD treatment services will be critical to its success in gaining support for its outcomes monitoring proposal.

Organization of service delivery systems within the State. Structures within State government, their relationship to treatment service providers, and issues of local and State control will clearly influence the developmental process of an OMS. Limited statutory authority of the SSA in some States, for example, will require that the agency find other ways of gathering needed support for OMS implementation. The agency that does not have sufficient authority or leverage to muster support must develop relationships, networks, and incentives to foster progress toward developing the OMS and at the same time develop and implement strategies to obtain greater authority within the State's decisionmaking process.

Assumptions underlying current program policies. State treatment programs inevitably reflect a set of assumptions that may or may not be validated by the results of outcomes monitoring. The implications of such findings should be considered. A basic assumption is that treatment works, and treatment staff may be reluctant to develop a monitoring system that could call this assumption into question. Another common assumption is that a range of services with differing levels of intensity is appropriate to meeting the treatment needs of diverse clients. Many providers also assume that the only way to have good outcomes is through treatment.

Failure to recognize the assumptions operating within an agency or program can undermine development of an OMS and its usefulness as a planning and accountability tool. Consideration should be given to the appropriate interplay between such assumptions and plans for outcomes monitoring. A primary strategy for identifying assumptions operating within the State's treatment services will be a careful review of plans, policies, and statutes of both the SSA and the various treatment providers. The perspective of advisory groups may also be useful.

Rationing of healthcare. Rationing of care as a consequence of future cost containment measures is a frequently expressed concern in the context of healthcare reform. In fact, inconsistencies already exist between care that is reimbursable and care that is needed to achieve reasonable therapeutic goals. In some areas, resources are allocated to serve more patients with short stays, rather than fewer patients with longer stays. This allocation is a clear example of the rationing of care. How this rationing occurs and its prevalence should be considered in developing an OMS, since limitations on treatment services obviously influence outcomes.

The OMS can be designed to determine the relationship of setting and length of stay to outcomes, but if treatment is severely restricted throughout the system, the observed outcomes may be uniformly poor. The greater the variability in factors such as setting and length of stay, the more likely it is that recommendations can result with respect to optimal service levels.

Threats to the status quo. Outcomes monitoring may expose program weaknesses, and treatment providers may be understandably concerned about the impact of such findings. Emphasizing to providers that the focus is on improving outcomes—and not on penalizing programs—should help alleviate their concerns about negative repercussions of outcomes monitoring. Further suggestions on how to respond to the concerns of providers are contained later in this chapter in the section on building support for the OMS.

Political ramifications. Negative findings of the OMS will inevitably generate political ramifications. Some treatment programs or program types may enjoy strong political support, and results of outcomes monitoring may threaten their survival. Efforts may be mounted to defend treatment services that do not fare well in outcomes monitoring but have strong political or private support. Individuals who themselves have benefitted from particular treatment modalities may be defensive or threatened by findings that challenge the presumed value of services that they favor.

Needs and concerns of agency staff. Credentialing is increasingly important within the AOD treatment field, and the results of outcomes monitoring may point to the need to upgrade the qualifications and credentials of treatment staff. If so, any training required to accomplish this will entail additional expense. Linking funding to the presence of qualified, credentialed staff may motivate programs to undertake the training and preparation necessary to meet credentialing requirements.

Summary

This brief review of factors influencing the development of an OMS is by no means exhaustive; however, it does serve to make the point that outcomes monitoring cannot be done in isolation. Rather, monitoring must occur within the context of State politics and programs and may also be influenced by factors well beyond the purview or control of the SSA or the State. Such factors should not be viewed as barriers that preclude the development of a useful OMS, but as influences that will shape the process by which it is developed and its purposes and design. There should be general agreement on the ultimate goal of outcomes monitoring—improved outcomes for people who enter substance abuse treatment, their families, and society—and its importance should serve as a powerful motivator to the cooperation and compromise that are necessary to achieve it.

Successful development and implementation of a State OMS will depend in part upon the degree to which key constituencies and interest groups within the State are involved in the planning and development process. The participation of a wide range of interested parties can both enhance the design of the system and provide a broad base of support for its operation and use. The following mechanisms and strategies can help obtain this participation and support within the context of a systematic planning process.

Who Should Be Involved?

The issue of substance abuse is a community concern, and a wide range of participants can contribute to the success of an OMS as well as benefit from the information it produces. In general, the OMS planning process should include people with the following characteristics:

• May be affected as a result of information generated by the system
• Will be involved in the data collection
• May be able to improve the quality of the system or the data collected
• Can develop a network of support for implementation of the system once it is developed.

Specific groups to be considered include:

• Policymakers with responsibility for planning and implementation of AOD abuse treatment and other health and human services. These policymakers may be State agency heads or administrators to whom such responsibilities have been delegated. They have a shared interest in social issues, can offer the perspective of other services in the identification of appropriate data to be collected, and will be key collaborators in any efforts to integrate benefits and services.
• Treatment program administrators and clinical staff. As providers of substance abuse treatment services, this group will be responsible for most of the data collection; their support is critical to the success of those efforts. Moreover, they will bring a sense of realism to the consideration of what data can and should be collected, how they can be collected, and how data can be used by programs to improve outcomes. Finally, by their career choice of working in the treatment field, they have demonstrated a personal investment in successful treatment outcomes.

  One key to obtaining provider support for outcomes monitoring will be identifying outcome measures that programs are willing to accept and use as a basis for program decisions. These outcomes must be considered fair measures of program effectiveness and be developed in consultation with providers and clinicians, and the data collected must be considered adequate to provide a basis for judgments or decisions.

  This concept of fair measures has important implications for the design of the OMS, the purpose it serves, and the outcome variables that will be the focus of data collection efforts. Expectations about the kind of information that outcomes monitoring can provide and the uses it can serve must be realistic. In most States, outcomes monitoring systems or data probably will not be adequate for States to make definitive statements about specific programs. However, data and the OMS should enable the SSA to assess what strategies are working best across the State. Enough data on services must be provided to allow a determination of what mix of services seems to work best for whom. Programs can then be directed toward provision of the best service mix for different kinds of patients.

• Other health professionals. The cooperation of health professionals from a variety of disciplines and specialty areas will be important to establishing linkages among services, and their cross-disciplinary perspectives will help ensure that the scope of the proposed data set is not defined too narrowly.
• Criminal justice system personnel. Members of the criminal justice system—including judges, corrections personnel, law enforcement representatives, and providers of legal services—have an important perspective on substance abuse and its societal impact.
• Regional, county, and other local health and social service agencies. The goals of one agency cannot be met without the cooperation of collateral service agencies. Other health and social service agencies, such as departments of housing and departments of public health provide collateral services necessary for clients in AOD treatment.
• Benefits managers and managed care representatives. As purchasers of AOD services, members of this group have a vested interest in obtaining outcomes data in order to set appropriate service criteria. They may already have outcomes data of their own to offer and experience in collecting them. They also provide an important link to the private payer community.
• State legislators and their staffs. Of particular importance are legislators serving on committees with oversight responsibility for AOD services or other functions related to operation of those services, such as budget committees. They have a major impact on policy decisions and are in a position to alter and direct financing.
• Special populations. A broad-based consumer perspective can be obtained through inclusion of representatives of populations traditionally underserved, such as people of color, recent immigrants, prisoners, the developmentally disabled, mentally ill persons, pregnant women, and youth.
• Researchers and other experts in OMS design and implementation. The assistance of technical experts clearly can enhance the development and implementation of an OMS. Inclusion of these experts in the early stages of planning ensures their understanding of the purposes of the OMS and the myriad considerations that influence its final design.

The support and long-term commitment of these groups will depend upon the extent to which they become convinced of the relevance of the OMS to their own interests and goals. Gaining this support will require a process of cultivation much like that associated with community development and grass-roots organizing. Later sections of this chapter discuss how this support can be accomplished through formal mechanisms such as a steering committee or workgroups. In addition, a review of the literature on community organizing may be useful to gain an understanding of the procedures, strategies, and skills involved in this process.

Organizing the Planning Process

A systematic, collaborative approach to planning and developing the OMS will maximize prospects for its successful implementation. The approach outlined here emphasizes the formal involvement of organizations and groups already identified as important collaborators. Within the planning structure, the SSA serves a coordinating function and provides overall leadership for the planning process in the person of the project director. Additional staff resources will also be required. A steering committee and a series of task-specific workgroups provide mechanisms for gathering the assistance and support of other agencies and organizations.

Appointment of a Project Director

As the person charged with overall responsibility for development and implementation of the OMS, the project director must also have the authority necessary to meet that responsibility. In addition to understanding the intended uses of the OMS and the principles of program planning and evaluation, the project director must possess strong interpersonal skills and sound political instincts. He or she must be able to cultivate and maintain good working relationships with the leadership of all groups targeted for participation in the planning process. Establishing good communication channels with these groups should be one of the earliest steps in the planning process, and the success of this coalition-building will be key to OMS development, as will the ability to analyze and negotiate the political process.

These are other important attributes of a project director:

• Knowledge of basic research skills
• Knowledge and understanding of the position of the SSA within the State and its relationship to providers and other social service agencies
• Skills in group facilitation and consensus development
• Ability to gain respect and credibility with policymakers and other administrators of the agencies involved.

Formation of a Steering Committee

A steering committee that represents participating groups is recommended as a formal mechanism for obtaining the input of those groups. The members of this committee should be key decisionmakers within the organization or group they represent. Following are the primary responsibilities of the steering committee:

• Develop a mission statement and vision of the OMS
• Establish goals for the OMS
• Provide oversight for OMS development and operation.

Use of Working Groups

In addition to forming a steering committee, it may also prove useful to establish workgroups that will supplement efforts of the steering committee and assume much of the nuts-and-bolts responsibility for developing the OMS. Workgroup members, usually staff members within their respective organizations, should be selected by the steering committee. Specific workgroup assignments that might contribute useful information include:

• Framing objectives for the OMS, based on goals established by the steering committee
• Conducting a needs assessment
• Identifying research questions to be addressed by the OMS
• Developing a work plan and implementation strategies
• Reviewing current data sources
• Identifying a core set of data
• Outlining a research methodology.

Identifying Roles, Responsibilities, And Tasks

Within the collaborative framework created to guide the planning process, it will be important to delineate roles and responsibilities of each participant and to specify tasks and deadlines for their completion. Assignment of responsibilities will reinforce the participation and commitment of members, while contributing to the work of developing the OMS. Careful attention to planning meetings—for example, setting an agenda and identifying effective facilitators of the group process—will be important to gain maximum benefit from the group. A process for conducting meetings should be developed that will permit expression of divergent opinions without allowing these differing viewpoints to dominate proceedings. Work plans that identify who is doing what and projected dates of completion are a practical tool for ensuring common understanding of roles and responsibilities.

Establishing Goals and Ground Rules

Agreeing on common goals for the OMS is essential to obtaining commitment to the effort from all participants in the planning process. The buy-in that these shared goals can produce will be a key element in achieving success. When differences of opinion occur, the presence of these common goals for the OMS will help keep planning on track and maintain unity of support.

Equally important to establishing common goals is reaching agreement on the ground rules for group decisionmaking and determining how consensus will be defined. Whenever priorities are set or choices made, there is the likelihood that the interests or needs of some groups will not be served. If there is agreement on the purpose and goals of the OMS and if the agreed-upon process for development is adhered to, then participants should agree to support the resulting system plan, even if it does not satisfy all of their own requirements.

Experience in Minnesota offers a useful example of how this support can be achieved. The collaborative process used there to develop an OMS included an agreement by participants that they would accept group consensus regarding data priorities, even if their own preferences were not adopted. Consensus in this circumstance did not mean total agreement, but acceptance and support for the larger group decisions.

Maintaining Ongoing Involvement

A variety of strategies can be employed to maintain participation and commitment among participants in the developmental process:

• Meeting periodically to review progress and address problems
• Providing tangible evidence of progress and results on a regular basis
• Maintaining contact with key decisionmakers in each group, even though they may have delegated tasks to others.

Identifying Specific Questions To Be Asked by the OMS

Specifying questions to be asked by the SSA's outcomes monitoring system is a dynamic process. The questions will vary from State to State and over time. Even though an identified data need may provide the initial impetus for development of the OMS, the questions the OMS is designed to answer will undergo refinement during the process of development and consensus building. Establishing parameters for the types of questions that may be considered will be facilitated by the earlier development of a mission statement and agreement on goals of the OMS.

An information needs assessment may be undertaken to identify the key questions by each of the collaborating groups. The steering committee may then analyze the results of this needs assessment to establish priority questions that should be addressed. In establishing priorities among questions, it may be necessary to weigh the relative needs of the various groups. Prior agreement on the definition of consensus will become critical to winning acceptance of these priorities.

A more detailed discussion of identifying questions for the OMS can be found in Chapter 5 of this TIP.

Assessing Current Capacity To Answer Priority Questions

Once priority questions have been agreed upon, the State's current capacity to address the questions must be assessed. This process should not be limited to an examination of the information resources of the SSA. An inventory of other agencies' data collection efforts will be needed, a project requiring interagency cooperation and information sharing. Such an inventory must consider not only the data elements collected, but also the accuracy of the data and the utility of its format for SSA purposes.

In addition to examining existing data collection efforts, an assessment of the SSA's staff capabilities and available technology is also required. Some State agencies may not have the capability to undertake outcomes monitoring and will want to contract this function to another agency or institution.

Agencies will have limited resources for conducting outcomes monitoring, and the cost of collecting each data element must also be considered. Where data collection costs will be high, the importance of the information must be weighed against the costs to justify inclusion in the OMS. The results of these calculations may necessitate a reordering of priorities if this informal cost/benefit analysis does not meet guidelines or expectations regarding usefulness and importance of data to be collected.

Requirements must be determined for training SSA and program staff to enable them to use the data and for training the provider network staff to enable them to collect the data.

Assessing Potential Usefulness of External Databases and Instruments

Every effort should be made to avoid duplicating existing data collection activities or reinventing existing systems or instruments that have already been validated and may meet local needs for a data collection instrument. In addition to freeing resources for other uses, a valid instrument common to other groups allows for comparability of findings. If such instruments can be adapted to SSA data collection needs, the resulting outcomes data on substance abuse would then parallel data collected in other areas, an added benefit.

Identifying Data To Be Collected by The OMS

Based on results of the State capacity assessment, it should now be possible to specify those priority data elements for which no other source exists and which should be collected by the OMS. It may be desirable to identify a core set of data that will be able to answer whatever the priority questions may be at any particular time. From these data will emerge information on services, patient characteristics, social indicators, and other variables that can demonstrate the value of substance abuse treatment to legislators and other policymakers. In addition, if decisions about priority information needs have been reached through a collaborative process with other data users, then data generated by the OMS should be relevant to the network of potential users who developed it and whose continued support for the OMS is essential.

Last Updated 11-7-02