TIP 14 — TIPs — Documents — Home

Appendix B—Outcomes Monitoring Systems of Some State Agencies

Treatment outcomes monitoring systems developed by State agencies vary considerably in their content and methods as well as in their stage of development. Six State systems are profiled in this appendix to illustrate the variety as well as the commonalities of existing State systems: California, Colorado, Iowa, Minnesota, Tennessee, and Washington.

California and Washington are included among the examples even though they do not conduct outcomes monitoring as described in this Treatment Improvement Protocol (TIP). These two States, like the other four discussed, have developed and implemented sophisticated systems to collect data on patients at admission and discharge. Like the others, they have incorporated the Federal Client Data Set in their admission forms and modified and enhanced this minimum data set in a variety of ways to meet local needs. Like the others, they also collect data on patients from a variety of levels of care and treatment modalities in their service delivery systems.

California, with the largest treatment service delivery system among the States profiled, recently completed an expensive, methodologically rigorous followup study of a sample of patients to document treatment effectiveness and the cost benefits of treatment. This study involved in-person interviews that typically lasted over an hour, a method too expensive for broad-scale outcomes monitoring. While such a study is not a substitute for routine outcomes monitoring, it does dramatically illustrate the utility of a design that measures the cost offsets of treatment.

Washington has not instituted postdischarge followup of its patients, but this is the next stage in its developmental process. This State system is included as an example of a State with a phased-in development and implementation plan. Also, Washington includes the most comprehensive assessment information on all patients, as well as a tracking mechanism for patients eligible for publicly funded treatment. Washington also has the most sophisticated system in terms of electronic data transfer and automated summary reports. The system is designed so that each program maintains its own individual database as well as forwards data for central processing.

Among the States, only Minnesota collects data on virtually all private-pay as well as public-pay treatment patients. Minnesota's provider reimbursement system for public-pay clients is unique, resulting in a competitive marketplace where all programs in the State compete for public-pay clients. Since all but a few programs serve public-pay clients, all are subject to data collection requirements. In Iowa, private programs can report voluntarily, but few choose to do so.

Of the four States described with systematic outcomes monitoring in place, Colorado, Minnesota, and Tennessee rely primarily on telephone interviews for patient followup. Iowa uses telephone and in-person interviews during clinic visits by the patient. Colorado and Iowa rely on treatment program staff to conduct the followup interviews, whereas Minnesota and Tennessee contract with researchers to conduct the interviews. Minnesota used treatment staff for several years but changed in 1993 because of the low contact rate and the skewed followup sample (composed largely of patients with better prognoses). Minnesota's contracted services include only the interviews; State agency staff conduct the data analyses and write the reports. Tennessee's contracted services include both data analyses and reports.

Postdischarge followup intervals also vary from State to State. Iowa and Minnesota conduct a single followup at 6 months postdischarge. Tennessee conducts followups at 6 and 12 months and reports only on patients for whom both contacts were successful. In Colorado, the contact intervals range from approximately 12 to 18 months. The California outcome study also involved variable intervals ranging up to 24 months, with an average postdischarge interval of about 15 months.

Among the six States described here, only California has offered patients an incentive ($15) to participate in the followup interview. The interview was much longer (75 minutes) than those in other States, (about 15 to 20 minutes), and it was conducted in person rather than over the telephone.

California, Colorado, Minnesota, and Tennessee all used or are using a sample of the total patient population for followup. California, Colorado, and Tennessee used a retrospective random sampling design, identifying the sample from discharge patients. Minnesota uses a prospective convenience sample. After all treatment program staff are trained, staff seek informed consent from 30 consecutive patients (or some predetermined pattern of alternate admissions). Consenting patients are asked to participate in additional comprehensive in-treatment data collection, as well as the followup interview. Iowa attempts to follow all patients who successfully complete treatment.

Because of ongoing refinements to State systems, interested readers are encouraged to contact State agencies directly to request sets of questionnaires, forms completion manuals and other training materials, and available reports. The summary provided here also does not address followup consent rates and contact rates, both important to consider along with other methodological issues in weighing the potential effects of sample bias. Because these issues are too complex for the brief overview provided here, no attempt has been made to compare State systems in this regard. Outcomes monitoring system planners are urged to discuss design issues with State personnel with previous experience to determine why they chose the design they have adopted, what previous experiences influenced their decisions, and whether they have plans to make refinements in the future.

California Department of Alcohol and Drug Programs

Contacts: Dennis Johnson
Dorothy Torres
Research and Policy Analysis Branch
Telephone (916) 322-2285
Fax (916) 323-5873

Treatment Data

California Alcohol and Drug Data Systems (CADDS):
Target programs. All treatment providers that receive any public funding for treatment services or that are required to report as a condition of State licensing; approximately 800 programs report on CADDS.
Target population. All admissions to CADDS reporting programs; approximately 143,000 annually.
Levels of care. Three types of residential treatment (short-term, long-term, and residential) and four types of outpatient (methadone maintenance, nonmethadone, detoxification, and intensive).
Forms. Participant Record Admission Copy, which incorporates the Federal Client Data Set.
Participant Record Discharge Copy, which includes Client Data Set items at discharge as well as level of care, modality, and length of stay.
Data entry. Keyboard.

Followup Data

Followup method. Study design and interviews done under contract with the National Opinion Research Center (NORC) at the University of Chicago and Lewin-VHI, Inc., Fairfax, Virginia, between September 1992 and March 1994; in-person interviews with patient, lasting an average of 75 minutes, conducted by trained researchers from NORC. Average postdischarge interval was 15 months; intervals ranged up to 24 months.
Target programs. Sample of 97 providers from 16 counties selected, based on principles of geographically balanced, size-weighted random selection.
Target population. Random selection of patients discharged (or in methadone maintenance) between October 1, 1991, and September 30, 1992. A total of 1,859 interviews were completed from a base sample of 3,055.
Data entry. Keyboard.
Estimated costs. $2 million for followup study.

Future Plans

Conduct a 4- to 5-year followup on the patients interviewed for the 1994 report.

Report

Evaluating Recovery Services: The California Drug and Alcohol Treatment Assessment (CALDATA) General Report. Submitted to the State of California Department of Alcohol and Drug Programs by the National Opinion Research Center and Lewin-VHI, April 1994. Authored by Dean R. Gerstein, Robert A. Johnson, Henrick J. Harwood, Douglas Fountain, Natalie Suter, and Kathryn Malloy. A copy of the report can be obtained by contacting the California Department of Alcohol and Drug Programs Resource Center at (916) 327-3728 or (800) 879-2772.

Colorado Department of Human Services Alcohol and Drug Abuse Division (ADAD)

Contact Linda Harrison
Data Analysis and Evaluation Section
Telephone (303) 692-2940
Fax (303) 753-9775

Treatment Data

Target programs. All treatment providers that receive any public funding for treatment services, as well as those under special reporting requirements; approximately 40 programs.
Target population. All admissions to reporting programs; approximately 63,000 annually.
Levels of care. Free-standing residential detoxification, psychiatric residential, therapeutic community, transitional residential, intensive residential, and outpatient.
Forms. DACODS Answer Sheet includes Admission-only items, items coded at Admission and Discharge, and Discharge- only items; incorporates Federal Client Data Set.
Data entry. Keyboard; currently developing automated data submission whereby clinics would submit data on diskette.

Followup Data

Followup method. Telephone interviews with patient or third party conducted by treatment provider staff approximately 12 to 18 months postdischarge.
Target programs. All programs except detoxification.
Target population. Random selection from discharges between June and December 1992 stratified by treatment history; redesign of study conducted about 2 years earlier.
Data entry. Keyboard.
Estimated costs. Personnel and other administrative costs.

Future Plans

Publications

Iowa Department of Public Health Division of Substance Abuse and Health Promotion

Address Lucas State Office Building
Des Moines, IA 50319

Contacts Janet Zwick, Director
Telephone (515) 281-4417
Fax (515) 281-4958

Linda Holt
Telephone (515) 281-4643
Fax (515) 281-4535

Treatment Data

Target programs. All treatment providers that receive any public funding for treatment services, and private programs that voluntarily report. Approximately 46 programs report on SARS.
Target population. All admissions to reporting programs; approximately 25,000 annually.
Levels of care. Detoxification (medically managed, medically monitored, and outpatient), acute inpatient, medically monitored residential treatment, primary residential treatment, extended residential treatment, day treatment, halfway house, continuing care, extended outpatient, intensive outpatient.
Forms. Admission/Evaluation form incorporates Federal Client Data Set.
Services form records a variety of services and the number of days, sessions, or length of time involved in these services.
Discharge/Followup form includes relevant Client Data Set items, discharge status, and discharge date, as well as outcome measures and ratings of services received.
Data entry. More than half the programs enter data online and submit by modem or diskette; the others send hard copies for keyboard entry.

Followup Data

Followup method. Telephone interviews by treatment program staff or clinic visits 6 months postdischarge; followup conducted with patient or a significant other.
Target programs. All reporting programs.
Data entry. Same as above.
Estimated costs. Personnel and other administrative costs.

Minnesota Department of Human Services

Chemical Dependency Division

Address 444 Lafeyette Road
St. Paul, MN 55155-3823

Contact Patricia Ann Harrison, Ph.D.
Administrative Planning Director
Telephone (612) 296-8574
Fax (612) 297-1862

Treatment Data

Target programs. All treatment providers that receive any public funding for treatment services (356 of 365 programs), including almost all private programs; approximately 35 detoxification facilities also report voluntarily on a separate DAANES system.
Target population. All private- and public-pay admissions; approximately 32,000 treatment admissions and 36,000 detoxification admissions annually.
Levels of care. Primary inpatient, primary outpatient, residential extended care, and halfway house; detoxification facilities report on a separate system.
Forms. Intake form incorporates portion of Federal Client Data Set.
  History form incorporates an expanded version of the alcohol and other drug (AOD) use items from the Client Data Set and an arrest summary.
  Discharge form includes discharge status, referrals, payment source and charges, inpatient days, and outpatient treatment hours.
Data entry. Optical scanner.

Followup Data

Followup method. Telephone interview with patient 6 months after discharge conducted by researchers at New Standards, Inc., St. Paul (formerly CATOR).
Target programs. All reporting treatment programs (not detoxification facilities). Staff from approximately 15 programs are trained to begin TAP data collection each month for 24 months.
Target population. 30 consecutive (or alternate) admissions from each program once during a 3-year cycle. In addition to DAANES, TAP sample patients complete a modified Minnesota Addiction Severity Index (ASI) and weekly Treatment Services Review (TSR).
Data entry. Online.
Estimated costs. $20 per patient interview attempted or completed plus SSA personnel and administrative costs.

Future Plans

Publications

Tennessee Department of Health

Bureau of Alcohol and Drug Abuse Services

Address Tennessee Tower
312 8th Avenue North
12th Floor
Nashville, TN 37247-4401

Contacts Rick Bradley
Director of Contract Compliance
Telephone (615) 741-8515
Fax (615) 741-2491

Treatment Data

Target programs. All treatment providers that receive any public funding for treatment services; approximately 55 agencies.
Target population. All admissions to publicly funded programs; approximately 12,000 annually.
Levels of care. Residential and outpatient.
Forms. Admission form incorporates Federal Client Data Set.
Discharge form includes a record of services and number of visits, discharge status and referrals, rating of adaptive functioning at last encounter, and Client Data Set drug use items at termination.
Data entry. Keyboard for half the agencies; the others submit data on diskette; converting to diskette data submission for all programs in 1995.

Followup Data

Followup method. Telephone interviews with patients and collaterals conducted by research assistants under supervision of project director at University of Memphis; interviews conducted 6 and 12 months postdischarge (report sample limited to patients who complete both interviews).
Target programs. 25 facilities in 1991/1992; expanded to all programs willing to participate.
Target population. Random selection of patients who complete treatment; sample totaled 1,846 in fiscal years 1991 and 1992. Future target of 15 percent of all admissions.
Incentives. None.
Data entry. Optical scanner.
Estimated costs. Approximately $225,000 annually for followup interviews and reports.

Future Plans

Publications

Washington State Department of Social and Health Services, Division of Alcohol and Substance Abuse (DASA)

Address P.O. Box 45330
Olympia, WA 98504-5330

Contacts Kenneth Stark, Director
Telephone (206) 438-8200
Fax (206) 438-8078

Fritz Wrede
Telephone (206) 438-8224
Fax (206) 438-8078

Treatment Data

Treatment and Assessment Report Generation Tool (TARGET):

Target programs. All treatment providers that receive any public funding for treatment services; approximately 194 programs.
Target population. All admissions to publicly funded programs; approximately 13,000 detoxification admissions and 32,000 treatment admissions annually.
Levels of care. Detoxification, intensive inpatient, intensive outpatient, outpatient methadone, long-term residential, extended care, recovery house.
Forms. Assessment/Admission Setup form includes patient identification and demographics, provider identification, and referral information for patients eligible for publicly funded treatment.
  Assessment/Admission and Discharge form includes Federal Client Data Set items as well as assessment information related to physical health, psychological condition, illegal activity, and substance use history.
  Discharge form includes discharge status, referrals.
Data entry. Online; each participating agency maintains its own local database as well as forwarding data to central database. System features a variety of automated reports.

Followup Data

Last Updated 11-7-02