TIP 14 — TIPs — Documents — Home

Chapter 7—OMS Budget Considerations

A variety of factors will influence the costs of an outcomes monitoring system (OMS) and these are worth considering during the planning phase. All single State agencies (SSAs) will be challenged to find a compromise between the ideal design and what they can afford to spend.

From the initial design stages of an OMS, a partnership must be forged among the SSA, independent researchers, and alcohol and other drug abuse (AOD) abuse treatment providers, with the single State agency and researchers providing primary oversight for the process. This same partnership should continue as the design is implemented and data collection efforts begin. However, there are expenses associated with the involvement of these various personnel.

Steering Committee and Working Groups

As described in Chapter 2, the support and involvement of a variety of stakeholders are essential to the successful design and implementation of an OMS. The State agency must consider to what extent expenses associated with the steering committee or workgroups will be borne by the participants or by the State. Expenses will include the cost of meeting rooms, refreshments, mileage reimbursements and other travel costs, and overnight lodging for those who travel a great distance.

Participants' time can be used most efficiently if meeting agendas are well planned and objectives for each discussion are clear. Participants can be mailed drafts of instruments and design plans in advance of meetings to maximize efficient use of meeting time. Some tasks can be assigned to small workgroups, and they can report on these to the whole group. For some decisions, such as approval of interim or final instrument or report drafts, reading the mailed materials may be just as productive as attending a meeting. In some areas, telephone conferences or videoconferencing may save costs, particularly when many participants must travel a great distance to the meeting site.

Expert Researchers

The role of independent expert researchers in developing and implementing an OMS can be significant. Outside academic researchers are subject to peer review standards and are likely to be able to create a rigorous data collection design. Their input is particularly important at two points: in the early design and implementation phases of the system, when they help ensure the integrity and credibility of the data, and in the data analysis phase of the project, when they can help interpret results. Another advantage of hiring independent researchers is that they are unlikely to have conflicts with any of the interests involved.

However, State agencies lacking the resources to hire independent researchers should not view this lack of resources as an obstacle to the successful implementation of an OMS. This three-way collaboration among outside experts, treatment providers, and State personnel may not be the answer for all States and systems. Some State agency managers may be reluctant to relinquish control of their OMS to an external source. In some instances, sufficient research expertise exists within the single State agency to design and implement an effective OMS without hiring outside experts, and additional staff can be added as needed. The possibility of conflict of interest does not warrant an expensive duplication of resources and can often be addressed with some external oversight procedure.

Project Staff

Whether the project is based at the SSA or contracted to a research agency, a full-time project manager is essential. In addition, depending on the scope of the project, one or more of the following staff will also be needed: project secretaries, systems analysts, programmers, and data clerks. Research analysts with statistical expertise will also be needed for monitoring data quality, analyzing results, and writing reports and presentations.

Training

Treatment program staff can collect patient admission and treatment data more efficiently than anyone else. However, specialized training will usually be necessary, and major costs will be related to staff training. Training sessions can be conducted by SSA staff or outside contractors. Treatment providers can be trained in groups of 25-30 personnel at a time. It is recommended that each program send at least two staff to training—including those responsible for administering structured interviews and filling out forms. Depending on the complexity of the data collection instruments, training sessions will probably last 1 to 2 days. While a detailed forms completion guide will be essential, there is no adequate substitute for in-person training. Whether travel expenses for program staff will be borne by the programs themselves or reimbursed by the SSA must be taken into account when setting up the training budget.

Effects of Sample Size

The single biggest driver of OMS project costs will be sample size. In fact, the availability or limits of funding may determine sample size. The largest cost directly associated with sample size is followup interview costs, which are discussed in greater detail later in this chapter. However, the sample size will also determine both the number of staff necessary to perform data entry and the printing costs for forms.

Instrument Selection and Design

As discussed in Chapters 4 and 5, instrument design is an expensive project in itself. New instruments require expert consultation, field tests, extensive data analyses, and refinements. To assure validity, they must be tested on the populations in which they will eventually be used. The need to design new instruments may delay the startup of an OMS. Because of the high costs associated with instrument design, the use of existing instruments is recommended. When these are inadequate for a State agency's needs, modifications will usually be cheaper than starting from scratch.

Existing instruments developed with funding through public dollars are in the public domain. This classification means that no costs are associated with using the content of the instrument. However, there will be costs associated with duplicating an instrument or reformatting it and printing it for project specifications. Software programs have been developed for some instruments in the public domain, and these are likely to have a licensing fee. Existing software may serve a State agency's needs and may justify the cost.

Copyrighted instruments are usually sold on a per-unit basis and may cost up to several dollars per copy. For large quantities, it may be possible to negotiate a user's fee with the copyright holder rather than having to pay on a per-unit basis. Copyrighted instruments may not be modified without the copyright holder's permission.

Miscellaneous Expenses

There are a number of other expenses that will have to be included in the OMS budget. These include costs for communications (telephone and mailing costs); meeting rooms and presentation equipment; costs for travel and accommodations, printing of forms, training manuals and reports; and supplies, including graphics software, training and presentation materials, and basic office supplies.

Patient followup can be conducted through telephone interviews, mailed questionnaires, or in-person interviews. Mailed questionnaires typically have an unacceptably low return rate and thus are not recommended as the primary contact method for an OMS. While in-person interviews may be the ideal, the costs are very high. Patients are unlikely to follow through on a visit to an interview site without an incentive. Having interviewers visit patients at home can cost between $50 and $100 per patient. Telephone interviews may offer the best compromise. Costs will vary, depending on area of the country and length of interview. In Minnesota, a private research agency is conducting 6-month followup telephone interviews for the OMS at an average cost of $20 per patient. Bids for this project, however, ranged up to $40 per patient.

While using treatment program staff to conduct followup interviews eliminates these costs, it is not recommended because of the burden imposed on staff and the potential threats to validity of data. Research agencies also have more experience locating respondents and conducting telephone interviews.

Followup costs are directly associated with the number of contacts attempted. Conducting followups at both 6 and 12 months posttreatment will double the costs of 6-month interviews only. With a limited budget, it will be necessary to weigh the advantages of a larger sample that is contacted once versus that of a sample half the size that is contacted twice.

Hardware and Software

When developing an outcomes monitoring system, State agencies must address issues related to computer technology. Many factors come into play, including the availability of existing resources. The primary focus of OMS development should not be hardware and software needs but rather the development of a well-planned system to meet the needs of the users. As system requirements are defined, answers to questions regarding appropriate computer software and hardware will become more apparent. The goal is to develop a system for gathering the required data while minimizing the number of intervening tasks that raise costs and increase the chance of error.

Once system requirements are defined, SSA managers will need to consider the design in light of available funding and other resources.

Interactive Data Collection Systems

The ideal, most cost-effective system is a computerized interactive data collection system. Such a system allows staff at treatment centers to enter data directly into the OMS during patient interviews. The system would be designed to alert the interviewer immediately to inconsistent data or unacceptable responses and would allow the staff to resolve data errors as they are identified. Online data entry significantly improves data validity and greatly reduces the number of intervening tasks that must be performed by the State agency. Washington State has made great strides with the use of online data entry.

Given the current cost and power of microcomputer technology, the hardware cost for a single microcomputer workstation is approximately $2,000, depending on the manufacturer and model. In large treatment centers where more than one microcomputer workstation would be required, a local area network (LAN) would have to be installed so that all workstations could share a common set of computer software and patient databases from a central computer or file server. The costs associated with a LAN vary greatly depending on the number of workstations, the distances between them, and the size of the network file server. A small LAN with one file server, serving fewer than 10 workstations in one location, would cost approximately $15,000.

Software to support an interactive system can be purchased from a software vendor, provided the software complies with system requirements, or new software can be developed by either inhouse programming staff or an outside software programming firm. For example, ASI software is available. The costs associated with each approach may vary greatly. Purchasing existing software is usually less expensive than developing new software. Prepackaged software is usually purchased on a per-user basis, and for assessment/testing software there may also be a per-issuance charge. When purchasing software, the buyer usually is also purchasing the right to use the software. Therefore, the total number of treatment centers using the software will have a direct impact on the costs associated with its purchase. Some software vendors may be willing to modify their software in order to bring it into compliance with a State's systems requirements. However, the charges associated with the software modification plus the normal purchase price may exceed the cost of developing new software.

Other Data Entry Methods

Data entry methods include keyboard entry, optical scanning, and online. With forms for keyboard data entry, answers are handwritten in blocks or boxes, and data entry personnel key in the responses. This method is the most labor intensive because of the extensive data entry required. Since newer, more cost-efficient technologies are available, this method is not recommended.

Optical scannable forms have "bubbles" or circles to be filled in; these forms are "read" by a scanner and require less staff labor. An optical mark scanning system is a relatively inexpensive method for converting data collection forms into electronic data. Scanners can effectively process approximately 2,000 forms per hour, depending on the manufacturer. Scanners are usually connected to a microcomputer, using a serial interface cable. Software provided by the manufacturer handles the communications between the computer and the scanner. The software is also capable of performing basic editing on individual data fields and is able to alert the data clerk to an identified error. However, more complex editing—such as cross-field validations—would have to be performed using other software programs. The components needed to implement an optical mark scanning system cost approximately $12,000, depending on the hardware and software configuration. Data collection forms especially designed and printed for optical mark scanning cost approximately 5 cents per form, depending on the amount of time spent designing the form and quantity of forms being printed.

Because of the great variety of local needs and existing capacities, it is impossible to adequately address all budgetary considerations in a brief chapter. As State agencies initiate the OMS planning process, they are advised to consult with agencies in States that already have outcomes monitoring systems in operation. They can also consult with researchers who have experience with treatment outcome studies. While aiming for cost-efficiency is always a worthy endeavor, it must be accepted that valid outcomes monitoring will entail substantial costs. The key is to ensure that the ultimate utility and benefits of the OMS justify the investment.

Last Updated 11-7-02