Follow Us On
Contact Us
info@ampainsoc.org847.375.4715
American Pain Society
4700 W Lake Ave
Glenview, IL 60025
Robert J. Gatchel, PhD, Department Editor
Robert J. Gatchel, PhD
The dramatic changes in the managed care and workers' compensation environments that have occurred during recent years have necessitated that treatment professionals become familiar with and conduct evaluations of treatment outcomes. This is particularly important in the area of occupation-related musculoskeletal disorders because of the dearth of standardized outcome measures to meet this need.
This absence of standardized outcome measures stems from several factors. First, professionals have achieved only limited agreement on what mix of subjective and objective outcome criteria to use. Secondly, there has never been a medicolegal or economic motivation to carefully monitor outcomes. Finally, many clinicians resist evaluation of their therapeutic interventions solely on the basis of disability-related socioeconomic outcomes because all of their past training has led them to emphasize the importance of diagnosing and treating an anatomical, rather than a more complex biopsychosocial, disorder. However, changes in healthcare policy and the public demand for improved allocation of health resources have recently created, and will continue to stimulate, a great deal of pressure on healthcare professionals to validate the present utilization of health resources, especially since healthcare expenditures in the United States are approaching 14% of the gross domestic product.
A recent conference, "Navigating the New Managed Care and Workers' Compensation Environments: A Guide to Health-Care Professional Survival," took place in Charleston, SC, in April of this year because of the perceived need to help healthcare professionals focus on important issues of professional survival. One key issue at this conference was the importance and necessity of conducting research on high-quality treatment outcomes. Because this was the first conference jointly sponsored by the North American Spine Society (NASS) and the American Pain Society (APS), it indicates that professional healthcare societies are taking a proactive stance in developing outcomes research. If we do not, we will suffer the consequences of having managed care organizations "beat us to the punch" and presenting their own outcome data (which can be potentially biased because of a cost-savings motivation). This article reviews some of the major topics concerning outcomes research discussed at the joint NASS-APS meeting in Charleston.
Numerous issues merit consideration when considering outcome measures: What results should be measured, and how should they be quantified? From whose perspective (e.g., the patient's, the physician's, the employer's, the insurance company's) should we evaluate the results? How confident must we be about the results? Should an outcome simply be defined as the result or consequence of some treatment process? The United Kingdom Department of Health has provided a precise and workable definition of an outcome: "...the attributable effect of an intervention or its lack on a previous health state." This definition addresses a number of important variables in the measurement of outcomes. First of all, the attribution-of-effect aspect is not an either-or concept; rather, it is one that is measured on a continuum. Second, there must be evidence of a process-outcome connection or the likelihood of one (Long, 1997). Finally, there must be an accurate evaluation of the previous health status of the individual against which a treatment-effect change can be assessed.
Judith Turner, PhD, who provided a comprehensive discussion of the availability of various measures to document outcomes, is no stranger to the issue of documenting outcomes. She was the lead author of the large and widely publicized meta-analysis of lumbar spine fusion outcomes that documented a range of 16%-95% for "satisfactory" outcomes (with a mean of 68%), and the poorest outcomes associated with prospective studies conducted to date (Turner, et al., 1992). Often, many studies employ less than rigorous historical outcome methodology, which have demonstrated 38% unsatisfactory outcomes for vocational status, 37% with unsatisfactory functional ability, and a 9.6% reoperation rate (0-31% range) (Herron & Pheasant, 1983; Stauffer & Coventry, 1972; Taylor, 1980). More recently, Mayer et al. (1998) further reviewed these less than acceptable outcomes for spinal surgery. Louis (1998) has also recently suggested in his presidential address to the American Society for Surgery of the Hand that a more careful evaluation needs to take place before a further proliferation of surgery for repetitive motion disorders occurs.
In her presentation, Turner emphasized the need for more standardized outcome measures that all investigators could adopt to ensure the possibility of comparing similar outcomes from different clinical research studies. A great many clinical research studies have taken place over the years. For example, Jadad, Moore, and McQuay (1996) reported that between 1950 and 1990, more than 8,000 randomized clinical trials (RCTs) in pain research, using various outcome measures, appeared in biomedical journals. Indeed, Turner appropriately noted that investigators now recognize a need for more standardization of outcome measures. This perceived need has already stimulated the publication of review articles delineating the type of outcome measures that have been developed for studying patients with disorders such as low back pain-related disability and other musculoskeletal pain disabilities (Deyo et al., 1994; Flores, Gatchel, & Polatin, 1997). Turner also appropriately noted that the Medical Outcomes Study 36-item Short Form Health Survey (the SF-36) has rapidly become the outcome measures "darling" of many healthcare organizations. It was developed for various uses, including clinical practice and research, and it has good psychometric properties (Ware & Sherbourne, 1992).
At the conference, Turner also chaired a symposium, "Monitoring Outcomes for Cost-Effectiveness and Efficacy: Why It's Now a Necessity," which reiterated the importance of treatment outcomes for ensuring that there is no future attack against occupational and pain management professionals for neglecting their responsibility to carefully document treatment efficacy in a reliable and valid manner. The relative lack of conscientiousness in documenting treatment outcomes is somewhat of a paradox in the medical community because, as recently noted by Kaska and Weinstein (1998), Ernest Codman (1869-1940) initially made significant contributions to medical outcomes research. Codman proposed his "end result idea," which emphasized that every hospital should routinely follow up every patient it treated for a long enough period to document whether the treatment was successful. If a treatment was unsuccessful, the question "Why not?" was raised in order to prevent future similar failures. Kaska and Weinstein note:
In Codman's system each patient who entered the operating room was provided with a 5-inch by 8-inch card on which the operating surgeon filled out the details of the case before and after surgery. This card was brought up 1 year later, the patient was examined, and the previous year's treatment was then evaluated based on the patient's condition. This system enabled the hospital and the public to evaluate the results of treatments and to provide comparisons among individual surgeons and different hospitals (p. 630).
Obviously, we need to heed this sage advice.
Bernard Pfeifer, MD, also reviewed the new Musculoskeletal Outcomes Data Evaluation And Management System (MODEMS), a peer-developed computerized system to assess the functional outcomes of patients and compare them with national norms. The major focus of MODEMS is to provide an evaluation tool that will enable clinicians to make comparable functional outcomes assessment. Mayer, Gatchel, and Prescott (in press) have developed a protocol for monitoring socioeconomic outcomes after treating patients with occupational musculoskeletal disorders.
Gary Franklin, MD MPH, provided an excellent overview of the various factors that healthcare agencies look for when purchasing high-quality care in workers' compensation systems. As he has noted, a number of treatment guidelines have attempted to standardize care. Such treatment guidelines, if effective, along with special occupational health-centered care (i.e., "centers of excellence") and more integrated case management, provide the greatest potential for cost savings and improved outcomes. Many clinical practices often overlook the importance of having a competent case manager. Case managers can serve the important role of the liaison between employers and the injured patient or the insurance company and the injured patient, as well as coordinate the important tasks of documenting treatment precertification, medical necessity, treatment preauthorization, and denial-of-treatment appeals. They can also provide data that demonstrate the effectiveness of the proposed treatment plan for a patient.
Moreover, Franklin emphasized the necessity of coupling consistent and reliable outcome measures to standard treatment guidelines to produce the most integrated program of treatment and documentation of the outcome of such treatment. On a related topic, Michael Ashburn, MD MPH, also emphasized, in his talk on pain management and healthcare policy, the importance of having clear and consistent treatment guidelines in the workers' compensation environment. Healthcare professionals must learn more about developing consistency in correlating reimbursement guidelines with treatment guidelines, and documenting such guidelines through outcomes measurement.
In another symposium, "What Works? Evidence-Based Treatment Guidelines," chaired by Dennis Turk, PhD, the importance of tying outcome measures to treatment in order to produce valid and acceptable evidence-based guidelines was described. The methodology now exists to develop such evidence-based guidelines through the use of scientific methods such as meta-analysis, randomized controlled trials, controlled-cohort studies, controlled case series studies, case series, and case reports. Another important point emphasized in this symposium was that clinicians should not assume that published treatment guidelines will automatically gain acceptance as standards of care. For example, many spinal pain disorder treatment practitioners do not accept the AHCPR guidelines, which are now controversial for low back pain treatment, as the best standard of care.
John Banja, PhD, provided an impressive discussion of the evaluation of rehabilitation outcomes issues. One issue involved deciding, when determining treatment outcome, whose version should prevail or be given greater weight--the patient's, the therapist's, the physician's, the family's, the third-party payer's, or society's? This is significant because, as I will discuss later, one cannot always assume that these different outcome perspectives will always correlate highly with one another.
Banja also emphasized that when attempting to document treatment or rehabilitation effectiveness, one cannot always conduct an ideal randomized clinical trial because of the constraints of a treatment environment and the ethics of denial of treatment. Moreover, many new treatments evolve historically over time before we know whether they are reasonable and necessary. As I have recently reviewed elsewhere (Gatchel & Matt Maddrey, in press), there is no disagreement that an RCT is the highest quality type of clinical outcome study from which to infer cause-and-effect relationships, as long as threats to the internal and external validity characteristics of the study are ruled out. This is in keeping with the tradition of the hypothetico-deductive method in Western science for selecting specific treatment approaches. However, science is an inferential process, in which the significance of results are ascertained according to probability theory. Quite often, an RCT cannot fully evaluate a new treatment method without at least some inductive method of assessing results, based on approaches such as retrospective analyses of uncontrolled clinical observations or inferences based on the current understanding of a disorder.
In the symposium "Pain, Disability, and Impairment: Assessment and Diagnosis," chaired by Gunnar Andersson, MD, I reviewed the difficulties that often arise when attempting to use and measure three constructs--pain, disability, and impairment--the most significant being that they are not always correlated with one another. Webster's dictionary defines a construct as "an abstraction used to synthesize/interpret an assumed 'real' cause-effect relationship or phenomenon." Examples of commonly used constructs are intelligence, anxiety, essential hypertension, pain, impairment, and disability. To demonstrate the validity of a construct, it is important to develop a nomological network of objective referents, or criteria, to consistently account for a presumed cause-effect relationship or phenomenon. The construct of impairment is a medical phenomenon used to refer to an alteration of a patient's usual health status (i.e., some anatomical or pathological abnormality) that is evaluated by medical means. This construct implies objective evaluation of an individual's structural limitations. The construct of disability is an administrative concept representing the diminished capacity or inability of an individual to perform certain activities of daily living. Finally, the construct of pain is a psychophysiological phenomenon based primarily on experiential or subjective evaluation. As Waddell (1987) has noted, even though these constructs may overlap, perfect correlations among them do not always exist. Waddell observed that there is often a discordance or low correlation among these three constructs. One, therefore, cannot assume that they will be highly correlated with one another. A high-level impairment does not necessarily translate into a high-level disability; there can be a high level of self-reported pain and little impairment, for example. Physicians, insurance adjusters, and adjudicators must be aware of this fact in their decision-making processes.
I have further expanded upon the usual less-than-perfect concordance among these constructs (Gatchel, 1994). Moreover, as I have indicated, even for a construct such as pain, there are three major components of behavior that make measurement of this construct possible: self-report, overt behavior, and physiological indices. These components, in turn, are not always highly correlated with one another. What makes the issue even more complex is when a clinician views one specific measurement component of pain, such as a patient's self-reported pain. One can measure self-reported pain in a variety of ways, such as the use of the pain drawing, the pain analog scale, and the McGill Pain Questionnaire, to name just a few. These self-reported measures of pain do not always correlate with one another.
Figure 1 illustrates the difficulty of measuring a construct such as pain because of the different dimensions that must be considered, such as the three major components of behavior, as well as the various means of assessing each of these components by different measures.
As an additional example of this complexity for another construct--impairment--in one of the numerous studies of its kind, Brand and Lehmann (1983) reported that orthopedists in Iowa demonstrated a marked variability in their impairment ratings when they rated the same spine condition. Andersson (1991) summarized other such studies that prompted him to observe "...the difficulty in an impairment rating based on patient history and physical examination. Value judgments result in variation and contribute to measurement errors" (p. 534).
Finally, I often use the following metaphor to illustrate how it is frequently the case that different constructs, even though they appear to be discordant, may be measuring differing parts of the same phenomenon:
It was six men of Indostan to learning much inclined,
Who went to see the Elephant (Though all of them were blind),
That each by observation might satisfy his mind.The First approached the Elephant,
And happening to fall Against his broad and sturdy side,
At once began to bawl: "Bless me! But the elephant Is very like a wall!"The Second, feeling of the tusk, Cried, "Ho! What have we here, so very round and smooth and sharp? To me 'tis mighty clear This wonder of an Elephant is very like a spear!"
The Third approached the animal, And happening to take the squirming trunk within his hands,
Thus boldly up and spake: "I see," quote he, "the Elephant is very like a snake!"....And so these men of Indostan disputed loud and long,
Each in his own opinion exceeding stiff and strong,
Though each was partly in the right, and all were in the wrong!--"The Blind Men and the Elephant: An Old Indian Folk Tale," version written by John Godfrey Saxe
Just as the blind men in this folktale perceived and described the "whole" or in this case, the elephant, differently from one another, depending upon the part of the elephant they had touched or evaluated, so can musculoskeletal or other medical improvements be described quite differently, depending on what referent or construct of improvement clinicians touch or attempt to quantify. At first blush, they may not appear to be closely related in their descriptions of the "whole." After more careful scrutiny and analysis, however, they may simply be measuring different parts of the whole person's performance. Again, this indicates that measurement of outcomes is no easy task; it requires careful attention. Healthcare professionals have never perceived a need to invest time and energy into such careful evaluation. However, the time for careful evaluation has arrived, because the healthcare insurance community and the government demand careful documentation of outcomes. As healthcare professionals, we are at a point where we cannot assume that the public will automatically accept everything we do. This reminds me of a poem by Stephen Crane, which says this:
A man said to the universe:
"Sir, I exist!"
"However," replied the universe,
"The fact has not created in me
a sense of obligation."
As Martin Grabois, MD, emphasized in his lecture, "The Future of Pain Management in a Managed Care Environment," healthcare professionals no longer can expect that the healthcare universe assumes a sense of obligation to blindly accept all of the information that we give concerning the necessities and costs associated with the delivery of pain management. We now must justify the value of that delivery in order to validate our own existence! Moreover, we can no longer automatically assume that our "fathers"--be it the American Medical Association, NASS, APS--will be politically powerful enough to look after our professional needs. We, as individual professionals, must become more proactive in order to retain our status in the field of pain management. Reliably documenting outcomes is becoming increasingly important when determining the efficacy of treatment.
We all know that with the advent of managed health care came the movement to evaluate healthcare providers in a quantifiable manner. The results of these evaluations are used to assess providers, with performance measured in terms of quality and cost. This same evaluation approach is being applied to managed healthcare organizations to document the quality of care they provide to inform the purchasers (i.e., employers) of their healthcare plans. While these organizations have been a major force in driving down healthcare costs, many experts have argued that the quality of patient care has declined significantly. This has produced the recent trend toward evaluating managed care organizations. The quality of care they provide and advertise to the purchasers (employers) of their health plans is now undergoing evaluation.
The National Committee of Quality Assurance, a nonprofit organization, has set up specific guidelines for health plans for the collection and reporting of data. These guidelines make it possible to compare one healthcare plan with another (Spoeri & Ullman, 1997). This will, one hopes, exert much needed pressure on managed care organizations to provide appropriate quality care to patients. It is now becoming more critical for healthcare professionals to present treatment-related outcomes data to "beat managed care organizations to the punch" and to guard against the possibility that these organizations will present their own outcome data (which have the potential of being biased because of a cost-savings motivation).
Andersson, G.B.J. (1991). Impairment evaluation issues and the disability system. In T.G. Mayer, V. Mooney, &
R.J. Gatchel (Eds.), Contemporary conservative care for painful spinal disorders. Philadelphia: Lea &
Febiger.
Brand, R.A., & Lehmann, T.R. (1983). Low back impairment rating practices of orthopaedic surgeons.
Spine, 8, 75.
Deyo, R.A., Andersson, G., Bombardier, C., Cherkin, D.C., Keller, R.B, Lee, C.K., Liang, M.H., Lipscomb, B., Shekelle, P., Spratt, K.F., & Weinstein, J.N. (1994). Outcome measures for studying patients with low back pain. Spine, 19 (18S), 115-119.
Flores, L., Gatchel, R.J., & Polatin, P. (1997).
Objectification of functional improvement following nonoperative care. Spine, 22, 1622-1633.
Gatchel, R.J. (1994). Occupational low back pain disability: Why function needs to "drive" the rehabilitation process, APS Journal, 3 (2), 107-110.
Gatchel, R.J., & Matt Maddrey, A. (in press). The evaluation of complementary/alternative medicine therapy outcomes: An overview of important experimental design and analysis issues for the clinical investigator.
Alternative Therapies in Health and Medicine.
Herron, L., & Pheasant, H. (1983) Bilateral laminectomy and discectomy for segmental lumbar disc disease: Decompression with stability. Spine, 8, 86-97.
Jadad, A.R., Moore, C.D., & McQuay, H. (1996).
Developing a database of published reports of randomized clinical trials in pain research. Pain, 66, 239-246.
Kaska, S.C., & Weinstein, J.N. (1998). Historical perspective: Ernest Amory Codman, 1869-1940. A pioneer of evidence-based medicine: The end result idea. Spine, 23, 629-633.
Long, A.F. (1997). Key issues in outcomes measurements.
International Journal of STD and AIDS, 8, 663-667.
Louis, D.S. (1998). Presidential address: Are we there yet? The Journal of Hand Surgery, 23A (2), 191-195.
Mayer, T., Gatchel, R.J., & Prescott, M. (in press).
Socioeconomic outcomes following treatment of occupational musculoskeletal disorders: Methodology and data collection.
In T.G. Mayer, R.J. Gatchel, & P.B. Polatin (Eds.), Occupational musculoskeletal disorders: Function, outcomes and evidence. Philadelphia: Lippincott-Raven Publishers.
Mayer, T., McMahon, M.J., Gatchel, R.J., Sparks, B., Wright, A., & Pegues, P. (1998). Socioeconomic outcomes of combined spine surgery and functional restoration in workers' compensation spinal disorders with matched controls. Spine, 23, 598-606.
Spoeri, R.K., & Ullman, R. (1997). Measuring and reporting managed care performance: Lessons learned and new initiatives. Annals of Internal Medicine, 127,
726-732.
Stauffer, R., & Coventry, M. (1972). Posterolateral lumbar spine fusion: Analysis of Mayo Clinic series.
Journal of Bone Joint Surgery [Am], 54, 1195-1204.
Taylor, M. (1980). Return to work following back surgery:
A review. American Journal of Internal Medicine, 16,
79-88.
Turner, J., Ersek, M., Herron, L., H., Hasselkorn, J., Kent, D., Ciol, M.A., & Deyo, R. (1992). Patient outcomes after lumbar spinal fusions. Journal of the American Medical Association, 268, 907-911.
Waddell, G. (1987). Clinical assessment of lumbar impairment. Clinical Orthopaedics Research, 221,
110-120.
Ware, J.E., Jr., & Sherbourne, C.D. (1992). The MOS 36-item short-form health survey (SF-36): I. Conceptual framework and item selection. Medical Care, 30, 473-483.
Robert Gatchel is distinguished professor of clinical psychology and professor in the departments of psychiatry and rehabilitation science at the University of Texas Southwestern Medical Center in Dallas.
