ARCHIVES OF INTERNAL MEDICINE — Vol. 161 No. 3, February 12, 2001
Author Information
Kristen M. Coppola, PhD; Peter H. Ditto, PhD; Joseph H. Danks, PhD; William D. Smucker, MD
Background Past research has documented that primary care physicians and family members are often inaccurate when making substituted judgments for patients without advance directives (ADs). This study compared the accuracy of substituted judgments made by primary care physicians, hospital-based physicians, and family surrogates on behalf of elderly outpatients and examined the effectiveness of ADs in improving the accuracy of these judgments.
Participants and Methods
Participants were 24 primary care physicians of 82 elderly outpatients, 17 emergency and critical care physicians who had no prior experience with the patients, and a baseline comparison group of family surrogates. The primary outcome was accuracy of physicians’ predictions of patients’ preferences for 4 life-sustaining treatments in 9 hypothetical illness scenarios. Physicians made substituted judgments after being provided with no patient AD, patient’s value-based AD, or patient’s scenario-based AD.
Results
Family surrogates’ judgments were more accurate than physicians’. Hospital-based physicians making predictions without ADs had the lowest accuracy. Primary care physicians’ accuracy was not improved by either AD. Accuracy and confidence in predictions of hospital-based physicians was significantly improved for some scenarios using a scenario-based AD.
Conclusions
Although ADs do not improve the accuracy of substituted judgments for primary care physicians or family surrogates, they increase the accuracy of hospital-based physicians. Primary care physicians are withdrawing from hospital-based care in growing numbers, and emergency medicine and critical care specialists most often are involved in decisions about whether to begin life-sustaining treatments. If ADs can help these physicians better understand patients’ preferences, patient autonomy more likely will be preserved when patients become incapacitated.
Arch Intern Med. 2001;161:431-440
ADVANCE DIRECTIVES (ADs) were created to ensure patient autonomy at the end of life. Ideally, patients who became decisionally incapacitated in their final weeks and days of life could still “voice” their preferences for medical care through a prerecorded document, known as the living will, or through a preappointed decision maker referred to as a proxy or surrogate decision maker. Autonomy is preserved to the extent that life-sustaining treatment decisions that physicians and family members make on behalf of the patient (based on AD information) are the same decisions that the patient would have made for himself or herself.
Considerable attention has been given to the difficulties associated with implementing policies designed to encourage completion and use of ADs. For example, surveys estimate that between 2% and 30% of Americans have actually completed an AD.1-3 Even if patients do have an AD, physicians are unlikely to be aware of patient preferences or of the existence of the document.4, 5 Other researchers have reported that ADs are often unavailable when treatment decisions need to be made.6-8 Finally, even if the physician is aware of the AD, the document still may be overruled by physicians or families who must make decisions for the patient,6, 9 possibly because the information contained in the document is vague or not easily applied in clinical situations.10, 11
The fact that ADs are not usually completed, are not available when decisions need to be made, or are ignored or overridden by physicians or family members indicates that there is a problem with policy implementation. The difficulties of policy implementation are shortcomings of “process” and do not necessarily corrupt the underlying assumption of the ADs, which is whether they contain information that can guide patient care and promote patient autonomy. The little information that we know about the content of ADs and their value as decision-making tools has been purely descriptive.10, 11
Although much research has documented that physicians and family members are often inaccurate when making substituted judgments for patients without ADs,12-26 until recently, no research had examined whether ADs improve the accuracy of substituted judgment. In their article in this issue, Ditto and colleagues27 showed that substituted judgment decisions informed by 1 of 2 types of ADs and enhanced with patient-surrogate discussion did not improve the accuracy of family surrogates’ substituted judgments. No research has examined if ADs can improve the accuracy of primary care physicians asked to make substituted judgments for patients. Arguably, physicians have a greater need for AD information, since prior studies19, 20, 25 indicate that primary care physicians have poorer accuracy compared with family members.
In addition, AD information may be particularly valuable for physicians who do not have a close relationship with the patient or have not had the opportunity to discuss treatment preferences with the patient. For example, in the emergency department, physicians are often faced with life-sustaining treatment decisions for patients with whom they have no preexisting relationship, and for whom there is no designated or available surrogate.28 Indeed, as primary care physicians delegate their care of seriously ill patients to hospital-based physicians, more decisions about initiation of life-sustaining treatments will be made by physicians who do not know the patient. The previous evidence concerning accuracy of decision making for well-acquainted surrogates indicates that accuracy would likely be even lower for physicians who are not acquainted with the patient. No research has examined the accuracy of decisions made by hospital-based physicians for unfamiliar patients, how this accuracy compares with primary care physicians’ or family surrogates’ accuracy, or whether ADs play a role in informing these decisions.
This study had 2 major purposes. The first was to examine the accuracy of substituted judgments made by hospital-based physicians for unfamiliar patients in comparison to patients’ primary care physician and family surrogate. The second was to examine whether ADs can improve the accuracy of substituted judgments made by primary care physicians and hospital-based physicians.
PARTICIPANTS AND METHODS
The Data for the present study were collected from physicians of patients involved in phase 1 of the Advance Directives Values Assessment and Communication Enhancement (ADVANCE) project, which examined the ability of ADs to improve the accuracy of decisions made by family surrogates.27 Primary care physicians and hospital-based physicians made predictions about the life-sustaining treatment preferences of a subset of phase 1 patients.
ELICITATION OF PATIENTS’ PREFERENCES
Participants in phase 1 of the ADVANCE project were recruited from a network of 6 group primary care practices, which included 24 primary care physicians, affiliated with Summa Health System in Akron, Ohio. Randomly selected patients 65 years or older were initially contacted by letter introducing the study. Unless patients telephoned the project office to decline participation, trained interviewers telephoned patients to solicit participation. Interviews took place in the patients’ homes and were approximately 1 hour long. A total of 401 patients and their designated family surrogates were interviewed in the ADVANCE project.
Patients completed the Life-Support Preferences-Predictions Questionnaire29 (LSPQ), which measures patient treatment preferences across a broad spectrum of realistic life-sustaining treatment decisions. The LSPQ began with descriptions of 4 life-sustaining medical treatments chosen to vary in their invasiveness: (1) antibiotics, (2) cardiopulmonary resuscitation, (3) gallbladder surgery, and (4) artificial nutrition and hydration. Patients were read standard descriptions of the treatments including why the treatment is provided and general risks and benefits of the treatments. Nine different medical scenarios were described that were chosen to capture a broad range of conditions varying in their severity, nature of impairment (eg, cognitive vs physical), prognosis, and level of pain: (1) the patient’s current health (current health); (2) Alzheimer disease with moderately severe cognitive impairment and a certain but indeterminate rate of decline (Alzheimer disease); (3) emphysema with constant shortness of breath, severe physical limitations, and a certain but indeterminate rate of progression (emphysema); (4) coma persisting 6 weeks after a stroke with no obvious cognitive abilities, indeterminate rate of decline, and physician opinion of no chance of recovery (coma no chance); (5) the same coma scenario as in scenario 4 with a physician opinion of a very slight chance of recovery (coma slight chance); (6) stroke resulting in partial paralysis, language deficits, total dependence in activities of daily living, and physician opinion of no chance of improvement (stroke no chance); (7) the same stroke scenario as in scenario 6 with a physician opinion of a very slight chance of improvement (stroke slight chance); (8) terminal colon cancer with fatigue, no pain, and a life expectancy of 6 months (cancer no pain); and (9) the same cancer scenario as in scenario 8 with pain that requires the constant use of medication to control symptoms (cancer with pain).
In the patient version of the LSPQ, patients imagined themselves in each medical scenario and indicated their preference for receiving each of the 4 medical treatments (in the current health scenario, the artificial nutrition and hydration question was omitted). Patients indicated their treatment preferences using a 5-point Likert scale: “definitely don’t want treatment,” “probably don’t want treatment,” “unsure,” “probably want treatment,” and “definitely want treatment.” Surrogates were asked to imagine the patient in each medical scenario and predict the patient’s treatment preferences using the same 5-point scale.
SURROGATES’ PREDICTIONS OF PATIENTS’ PREFERENCES
Family Surrogates
A subsample (n = 82) of family surrogates from the phase 1 study were used as a baseline comparison group with primary care physician and hospital-based physician accuracy. These surrogates were chosen by the patients in the phase 1 study as the individual whom they would want to make medical decisions for them if they were no longer able. The majority of family surrogates were spouses and adult children of patients. However, “family” is used in a broad sense, including friends and clergy.
As a part of the phase 1 study, patients and family surrogates were randomly assigned to either a control condition in which they did not complete an AD (no-AD) or 1 of 4 intervention conditions in which surrogates made predictions after exposure to a patient-completed AD (completed with or without discussion with the surrogate). The 4 phase 1 intervention conditions were created by the orthogonal application of 2 intervention components: (1) “type of AD,” family surrogates were provided with either a scenario-based AD (the health care directive [HCD]30, 31) or an outcome-based AD (the valued life activities directive [VLA]32) and (2) “opportunity for discussion,” family surrogates either were provided with the directive without the opportunity to discuss its contents with the patient or were present when the patient completed the AD and were encouraged to discuss the directive with the patient. (Copies of both ADs, as well as a full description of all phases of the ADVANCE project can be obtained from the author, P.H.D., on request.)
The family surrogates reviewed the patient’s AD (when applicable), made predictions on the LSPQ regarding patient preferences, and rated how confident they were that they accurately predicted the wishes of the patient on a 5-point scale.
Primary Care Physicians
All of the primary care physicians (N = 24) from the primary care practices affiliated with Summa Health System were contacted and agreed to participate.
Each primary care physician was to complete 5 substituted judgment tasks: predicting preferences of 1 patient who did not complete an AD (the no-AD condition), predicting preferences of 2 patients who had completed the VLA (1 patient who had completed the VLA with discussion with their family surrogate and 1 who had completed the VLA alone), and predicting preferences of 2 patients in the HCD condition (1 patient who had completed the HCD with discussion with their family surrogate and 1 who had completed the HCD alone). Primary care physicians were told the name of the patient whom they would be making substituted judgments for and were asked to answer questions regarding their relationship with that patient (eg, “How long have you been seeing this person as a patient?”). Four patient names were not recognized by primary care physicians and were not included in the study. Physicians were then instructed to review the patient’s AD (in the VLA and HCD conditions) and to predict their patients’ preferences for life-sustaining treatment on the LSPQ. Those in the AD conditions had the option of reviewing the patient’s directive at any time during the completion of the LSPQ. After completing the LSPQ, the physicians rated how confident they were that they accurately predicted the wishes of the patient on a 5-point scale and how helpful they found the AD. This procedure was then repeated with another patient until the physician had made substituted judgments for 1 patient from each of the 5 phase 1 conditions.
Because patients were randomly assigned to a study condition in phase 1, not all primary care physicians had patients in each of the 5 study conditions. In addition, 47 (12%) patients indicated on a question from their phase 1 participation that they did not want their AD shared with their physician. Because the “discussion” conditions did not directly apply to the physicians (ie, the physicians only reviewed the patients’ ADs and were not present for discussion when the patient completed the VLA or HCD documents) having them make predictions using ADs created from discussion was not essential. To maximize the number of physician predictions, physicians without 5 unique prediction conditions completed as many unique condition interviews as possible, with the goal of having predictions for 1 no-AD patient, 2 HCD patients (either discussion or no discussion), and 2 VLA patients (either discussion or no discussion) (Figure 1).
Thirteen primary care physicians completed judgments for 5 patients. The following list describes the remaining patient judgments: 1 physician completed judgments for 4 patients, 1 physician completed judgments for 3 patients, 3 physicians completed judgments for 2 patients each, and 5 physicians completed judgments for 1 patient each. The total number of predictions were 83, although 1 patient was subsequently not included in the study due to incomplete data.
Hospital-Based Physicians
Phase 1 of the ADVANCE project had enrolled physicians and patients from 2 hospitals associated with Summa Health System. Physicians from these hospitals who specialized in emergency or critical care medicine (n = 17) and spent at least 50% of their time working in a hospital setting were contacted by letter to participate. Each hospital-based physician was yoked to 1 or more primary care physicians to complete predictions for 1 patient in each of the 5 possible AD conditions. For example, if primary care physician 1 had completed predictions for 1 patient in all 5 conditions, then hospital-based physician 1 would complete predictions for the same 5 patients. However, if primary care physician 2 only made predictions for 2 patients in the HCD condition, then hospital-based physician 2 would make predictions for those 2 patients in addition to predictions for another primary care physician’s patients who were in the no-AD and VLA conditions. Therefore, fewer hospital-based physicians were needed to complete all of the predictions.
Hospital-based physicians were provided with basic demographic information about each patient (ie, age, sex, and race) but were blinded to patients’ names. They then reviewed the patient’s AD (when applicable) and made predictions on the LSPQ regarding patient preferences. After completing the LSPQ with predictions of the patient’s preferences, the hospital-based physicians also rated how confident they were that they accurately predicted the wishes of the patient on a 5-point scale and whether they found the AD helpful.
STATISTICAL ANALYSIS
Patients’ and surrogates’ responses on the LSPQ were dichotomized into “want treatment” (“definitely want treatment,” “probably want treatment,” or “unsure”) and “don’t want treatment” (“probably do not want treatment” and “definitely do not want treatment”) responses for each of the 35 treatment decisions. On the basis of past research, “unsure” responses were categorized with “want treatment” responses because in most instances the clinical default is to provide treatment unless specifically refused.17, 20, 24, 25 Analyzing data excluding “unsure” responses, treating “unsure” as a third response category, and treating “definitely” and “probably” as separate response categories produced no significant differences in study results. Proportion scores were generated for preferences and predictions made in each scenario (by summing the number of “want treatment” responses within each scenario and dividing by the number of treatment decisions in that scenario).
Three measures were generated as indicators of the accuracy of substituted judgments. Predictions were defined as accurate if, for a given treatment decision, the surrogate gave the same dichotomized response as the corresponding patient. Inaccurate predictions were further categorized into “overtreatment errors” (surrogate predicted the patient would want treatment and patient did not want treatment) and “undertreatment errors” (surrogate predicted patient would not want treatment and patient wanted treatment). Two approaches were used to compare the proportion of accurate predictions and of each type of prediction error across the 3 collapsed study conditions (ie, no-AD, HCD, VLA). First, an overall index was created by summing the number of accurate predictions (or alternatively, the number of overtreatment or undertreatment errors) for all treatments in all scenarios and dividing by the total number of predictions. Second, scenario indexes were created by summing the number of accurate predictions (or alternatively, the number of overtreatment or undertreatment errors) within each scenario and dividing by the number of treatment decisions in that scenario.
The overall ability of primary care physicians, hospital-based physicians, and families to predict patients’ treatment preferences was examined initially with repeated measures analyses of variance (ANOVAs) using the overall accuracy, overtreatment, and undertreatment indexes (ie, collapsed across all scenarios and treatments). When significant differences were found for an overall index, individual scenario indexes were then examined. Post hoc comparisons were then conducted using paired t tests with a Bonferroni correction for multiple comparisons.33
Next, accuracy by AD condition was examined for the 2 physician groups. A repeated-measures ANOVA with 1 within-subjects factor (type of physician) and 1 between-subjects factor (AD condition) was conducted on overall proportion accuracy, overall overtreatment, and overall undertreatment scores. For the between-subjects factor, Dunnett post hoc comparisons were used to examine differences between the means within each group. For the within-subjects factor, paired t tests with a Bonferroni correction were used for post hoc comparisons. When significant differences were found for an overall index, individual scenario and treatment indexes were then examined. Identical analyses and post hoc comparisons were conducted on the confidence measure for physician group by AD condition. To evaluate the extent to which significant results could be accounted for by within-physician variation, intraclass correlations were computed for those ANOVAs showing between-group differences. Since all intraclass correlations were nonsignificant, it is unlikely that within-physician variance significantly altered the present results.
RESULTS
SAMPLE CHARACTERISTICS
Descriptive information about primary care physicians, hospital-based physicians, and family surrogates is presented in Table 1. Primary care physicians were on average 39.8 years old and 59% were male. Most of the primary care physicians were European American (87.5%), married (87.5%), and had been trained as family practice physicians (79.2%). The majority of primary care physicians (76.4%) reported having made life-sustaining medical treatment decisions (ie, to withhold or withdraw treatment) less than 10 times in the past year. Approximately 24% of primary care physicians reported that they had an AD. The primary care physicians had seen the majority of patients in the sample (96.4% of patients) at least once in the past year and about half of the patients had been seen multiple times during the past year.
Similar to the primary care physicians, the hospital-based physicians had a mean age of 39.2 years, were European American (100%), and married (82.4%). Eighty-eight percent were male and the majority (70.6%) specialized in emergency medicine. Hospital-based physicians spend on average 94% of their time working in a hospital setting (range, 60%-100%). Compared with primary care physicians, a higher percentage of hospital-based physicians (35.7%) reported having an AD. The majority of hospital-based physicians (64.7%) reported having made life-sustaining medical treatment decisions (ie, to withhold or withdraw treatment) more than 50 times in the past year.
The demographics of the subsample of family surrogates did not significantly differ from those of the original sample of 401 family surrogates from the phase 1 study.27 Unlike the physician groups, the family surrogates had a mean age of 62.7 years and the majority were female (64.6%). The majority of family surrogates were European American (91.5%), married (87.8 %), and of the Protestant faith (59.8%). Patients and family surrogates had known each other for an average of 45 years. Information about ADs of these surrogates was not collected.
OVERALL ACCURACY OF SURROGATES’ PREDICTIONS
We first examined the overall accuracy of predictions of patient preferences, as well as accuracy in predictions by type of scenario for primary care physicians, hospital-based physicians, and family surrogates. To do this, we collapsed accuracy scores across the intervention conditions. These means are reported in Table 2. Overall, primary care physicians and hospital-based physicians were only accurate for an average of 0.66 and 0.64 of treatment decisions, respectively (Table 2), although accuracy scores differed by scenario. Accuracy scores were highest for “extreme” scenarios (ie, current health, coma with no chance of recovery, and terminal cancer with pain) when most patients wanted all or none of the treatment options.
When we compared the accuracy of the physician groups and family surrogates, we found differences for overall accuracy (F2,162 = 12.02, P<.001), and accuracy for the emphysema, stroke with no chance of recovery, stroke with a slight chance of recovery, cancer with no pain, and cancer with pain scenarios (all P<.05). Post hoc comparisons indicated that for overall scores, family surrogates were more accurate than both physician groups, while the 2 physician groups did not differ from each other. For emphysema, stroke with a slight chance of recovery, and cancer with no pain, family surrogates were more accurate than both physician groups. For the remaining 2 scenarios, only family surrogates were significantly more accurate than the hospital-based physicians. We next examined the types of errors made by the primary care physicians, hospital-based physicians, and family surrogates. In Table 2, family surrogates primarily made overtreatment errors; primary care physicians consistently made undertreatment errors. Hospital-based physicians made both types of errors, with slightly more being overtreatment errors. Overtreatment errors are slightly masked in Table 2 because means are collapsed across AD conditions. Comparing overtreatment errors across surrogate type, an overall difference was found (F2,162 = 6.76, P<.01), as well as individual scenario differences for the coma no chance scenario, and cancer with and without pain scenarios (all P<.05). Post hoc comparisons indicated that for the overall score and for the coma scenario, primary care physicians made the least overtreatment errors. Hospital-based physicians made the most overtreatment errors for both cancer scenarios. Comparing undertreatment errors across surrogate type, significant differences were found for overall scores (F2,162 = 24.18, P<.001), as well as the Alzheimer disease, emphysema, coma slight chance, stroke no chance, stroke slight chance, cancer no pain, and cancer with pain scenarios (all P<.001). Post hoc analyses indicated that family surrogates made the least undertreatment errors overall. Primary care physicians made the most undertreatment errors overall, and in the stroke slight chance and both cancer scenarios. Primary care physicians made more undertreatment errors than family surrogates in the Alzheimer disease scenario and both physician groups made more undertreatment errors than the family surrogates for the remaining 3 scenarios. ACCURACY OF SURROGATES’ PREDICTIONS WITH AND WITHOUT ADs The second purpose of the present study was to examine whether ADs improved predictive accuracy for primary care physicians and hospital-based physicians. Overall accuracy and accuracy in each scenario were compared for decisions made without the benefit of the patient’s AD, with a patient’s VLA, and/or with a patient’s HCD. A significant physician type [INLINE] condition interaction was found (F2,79 = 3.51, P<.05). Post hoc tests indicated that for primary care physicians, neither type of AD improved the accuracy of substituted judgments over not having the patient's AD. However, for hospital-based physicians, the HCD was effective in improving the accuracy of decisions compared with not having AD information (Figure 2). As a baseline comparison, family surrogate accuracy scores by AD condition are indicated in Figure 2 with a dashed line. At the level of the individual illness scenarios, a significant interaction was only found for the coma with no chance scenario (F2,79 = 3.41, P<.05). Like the overall scores, primary care physician accuracy was not improved by the ADs, although for the hospital-based physicians, the HCD significantly improved the accuracy of judgments over the no-AD control condition. Although no other scenario differences achieved statistical significance, there was a consistent trend for the HCD to improve hospital-based physicians' accuracy scores over the no-AD and VLA conditions (Table 3). We next compared the types of errors that the physicians made when using the ADs and with no AD. Overall overtreatment errors by surrogate type can be seen in Figure 3. (Again, baseline scores of family surrogates are indicated with a dashed line.) A significant physician type [INLINE] condition interaction was found for overall overtreatment scores (F2,79 = 5.57, P<.01), as well as for the coma no chance and stroke no chance scenarios (F2,79 = 4.19, P<.05 and F2,79 = 3.33, P<.05, respectively). Post hoc analyses indicated that when using the HCD, hospital-based physicians made fewer overtreatment errors than when they were provided with no AD. These findings were consistent across many of the individual scenarios, although the differences did not achieve statistical significance (Table 3). No significant differences were found for primary care physicians. In addition, no differences were found for the overall undertreatment index by physician type. Therefore, differences in individual scenarios undertreatment indexes by physicians were not examined. Finally, we examined whether ADs had any influence on the confidence that physicians had in the accuracy of their predictions of patients’ preferences (Figure 4). A significant physician type [INLINE] condition interaction was found (F2,79 = 4.36, P<.05). Post hoc comparisons indicated that hospital-based physicians were more confident in their predictions using either type of AD compared with no directive. Primary care physicians had no differences in confidence using either directive compared with no directive. COMMENT Three noteworthy findings are reported in this study. First, without the benefit of ADs, hospital-based physicians’ accuracy when predicting elderly outpatients’ life-sustaining treatment preferences is significantly lower than that of primary care physicians and family surrogates. Second, although primary care physicians were significantly less accurate in their decisions than family surrogates, neither a value-based nor a scenario-based AD helped to improve the accuracy of their substituted judgments for their patients. Finally, a scenario-based AD improved the accuracy of hospital-based physicians’ judgments in some scenarios, such that their level of accuracy reached similar levels of other well-acquainted surrogates. ACCURACY OF PHYSICIANS’ PREDICTIONS OF PATIENTS’ PREFERENCES Consistent with past research, primary care physicians were not highly accurate in predicting elderly patients’ life-sustaining treatment preferences and frequently predicted that patients would not want treatment when patients indicated that they would.15, 20, 26 Primary care physicians were most accurate in predictions for scenarios that can be considered extremes on a continuum of impairment (ie, current health, coma with no chance of recovery, and terminal cancer with pain). However, for scenarios between the extremes, where perhaps there may not be an overwhelming consensus for or against treatment, accuracy was considerably lower. Although hospital-based physicians often must make life-sustaining treatment decisions for patients with whom they have no preexisting relationship, little is known about the accuracy of these decisions compared with patients’ recorded wishes. We found that without ADs, hospital-based physicians have considerable difficulty making accurate predictions of patient preferences. Unlike primary care physicians who made undertreatment errors, hospital-based physicians made more overtreatment errors in their predictions. This may be due to the default assumption that in an emergency situation everything should be done to preserve life. Therefore, without explicit directions to withhold treatment, hospital-based physicians presented with an unfamiliar patient provided life-sustaining therapies. This may be particularly problematic given that in this study, hospital-based physicians overtreated in scenarios that involved significant pain, loss of reasoning abilities, and poor prognoses for recovery [INLINE] situations that many patients consider to be “fates worse than death.”32 Comparing the 3 types of surrogates (family members, primary care physicians, hospital-based physicians), our overall results are consistent with those of others who have found that family surrogates are more accurate than physicians.20, 21 However, a closer examination of accuracy by type of scen