Choose Your Words Wisely. How Laypeople’s Health Decisions are Shaped by Presentation Format

Table of Contents

Abstract
English Version
Deutsche Version

1 Empirical Background
1.1 Introduction
1.2 Communicating Risk
1.3 State of Research
1.4 Research Question

2 Method
2.1 Sample
2.2 Experimental Design
2.3 Test Material
2.4 Preparation for Data Analysis
2.5 Statistical Analysis

3 Results
3.1 The Impact of Presentation Format
3.2 Further Analysis

4 Discussion
4.1 Summary of Main Results
4.2 Integration of Results in Current State of Research
4.3 Effect Size and Power
4.4 Internal Validity
4.5 External Validity
4.6 Prospective future research

References

Appendix

Questionnaire

Supplementary Tables and Figures

Abstract

English Version

How efficacy data is presented influences information processing and treatment decisions of patients and health professionals. The present study investigated the impact of risk reduction format on the understanding and recall of treatment effects, as well as on the acceptance of the treatment in question. The effects of intelligence and prior experience were examined in an explorative way. In an online questionnaire, 172 laypersons read a hypothetical scenario of a visit to the dentist and the possibility to take paracetamol for pain relief. Participants received efficacy information either as absolute risk reduction, relative risk reduction or number needed to treat, interpreted these figures and indicated the likelihood of them taking the medication. In the end, they were asked to recall the initially presented effect. Risk reduction in relative terms was understood least well and more persuasive than in absolute terms. Prior experience with the medication had an impact on its acceptance. Probably, the overestimation of relative risk information mediates its higher persuasiveness.

Deutsche Version

Die Darstellungsart von Effektivitätsinformationen beeinflusst die Informationsverarbeitung und Therapieentscheidungen von Patienten und Medizinern. Die vorliegende Studie untersuchte den Einfluss des Risikoreduktionsformats auf das Verständnis und Behalten von Therapieeffekten sowie auf die Akzeptanz der betreffenden Therapie. Die Effekte von Intelligenz und Vorerfahrung wurden explorativ betrachtet. In einem Online-Fragebogen lasen 172 Laien ein hypothetisches Szenario eines Zahnarztbesuches, nach welchem sie die Möglichkeit hätten, Paracetamol zur Schmerzreduktion einzunehmen. Die Probanden erhielten die Effektivitätsinformation entweder als absolute Risikoreduktion, relative Risikoreduktion oder Number Needed to Treat, interpretierten diese und gaben die Wahrscheinlichkeit an, mit welcher sie das Medikament einnehmen würden. Am Ende wurden sie aufgefordert, den am Anfang präsentierten Effekt zu erinnern. Die relative Risikoreduktion wurde schlechter verstanden als die absoluten Maße und war überzeugender. Dabei hatte die Vorerfahrung mit dem Medikament einen Einfluss auf dessen Akzeptanz. Möglicherweise wird die stärkere Überzeugungskraft der relativen Risikoreduktion durch ihre Überschätzung vermittelt.

1 Empirical Background

1.1 Introduction

Under §223 of the ‘Strafgesetzbuch’ [German penal code], every medical operation does constitute bodily harm, unless - in accordance with §228 - the patient has been informed about the probable course and has agreed to the expected consequences previously. Not only are patients eligible for information about their disease and its course, but also they have a right to be informed about possible risks related to the treatment – the so called ‘risk disclosure’. Thus, proper doctor-patient communication is of particular importance. When asked about their opinion regarding mammography screening in an interview study by Davey, White, Gattellari, and Ward (2005), 42 percent of the 106 included women preferred an active role in decision making. This implies that patients also have a certain need to be involved in their therapy planning. An earlier study about how physicians give account of side effects when prescribing a new drug showed that only 24 percent of the physicians informed their patients about the mere existence of risks (Katz, Daltroy, Brennan, & Liang, 1992). Perhaps there is uncertainty about how to communicate treatment risks to patients, who are usually laypeople in the field of evidence-based medicine. Which changes have to be implemented to improve risk communication in medical settings? – To answer this question, one has to know how patients understand and interpret risk information and how this information shapes their health decisions. The present study investigated the comprehensibility, persuasiveness, and recall of different quantitative risk formats, with the aim of identifying the most appropriate format to be used in medical risk communication.

1.2 Communicating Risk

In evidence-based medicine, the results of randomised controlled trials are usually reported quantitatively as absolute risk reduction (ARR), relative risk reduction (RRR), or number needed to treat (NNT) (McHugh, 2008). These measures compare the effectiveness of a certain drug or therapy with the effectiveness of a control treatment, for instance placebo intake. As ARR represents the difference of the event rate between the treatment and control group, NNT is calculated as the inverse of ARR. The resulting figure can be interpreted as the number of persons needed to be treated for each case averted. This statistic takes into account that some patients will not get the desired result. Nevertheless, NNT has to be interpreted as a statement of probability with regard to the relevant population. Both ARR and NNT can be defined as absolute measures (Citrome, 2010). RRR represents the percentage reduction in the probability of an event in the treatment group compared to a control condition. It is always greater than ARR, with a particularly big difference in case of small baseline risks.

1.3 State of Research

The process of decision-making is influenced by the frame in which the concerning information is presented (Tversky & Kahneman, 1981). For instance, the effectiveness of a treatment is perceived to be larger if it is set in a ‘gain’ (e.g., reporting the number of patients surviving by the use of a medication) rather than a ‘loss’ frame (e.g., reporting the number of patients dying by the same medication). In theory, the presentation of treatment effectiveness in different risk reduction formats can also be described as framing and the perceptive variations thereby induced as framing effects. Insofar as framing is a considerably vague term and includes the decision-maker’s general conception of the premises associated with a particular choice (Tversky & Kahneman, 1981), hereafter the process of interest will be directly described as the impact of different risk reduction formats.

Malenka, Baron, Johansen, Wahrenberger, and Ross (1993) formed one of the first scientific working groups to investigate this process in patients. When having to choose between two equally efficacious medications to treat a hypothetical disease, one presented in relative, the other in absolute terms, more than half of the 470 included patients preferred the one given in relative numbers. An indifference between the two medications, which would be the rationally correct choice, was only stated by 16 percent of the patients. Supposing that patients are eager to identify the most effective treatment apparent in great figures, it seems that risk reduction in a relative format is perceived to be larger and therefore more likely to be chosen.

It stands to reason that this effect is at least partly driven by the different understanding and interpretation of absolute and relative risk formats. In fact, treatment effects presented as relative risk reduction are often erroneously interpreted as an absolute change and thus mostly overestimated - especially when given without baseline risk (Bodemer, Meder, & Gigerenzer, 2014). To counter those problems of comprehension, Schwartz, Woloshin, Black, and Welch (1997) call for a more understandable format. The number needed to treat, devised by Laupacis, Sackett, and Roberts (1988), was considered as a promising alternative. According to Citrome (2008), NNT helps to communicate clinical trial results effectively to patients as well as their families and the instances responsible for covering the therapy costs.

A number of studies systematically investigated the understanding as well as persuasiveness of treatment information presented as RRR compared to absolute measures like ARR or NNT. Here, it is important to point out the varying methodological implementations to correctly place the present study within the pool of previous research. Contrary to the initial enthusiasm with NNT, the results of a study by Sheridan, Pignone, and Lewis (2003) put NNT in a rather bad light. In comparison to RRR, ARR, and a combined format, subjects who received NNT were least able to identify the most profitable treatment out of two presented in the same format. Interestingly, subjects who received RRR performed better in the described task than those with ARR, although this difference could not be found when participants had to calculate the drug’s effectiveness on the given baseline risk of disease. The authors interpret the better performance in the RRR condition as a result of frequent exposure to relative measures in everyday life, for instance when calculating the reduced price of articles in sales. In contrast, Chao et al. (2003) found that participants were significantly less accurate in interpreting risk and benefit information when it was given as RRR in comparison to the ARR and NNT. Whereas the sample of Sheridan et al. (2003) consisted of actual patients aged 50 to 80 who waited to see the doctor, the one of Chao et al. (2003) contained medical students predominantly younger than 25 years. Another difference was the underlying hypothetical scenario: Sheridan et al. (2003) neither explicitly named the hypothetical disease, nor created a personal connection. On the contrary, Chao et al. (2003) instructed their participants to imagine their mother to be suffering from cancer and to advise an approval or rejection of chemotherapy.

A review of 35 studies (Akl et al., 2011), which included both publications previously described, revealed no significant difference in the understanding of RRR compared to ARR, but RRR was better understood than NNT. Furthermore, treatment efficacy was perceived to be larger when presented in relative rather than in absolute terms, whereas efficacy information presented as ARR was perceived to be larger than as NNT. If the effectiveness of a treatment is perceived to be larger depending on the applied presentation format, does this have any implications for its acceptance?

In fact, many studies showed that the presentation format of efficacy data influences patients’ acceptance of a treatment (e.g. Carling et al., 2009; Chao et al., 2003; Hux & Naylor, 1995; Misselbrook & Armstrong, 2001). Hux and Naylor (1995) manipulated format in a within- design, whereby the participants (outpatients) received several descriptions of lipid lowering drugs, respectively as ARR, RRR, and NNT. In reality, all three descriptions covered the effect of one and the same drug. In case of RRR, 88 percent of the outpatients approved therapy, whereas ARR elicited 42 percent and NNT 31 percent approval. Here, participants should imagine suffering from hyperlipidaemia and were thus not directly affected. However, a comparative study with hypertensive and normotensive control-patients found the same effect and no moderating influence of subjective involvement on the impact of presentation format on treatment acceptance (Misselbrook & Armstrong, 2001). Overall, the results regarding the decision-shaping role of presentation format are consistent in previous research (Akl et al., 2011). Compared with ARR and NNT, RRR is perceived to be larger and is more likely to be persuasive. That is, a treatment with its efficacy described in relative terms may elicit stronger acceptance when compared to absolute measures. Here, it is difficult to say whether the higher persuasiveness speaks in favour of or against a relative risk presentation. However, if this higher persuasiveness results from a fundamental misinterpretation of RRR, the perception of the effect’s magnitude seems to be distorted and thus, relative risk presentation might be less appropriate for supporting informed treatment decisions.

Differences concerning the extent of the presentation format’s impact on understanding and acceptance are due to the respective empirical design and thus moderating variables (Covey, 2007). The presentation of baseline risk alongside risk reduction figures improves understanding, especially when baseline risk is given as a frequency instead of percentage (Bodemer et al., 2014; Covey, 2007). In general, natural frequencies are better understood than percentages (Akl et al., 2011). Misinterpretations and greater persuasiveness of RRR is more likely when it is worded as just a percentage (e.g., ‘34% reduction in heart attacks’) rather than worded with emphasis on relativity (e.g., ‘heart attacks are reduced by 34%’ or ‘a relative reduction in heart attacks of 34%’) (Covey, 2007). Most of the studies found that presentation format also influences understanding and acceptance in health professionals, and there seems to be no difference of its extent in comparison to laypeople (Akl et al., 2011). Last but not least, numeracy – the ability to comprehend and use numerical information (Reyna, Nelson, Han, & Dieckmann, 2009) – and education are positively correlated with the understanding of efficacy data (Bodemer et al., 2014; Schwartz et al., 1997; Sheridan et al., 2003).

In previous research concerning the impact of presentation format, participants were mostly confronted with a hypothetical scenario of a serious disease in which a certain medication could prevent the occurrence of symptoms (e.g., a lipid-lowering drug prevents heart attacks in patients with hypertension). Although Misselbrook and Armstrong (2001) did not find any difference in the impact of presentation format between hypertensive, affected and normotensive, unaffected patients, a critical point is that uninvolved study participants may not be able to put themselves in all those hypothetical situations (Moxey, O’Connell, McGettigan, & Henry, 2003). Moreover, a decisional situation in which patients do exclusively get treatment information in one sentence and without any explanation is highly unlikely. Even if ‘lack of time’ is considered as a barrier to evidence-based care in general practice (Ward, Gordon, & Sanson-Fisher, 1991), it seems possible for physicians to shortly explain the given figures or for the patients to at least look up the meaning of a risk reduction format. This of course requires a sensitization of patients as well as health professionals for this type of information and its relevance.

As far as we know, no study ever investigated the impact of presentation format on the recall of the treatment effect. This variable could be important because health decisions are not always made ad-hoc, but can require some time. Desirably, patients should then still be able to make an informed treatment decision on the basis of the previously presented efficacy information.

In the present study, participants received a hypothetical but realistic scenario of a visit to the dentist and should afterwards interpret effect information and indicate how likely they would take an advised medication. At the same time, the given presentation format and its meaning were explained in short. After participants completed a distraction task, they were asked to recall the initially described drug and its effect.

1.4 Research Question

The underlying research question for the present study was the following: How does presentation format influence the understanding and recall of treatment effects, as well as the acceptance of a certain drug described by these formats?

Because of previous research results indicating that relative risk reduction is likely to be overestimated (Bodemer et al., 2014), and NNT is poorly understood (Akl et al., 2011), we hypothesise: Risk reduction given as ARR is better understood than as RRR and as NNT. NNT is the least well understood of the three. Consistent with the reported findings concerning format influence on acceptance, the second hypothesis is: A relative measure of risk reduction is more persuasive than an absolute one. NNT is least persuasive. Thirdly, we hypothesise: There are different recall rates for risk reduction given as ARR, RRR, and NNT. Additionally, we investigated the following explorative research question: Do intelligence and experience with the used medication have an impact on the comprehension, persuasiveness, and recall of efficacy information?

2 Method

2.1 Sample

Data assessment was realised by an online questionnaire which was distributed by mailing lists containing various universities in Germany, as well as by calls of participation in social networks and on the website of the journal ‘Psychologie Heute’. Of 402 persons who logged on to the study site, 181 gave their informed consent to participate. To analyse the format effects on understanding and persuasiveness, nine participants with suspected dentist phobia were excluded (exclusion items explained below). This resulted in a sample of 172 persons (68 percent female), aged 15 to 58 (M = 27.28, SD = 10.03). For the analysis of the effects on recall, this sample was additionally reduced by 27 participants for whom the distraction task seemed not to have succeeded (exclusion items explained below), to a total of 145 persons (70 percent female) with a similar age span (M = 27.07, SD = 9.65). Participants were randomly assigned to three experimental groups, which resulted in an unbalanced experimental design. Baseline characteristics (including group sizes) are summarised in Table A.1 and the sampling process is visualized in Figure A.1.

2.2 Experimental Design

As a cover story, participants were told that the authors are interested in how people process the information in package leaflets of drugs. Originally, the dependent variables were comprehension of the effect information, persuasiveness of the treatment presentation, and recall of the effect information. The independent variable - presentation format, which comprised three factor levels, was manipulated in a between-group design. This means each group received treatment effect information either as ARR, RRR, or NNT. Additionally, numeracy and educational level as proxy variables for intelligence and prior experience with analgesics, especially paracetamol, were assessed. The completion of the questionnaire, described in detail below, took approximately 15 minutes.

2.3 Test Material

The questionnaire was self-constructed, taking into account the test material of previous work in this field (e.g. Bodemer et al., 2014; Hux & Naylor, 1995; Schwartz et al., 1997). The assessment of comprehension, persuasiveness, and effect recall depending on presentation format refers to the following hypothetical scenario presented to participants, immediately following their informed consent: Subsequent to a planned tooth extraction in the dentist’s surgery, there is the opportunity to receive analgesics, more precisely paracetamol (600mg), for oral intake to be relieved of postoperative pain. Participants were informed that the costs would be completely covered by health insurance and were provided with a list of possible side-effects adapted from the original package leaflet of paracetamol. According to the randomised allocation, the participants then received information about the treatment effect of the given drug either as ARR, RRR, or NNT, calculated from a systematic review of ten placebo-controlled studies (Moore, Collins, Carroll, & McQuay, 1997). The results of this review are summarised in Table 1 and the concrete calculation of the risk reduction figures is specified in Table 2. The described hypothetical situation was chosen because it is highly likely that every person has experienced such a visit to the dentist or could at least vividly imagine it. Supplementary to this effect information, the given presentation format and its meaning were explained in one sentence (see Appendix).

Table 1 Experimental and Control Event Rate for Treatment with Paracetamol (from Moore et al., 1997)

illustration not visible in this excerpt

Note. In the systematic review by Moore et al. pain relief information was extracted and converted into dichotomous information (number of patients with at least 50% pain relief).

Table 2 Calculation of Different Risk Reduction Formats (using data from Table 1)

illustration not visible in this excerpt

Note. ERc = Error rate control group (Placebo), ERT = Error rate treatment group.

To assess understanding of the treatment effect information, participants had to estimate the remaining risk of suffering from postoperative pain in the experimental group after treatment in a forced-choice paradigm adapted from Bodemer et al. (2014), whereby the answer options were randomised (‘Please estimate the risk of suffering from pain despite of paracetamol intake.’). To ensure that the right answer technically can be calculated, they additionally received baseline information, that is the percentage of people who are relieved of pain by only taking placebo. The randomised response options were attuned to common mistakes in interpreting RRR reported in previous research (Bodemer et al., 2014; Sheridan et al., 2003). The two distractor options refer to the erroneous interpretations of RRR as an absolute reduction or as the event rate in the treatment group.

Then participants had to indicate the likelihood of them taking the drug outlined above in the hypothetical situation on an eleven-point scale anchored at zero, ‘far from likely’ to ten, ‘highly likely’. It was decided not to use a response format containing percentages in order to avoid an anchoring heuristic bias evoked through the information about risk reduction given in a percentage format beforehand (Tversky & Kahneman, 1974).

To examine recall ability of information given in the three different formats, participants firstly completed the personality inventory NEO-FFI (Costa & MacCrae, 1992), including 60 items to be answered on a five-point Likert scale, as a distraction task. The original instructions were adjusted for online implementation. Additionally, three items were created and mixed in to assess avoidance and fear, as well as the occurrence of vegetative symptoms before or during a visit to the dentist. Another three mixed-in items directly specified the answer option to be chosen for the purpose of checking the participants’ concentration. In this way, participants with a probable dentist phobia and those who did not read the items carefully could be identified and excluded. Afterwards, participants had to recall the described drug by choosing from a list of five randomised non-opioid analgesics including paracetamol and furthermore depict the correct magnitude of the treatment effect out of five randomised options.

Numeracy was then assessed as a covariate through the ‘Schwartz et al. three-item scale’ (Schwartz et al., 1997), Cronbach’s α = .55 and mean inter-item r = .32 (see Appendix for German translation). Information about analgesic intake was assessed by three questions: At first, participants had to specify how frequently they took analgesics in the past four weeks, then they were asked whether they had ever taken paracetamol and at last, participants had to indicate if they had taken paracetamol in the past four weeks. The reference frame of four weeks was chosen in order to impede retrospective distortion. At the end of the questionnaire, demographic data including sex, level of education, and an ongoing or completed course of study in psychology was collected.

2.4 Preparation for Data Analysis

To avoid possible inferences, participants with a suspected dentist phobia were excluded before the first analysis. The assessment of associated symptoms was based on the diagnostic criteria of a specific phobia listed in the ICD-10 (Dilling, Mombour, & Schmidt, 1991), which are either avoidance or fear of a specific object or situation and relating vegetative symptoms. It can be expected that reading the hypothetical scenario evokes fear or uneasiness in these persons and this could distort further data (Jerremalm, Jansson, & Öst, 1986). For the analysis of effect recall, cases in which the distraction task seemed unsuccessful were additionally excluded. Here, the exclusion criterion was more than one incorrect concentration item out of three. In these items, participants should choose a prescribed answer option on a five-point Likert scale. If they did not do so in more than one case, it can be assumed that the personality questionnaire was not worked through attentively and thus the distraction was not successful.

The prior randomisation check revealed no differences between the three groups according to baseline characteristics (see Table A.1 for p -levels). However, the variables ‘number of psychology students’ and ‘paracetamol intake in the last four weeks’ showed a rather unequal distribution between the three groups, partially with a trend towards significance.

2.5 Statistical Analysis

Analysis was carried out in R (R Core Team, 2015). To verify the success of randomization, the characteristics of participants who received each presentation format were compared, using χ2-tests for categorical variables and an analysis of variance for the continuous variable ‘age’. The relationship between ‘presentation format’ and the three dependent variables was assessed using a one-way analysis of variance (fixed factor, three factor levels), whereby a-priori planned contrasts were computed. Additional analyses of variance were run to examine the impact of intelligence and prior experience with paracetamol. To calculate statistical power, the program G*Power (Faul, Erdfelder, Lang, & Buchner, 2007) was used.

3 Results

3.1 The Impact of Presentation Format

Overall, 27 percent of the participants correctly chose the remaining risk of pain in the treatment group of ‘60%’, which is less than the proportion expected in case of pure chance (33 percent for three answer options). Almost half of the participants chose ‘25%’, that is, the interpretation of the risk reduction as directly referring to the remaining event rate in the treatment group. Table 3 shows the distribution of answers for all three groups.

Table 3 Distribution of answers to comprehension question

illustration not visible in this excerpt

Note. ‘60% of the treated patients still suffer from pain’ would be the correct answer.

A-priori planned contrasts revealed that participants were more likely to choose the correct answer with risk reduction given as ARR in comparison to RRR and NNT [ t (171) = -1.679, p = .048*, d = 0.273]. RRR did not elicit more correct answers than NNT [ t (171) = 1.018, p = .845, d = 0.189], descriptively even fewer. Consequently, the first hypotheses can be confirmed insofar as ARR is understood better than RRR and NNT, whereby NNT could not be shown as the risk reduction format which is least understood.

The effect of the underlying analysis of variance [ F (2,169) = 1.998, p = .139, ηp2 = .023] was small, according to Cohen’s classification (1988). Taking the covariates ‘level of education’ and ‘numeracy’ into account, the partial eta squared shrinks to .015, whereas ‘paracetamol intake in the last four weeks’ does not change the effect size. If all three covariates are included, the analysis of variance reveals an increased but still small effect (ηp2 = .025) of ‘presentation format’. None of the covariates had a significant main effect on ‘understanding’ (all p s > .34). The inclusion of participants with a suspected dentist phobia would not alter the significance of the results reported above.

The likelihood of taking the described drug was assessed on an eleven-point scale that can be transformed into percentages for better illustration, so that zero corresponds to a likelihood of zero percent and ten corresponds to a likelihood of 100 percent. When provided with risk reduction information as RRR, the average likelihood of participants to take paracetamol was 77 percent, whereas for ARR it was 68 percent, and for NNT 66 percent (see Figure 1).

[...]