1.3 Aptitude-Treatment-Interactions (ATI)
1.3.1 Learning strategies
1.3.2 Cognitive ability
1.3.5 Openness to experience
1.3.6 Goal orientation
1.3.7 Achievement motivation
2.2 Training Design
2.4 Personal Variables
2.5 Dependant Variables
2.6 Manipulation Checks and Controls
3.1 Preliminary Analysis
3.2 Manipulation Checks
3.3 Performance Effects
3.4 Interactions of Personal Variables and Training Method
4.1 Strengths and Limitations of the Study
4.2 Implications for Theory and Future Research
In many fields of modern life, training has become a very prominent method of personnel development. There are training programs for jobless persons to be reintegrated into workforce, office workers who need to be trained in new data processing systems, software experts to keep up with the rapid development of technology and many, many more. One of the largest area of training is the field of software training (Graf, 2001). We cannot ignore the presence of personal computers in nearly every aspect of modern life. But in times when money is limited, it is essential to look for the most cost effective forms of training.
One very effective method that has been examined for nearly fifteen years in the field of software training is the error-management-training. This training sets its focus on free exploration and the making of errors, who are seen as something useful to learn from. Trainees are confronted with so called error-management-instructions ("You can just learn from errors.", "There is always a way to leave the error situation – don't panic.") that should take away the negative emotionality of the error. Trainees are encouraged to explore issues on their own and use errors as feedback that shows the trainee what is still to learn. Several studies in the past were able to show that this kind of training leads to better performance than classic tutorial trainings that guide the trainee step by step to the solution of a task (e.g., Greif & Janikowski, 1987; Dormann & Frese, 1994; Frese, 1995; Ivancic & Hesketh, 2000). We will refer to this guided approach as error-avoidant training because the detailed instructions should prevent the learner from making errors. Some authors plea for a combination of both methods (Debowski et al., 2001).
Recently, these findings have been reconsidered according to the idea that there are personal characteristics that might interact with the training method. Cronbach and Snow (1981) called such interactions of personal characteristics, ability, motivation or attitudes with a certain treatment aptitude-treatment-interactions (ATI). Gully, Payne, Koles, and Whiteman (2002) found such interactions in the way that error-management-training should be beneficial for persons with high cognitive ability and openness to experience whereas error avoidant training should be beneficial for persons with high conscientiousness. Unexpectedly there was no general performance superiority of the error-management-training in this study. An explanation for this missing effect could be that Gully et al. (2002) used the last training trial in a radar tracking task as performance measure. A consequence of this procedure is that error-management-training participants were encouraged to make errors (by the training instructions) in their performance phase. So the results of this study should be interpreted with care. Nevertheless the authors argue for a consideration of personal characteristics in training research. Heimbeck, Frese, Sonnentag, and Keith (2003) examined the interaction of goal orientation and training method on task performance and found a benefiting effect of error avoidant training for people with a high prove and avoidance goal orientation. The hypothesized positive effect of error-management-training on high learning goal oriented people could not be demonstrated. Furthermore Heimbeck et al. (2003) compared an error training group with error-management-instructions to an error training group without error-management-instructions and an error avoidant group. The results showed that the error-management-instructions are essential for error-management-training to work because the error training group with error-management-instructions showed better performance than the other two groups who had no difference between them.
We agree with Gully et al. (2002) and support the research of ATI for two reasons. First, the findings from those studies can give information about what kind of training fits for what kind of person (practical implication). Second, ATI could provide hints about how and why error-management-training works (theoretical implication). For example when we get to know that personal characteristics (e.g., habitual learning strategies) are beneficial for training success in the error-avoidant training but have no impact in the error-management-training, we can make inferences about mechanisms of the error-management-training assuming that this training method might induce such characteristics. The study at hand takes up the results of former studies and tries to broaden the knowledge about the work mechanisms of the error-management-training approach. First, we examine the effect of an error avoidant training group that is provided with error-management-instructions which was – to our knowledge – never done before. Second, we explore the impact of a large number of personality variables that potentially interact with the training method using a large sample size as proposed by Gully et al. (2002), because ATI are difficult to detect in small samples.
In the following sections we give a short summary of the characteristics of error-management-training and ATI. After introducing the examined personal variables we describe the method of this study followed by the results. In the discussion section we try to interpret the findings and take a look at the strengths and limitations of the study at hand.
In every learning situation people make errors because of insufficient knowledge or skills, inappropriate goal setting and planning, interruptions during action or wrong interpretation of system feedback (Zapf, Brodbeck, Frese, Peters, & Prümper, 1992). In general, errors are linked with negative emotions and are said to lead to frustration, anger and despair (Brodbeck, Zapf, Prümper, & Frese, 1993). For a long time errors were seen as something that should be avoided to prevent the trainee from adopting wrong routines (Ausubel, 1968). Skinner (1953) equated errors with punishment that leads to emotional arousal and can therefore paralyse the learner and prevent him or her from further action. The result of this belief was the famous programmed learning machine which guides the trainee through the learning task and tries to minimize the possibility of errors to enhance learning by giving positive reinforcement (Skinner, 1968).
The error-management-training approach proposes just the opposite of these opinions. Errors are considered a functional part of the learning process and a very useful source of feedback (Frese & Altmann, 1989). This feedback is crucial to the learning process to evaluate the adequacy of action. So the principle of error-management-training is free exploration of the training task with the possibility of making a great number of errors. In practice the trainee gets a short introduction into the training material (e.g., a short description about the basic functions of a software program) and is then left to explore the training tasks on his or her own. Here the initiation of the 'Undo'-function is important because novices tend to go from one error to the next and therefore get deeper and deeper in the error state (Frese &Altmann, 1989). This is why a possibility to leave the error state is essential. Besides this, going backward in action to the point where the error occurred is the most used strategy to eliminate the error (Zapf, Lang, & Wittmann, 1991). In contrast to the concept of exploratory learning (Bruner, 1966), a concept very similar to error-management-training, the training tasks in error-management-training are rather difficult from the beginning, so there is the possibility of numerous errors. Trainees have to formulate and test hypotheses (by risking and making errors) concerning the training issue and can therefore develop a more sophisticated mental model of the software than trainees who just follow step by step instructions to the solution of the training task (Irmer, Pfeffer, & Frese, 1991).
At this point a very close linkage of error-management-training to action theory becomes clear. According to Hacker (1998) the operative image system (OIS) is such a mental model of all action relevant information a person has. It is the sum of internal long-term representations of condition-action-result interrelations and is therefore the cognitive basis of action regulation. It is to be seen as something like a cognitive model of all work situation-relevant issues. That means that an OIS is not a complete representation of the reality but a cost-optimizing system that contains only the action-relevant procedures. The difference to other cognitive systems is that an OIS is clearly action-oriented (which does not mean that it denies non-action-oriented knowledge). An OIS does not regulate actions but the better it is sophisticated and developed the better is the quality of the actions based on this specific OIS (Frese & Zapf, 1994).
In an error-management-training the construction of a well fitted OIS should be enhanced by the process of setting subgoals, developing hypotheses, testing them and using the feedback of occurring errors to further elaborate the existing OIS (Frese, 1995). During error-management-training the OIS should therefore become more appropriate than in an error-avoidant training, where participants only learn fixed routines and have no opportunity to test the adequateness of their OIS.
This process of elaboration of a mental model has an important consequence. In contrast to a guided, error avoidant form of training where you do not have to spend much cognitive resources following the instructions, error-management-training forces trainees to exert a great amount of effort into the solving of the training task and apply a deeper level of processing. According to Craik and Lockhart's (1972) level of processing approach Marton and Säljö (1976a) mention two approaches to learning: a) surface approach like repeating the learning contents very often to remember them; b) deep approach that connects the learning task to existing knowledge structures for a better understanding. The authors demonstrated that the level of processing can be experimentally manipulated and that these manipulations have effects on learning outcomes (Marton & Säljö, 1976a, 1976b). Students seem to adopt one of these approaches depending on their expectations of what is required of them (Marton & Säljö, 1976b). In error training participants are explicitly told to explore the material and perform a self guided learning. They are instructed to get down with the material to understand it. This fact and the difficulty of the training tasks should suggest a deep level of processing approach to the participants. In consequence the usage of deep level processing should lead to a superiority of error-management-training concerning performance.
To demonstrate this superiority performance measures are assessed in a separate test phase after the training. Tasks in this phase are again rather difficult and exceed the material presented in the training. Participants of an error-management-training who are expected to have a better mental model of the training material due to deeper level processing and exploration showed a better performance than participants in an error-avoidant training (Greif & Janikowski, 1987; Frese & Altmann, 1989; Dormann & Frese, 1994; Frese, 1995; Ivancic & Hesketh, 2000, Heimbeck et al., 2003). In contrast to participants of error-avoidant training who did not encounter error-management-instructions and are affected by the negative effects of errors, error-management-trainees have acquired error-management-competences because they got used to deal with errors and see them as learning opportunities (Heimbeck et al., 2003).
By presenting tasks that were not introduced in the training the test phase resembles the later real life situation at work because there will always be issues in complex training material that could not be covered in the training. But then they have to be dealt with without the help of a trainer. So error-management-training prepares trainees best for transfer to real life work situations.
The error-management-instructions are another fundamental part of error-management-training. These are rules of thumb that tell the learner about the usefulness of errors. Examples are: "There is always a way to leave the error situation.", "I have made an error. Great!" (Frese et al., 1991); "Errors are a natural part of the learning process. They inform you what you are still able to learn." (Heimbeck et al., 2003). These rules are said to work by two mechanisms (Heimbeck et al., 2003). The first one is a cognitive mechanism. The error-management-instructions enhance exploration because they stress the learning chances errors provide. People who made errors and are told that they could learn something from that should tend to start exploration why this error occurred. Furthermore these people should make more use of the negative feedback of errors (Dormann & Frese, 1994). The second mechanism functions in an emotional way. As mentioned above errors tend to induce stress and interrupt actions. This leads to frustration and despair. The error-management-instructions should take away these negative emotions by stating that there is no reason for panic, that errors appear everywhere, and that there is always a way to deal with them. It is even desired to make errors and there is no reason for shame. This is why error-management-training should only work in combination with these error-management-instructions. That this is the case was demonstrated by Heimbeck et al. (2003) who compared an error training group with error-management-instructions to an error training group without error-management-instructions and an error-avoidant training group. The results showed a superiority of the error training with error-management-instructions over the other two training conditions.
But there are still unanswered questions concerning these error-management-instructions. Are the effects of error-management-instructions limited to the knowledge acquisition phase or do the benefits carry over to impact later performance situations? The fact that the error-management-training effect on performance is a lasting one could be demonstrated by Heimbeck et al. (2003) by assessing performance at two different times. They found that the superior results of error-management-training participants were stable over a period of one week. If these better results are due to a better program knowledge (because of a better OIS) and a better way of dealing with errors (error-management strategies induced by the error-management-instructions) it seems likely that the effects of the error-management-instructions should work in the test phase and further on as well. To get indications for this assumption we need training participants who receive error-management-instructions but have no opportunity to benefit from their effects in the training. An error-avoidant training group with error-management-instructions meets this requirement exactly because there is no opportunity for errors in the training due to the detailed step by step instructions to task solution. If these trainees show better task performance in a test phase than trainees who did not get error-management-instructions, the results would indicate that the error-management-instructions work in the test phase because this is the first time participants of the error-avoidant training group could make errors and use the benefiting effects of the error-management-instructions. One might argue that it is not very reasonable to provide people with error-management-instructions when there is obviously no chance to make errors but the fact that there is always the possibility of clicking the wrong button or typing an incorrect key inadvertently should make these statements sound reasonable to participants in this kind of training.
In sum, we think that error training with error-management-instructions leads to highest performance because this approach supports a deeper understanding of the material through the usage of a deep level of processing and combines this with an error-management approach because of the error-management-instructions. Because error training without error-management-instructions and error-avoidant training showed no difference in the Heimbeck et al. (2003) study we expect similar results. Error-avoidant training with error-management-instructions should lead to better performance than the two training conditions without error-management-instructions because of the effects of the error-management-instructions in the test phase. In other words we predict a main effect of error-management-instructions and an interaction of these instructions with the training method (error training versus error-avoidant training). These effects should occur only for difficult tasks because for easy an medium tasks a deep understanding of the training material seems not necessary. See also Figure 1.
Abbildung in dieser Leseprobe nicht enthalten
Figure 1. Prediction of a main effect of error-management-instructions and an interaction of error-management-instructions and training method on performance
a) Training methods with error-management-instructions will show higher performance for difficult tasks and declarative knowledge than training methods without error-management-instructions.
b) Error-management-instructions will interact with training method on performance for difficult training tasks and declarative knowledge in the way that there will be a benefiting effect of error training when error-management-instructions are given. There will be no effect of training method when error-management-instructions are not given.
1.3 Aptitude-Treatment-Interactions (ATI)
According to Snow (1992), aptitudes are "initial states of persons that influence later development, given specified conditions" (p. 6). Snow sees this concept closely connected to readiness, susceptibility and proneness. It allows the examination of different treatments in the light of variables like personality, motivation, interests, attitudes or ability. Borg and Gall (1989) state that "the purpose of aptitude-treatment-interaction (ATI) research is to determine whether the effects of different instructional methods are influenced by cognitive or personality characteristics of the learner" (p. 700).
The idea is that error training might not be a 'one size fits all' approach to potential development. There is the possibility that different kinds of personal variables may have an impact on the usefulness of error training for different individuals.
As mentioned above, Gully et al. (2002) found some moderating effects of cognitive ability, openness to experience as well as conscientiousness and training method on test performance, declarative knowledge and self-efficacy. The authors come to the conclusion that error-management-training is not good for everyone. But the fact that they used the last training trial as a performance measure leads to a confoundation of training and test phase what puts these findings in question. Heimbeck et al. (2003) demonstrated that prove goal orientation and avoidance goal orientation interacted with training method in the way that high prove goal oriented and high avoidance goal oriented persons benefited from error-avoidant training but trainees in the error training with error-management-instructions still showed the best performance. The hypothesized positive effect of error-management-training on high learning goal oriented persons could not be found. Heimbeck et al. (2003) explained this by assuming that error-management-training is possibly a very strong situation which leads - according to Mischel (1977) – to a decreased impact of personal variables on human behavior. Mischel (1977) stated that situations that provide a clear and explicit frame for individuals to behave can be seen as "strong" situations whereas situations that leave more autonomy and less structure are considered "weak". Beaty, Cleveland, and Murphy (2001) provided evidence that the relationship of personality variables with contextual performance is moderated by situational characteristics (whether a situation is strong or weak). At first sight error-avoidant training seems to be the stronger situation according to the differentiation of Mischel (1977). But is this really the case? Hattrup and Jackson (1996) name different situational attributes on which situations can be classified as strong or weak: a) information from the environment, b) attributes of the task at hand, c) physical characteristics, and d) social norms. Information from the environment and social norms play in important role in our issue. On the other situational characteristics the training methods did not differ in our study.
Attributes of the task can be constructs that reflect the amount of autonomy provided by the task (Hattrup & Jackson, 1996). We already argued that error-management-training forces its participants to exert a great amount of effort to develop hypotheses and test them by exploring the issue. So the attributes of the tasks are designed in a way where you have no choice but to learn from your errors. Otherwise you will not manage to solve the task. In the error-avoidant training you are explicitly told what to do and this should be clear to everyone. But inside, participants can choose to simply follow the step by step instructions without exerting much effort in understanding the underlying principle of the software or to use more elaborated learning strategies. So from an external point of view error-avoidant training seems to meet the requirements of a strong situation, but from a cognitive point of view inside the individual, error-management-training may be a much more stronger situation by letting the trainees no choice how to learn the material but explore, make errors and learn from them whereas error-avoidant training participants seem to have more freedom of choice. So we think, error-management-training forces people to behave in a certain way and therefore does not provide much autonomy in knowledge acquisition.
In error-management-training the error-management-instructions should replace the actual social norm that says "errors are bad" by creating a very powerful social norm that says "errors are useful" and stimulates people to make errors. This norm is always present because the error-management-instructions are usually displayed before the eyes of the trainees during training sessions and are constantly repeated by the trainer.
These assumptions lead us to a different hypothesis than the one from Gully et al. (2002) or the goal orientation hypothesis from Heimbeck et al. (2003). Instead we follow the proposition of Heimbeck et al. (2003) and assume for all personal variables that error-management-training is able to wipe out the impact of such a personal variable that influences performance in error-avoidant training. So we included cognitive ability, conscientiousness, openness to experience, and goal orientation in the study at hand because these variables were examined in the two studies mentioned above. In addition we looked at achievement motivation and learning strategies because these variables are closely connected to the assumed processes in error-management-training (effort and deep level of processing strategies). We included neuroticism as well because we think that this construct is linked to fear of errors and an impairment of performance in case of interruptions of action (Eysenck & Eysenck, 1968). Errors could be a reason for such an interruption.
a) There will be no beneficial or detrimental effects on task performance for difficult tasks and knowledge tasks of any personal variable in error training.
There will be beneficial or detrimental effects on task performance for difficult tasks and knowledge tasks of personal variables in error-avoidant training.
For predictions concerning the direction of the personal variables' effects in error-avoidant training see the respective section (1.3.1 to 1.3.7).
1.3.1 Learning strategies
With increasing age learners are more and more expected to self regulate their learning processes (Wild, Schiefele & Winteler, 1992). Even in elementary schools there is a call for an explorative self guided approach to learning (Barnitzky, 1996). Pintrich and Garcia (1991) describe the term 'self regulated learning' as a set of learning strategies that the learner uses to manage a task in a flexible and effective way. In this context the term 'learning strategies' is not a well defined construct but a concept that refers to a behavior that enables the learner to manage learning tasks (Wild, Schiefele & Winteler, 1992). Related expressions are learning tactics (Pintrich & Garcia, 1991) or practices that people use to aid the acquisition and development of knowledge (Kardash & Amlund, 1991). Different concepts of learning strategies differ in their separation or combination of cognitive and motivational strategies (Wild, Schiefele & Winteler, 1992).
For the cognitive aspect of learning Craik and Lockhart (1972) proposed a level of processing approach in memory research that relates the amount of knowledge that is learned in a learning situation to the level of processing. According to this assumption Marton and Säljö (1976a) mention two different levels. The first one is a surface level which is characterized as a learning of facts and details only to master tests. The other level is a deep approach to learning like an active integration of the issue into the existing knowledge and a deeper understanding of the learning task. These approaches to learning are closely connected to strategies that are used by the learner (Cantwell & Millard, 1994). The most cited cognitive learning strategy categories are rehearsal strategies, organization strategies and elaboration strategies (e.g., Wild et al., 1992; Holman, Epitropaki, & Fernie, 2001). Holman et al. (2001) performed a factor analysis with participants from a call center and generalized these categories from educational situations to a non educational organizational setting. Organization and elaboration strategies are considered deep level strategies whereas rehearsal strategies represent the surface level.
So there should be a difference between people who use only surface level strategies and people who use the deeper level of processing strategies or both. The latter are expected to have a deeper understanding of learned material (Anderson, 1996) and therefore should perform better in more difficult learning tests and transfer tasks (these are tasks with issues that are not explicitly presented in the training). Keith (2003a) introduced the idea that in error training participants are expected to use this deep level of processing because they have to explore the learning tasks on their own. They have to formulate and test hypotheses and integrate the gained knowledge into their mental model of the training issue. This means that error training forces all participants to use such deep level strategies and therefore should produce a better performance in later tests for all trainees no matter what their habitual learning strategy is. In an error-avoidant training the trainees are not automatically forced to use deep level strategies, they can learn just by using rehearsal strategies for the presented instructions. So in such a training trainees who usually use surface level learning strategies in learning situations should perform worse than people who use deep level strategies habitually when they learn. These effects are expected for difficult tasks. For easy tasks the surface level strategies should be benefiting because they are not that complex and therefore a deeper understanding of the material is not necessary.
Hypothesis 2 b):
High scores for deep level learning strategies should be beneficial (and low scores for deep level learning strategies should be detrimental) for task performance for difficult tasks and declarative knowledge in error-avoidant training but not in error training.
High scores for surface level learning strategies (and low scores for surface level learning strategies should be detrimental) for task performance for easy tasks and declarative knowledge in error-avoidant training but not in error training.
1.3.2 Cognitive ability
Cognitive ability has been found to be a very good predictor of training performance (Ree & Earls, 1991). Individuals with a high cognitive ability or intellectual capacity show a better performance in nearly every job or job criteria (Hunter, 1986). In this study we use the general concept of the g-factor for cognitive ability. Our prediction for error training is that this training method is able to wipe out the effect of cognitive ability on performance and knowledge measures by forcing all participants to use deep strategies. In the error-avoidant training in which there should be no wipe out effect, it is reasonable to assume that high cognitive ability leads to high performance.
Hypothesis 2 c):
High cognitive ability should be beneficial (and low cognitive ability should be detrimental) for task performance for difficult tasks and declarative knowledge in error-avoidant training but not in error training.
Highly conscientious people are said to be reliable, planful, efficient, organized and precise (John, 1999). We assume these people to carry out the step by step instructions in an error-avoidant training very reliably and precisely. This should lead to a good training performance and can also affect test performance (at least for tasks that resemble the training tasks). Zhang (2003) found conscientiousness to be correlated with deep learning strategies so one could expect highly conscientious people to use such deeper levels of processing that lead to more understanding of the training issue and therefore to a better performance in difficult tasks. Another aspect of conscientiousness is cautiousness (Costa & McCrae, 1992). If highly conscientious people are also very cautious they might feel pretty safe in an error-avoidant training where errors are nearly impossible to occur. Because of these reasons we predict that error-avoidant training will lead to higher performance for highly conscientious individuals. On the other hand we expect error training to be able to force individuals scoring low on conscientiousness to be planful, organized and to use deep strategies and therefore level out the differences in training outcome due to differences in conscientiousness.
Hypothesis 2 d):
High conscientiousness should be beneficial (and low conscientiousness should be detrimental) for task performance for difficult tasks and declarative knowledge in error-avoidant training but not in error training.
People with a high level of neuroticism are said to be anxious, nervous and moody. They worry much and tend to be self-pitying and self-punishing (John, 1999). People scoring high on neuroticism have difficulties to get back to normal condition after a very emotional incident and tend to emotional over-sensitivity (Eysenck & Eysenck, 1968). This might make any situation with unknown material and the connection with a performance test a stressful situation for these people. Therefore individuals scoring high on neuroticism are not likely to benefit as much from a training as people scoring low on neuroticism. There have been some earlier attempts to find a relation between neuroticism and performance or learning (Schneller & Garske, 1976; Kline & Gale, 1971) but no clear results could be found. There are suggestions that the relation of basic personality traits and performance is mediated by learning strategies (Blickle, 1996). Zhang (2003) found neuroticism to be positively related with surface level strategies. Highly neurotic people should tend to use surface level strategies. As we mentioned above error training is said to force all participants to use the deep level of processing that should lead to better performance in difficult, complex tasks and might therefore be able to wipe out differences in performance - mediated by learning strategies - due to neuroticism.
Hypothesis 2 e):
Low neuroticism should be beneficial (and high neuroticism should be detrimental) for task performance for difficult tasks and declarative knowledge in error-avoidant training but not in error training.
1.3.5 Openness to experience
Individuals with a high openness to experience are said to be widely interested, curious and insightful (John, 1999). They welcome new unusual experiences without anxiety (Fitzgerald, 1966). This personality dimension was found to be correlated with training proficiency and might determine how willing a person is to engage in a learning experience and how positive a persons attitude is towards training (Barrick & Mount, 1991). The latter was found to correlate with training performance (e.g., Ryman & Biersner, 1975). This far we follow the arguments of Gully et al. (2002). But we predict that these influences of openness to experience will only play their role in an error-avoidant training. Openness to experience was found to be correlated with deep level learning strategies (Zhang, 2003). Blickle (1996) found a factor called 'elaboration' in a factor analysis of learning strategies who was also correlated with openness to experience. Error training may be able to equalize an individual's way to deal with new tasks by forcing all participants to explore the new issue and do not care about anxiety (counteracted by the error-management-instructions) as well as use a deep level of processing.
Hypothesis 2 f):
High openness to experience should be beneficial (and low openness to experience should be detrimental) for task performance for difficult tasks and declarative knowledge in error-avoidant training but not in error training.