The Time Course of Negative Priming

Contents

1 Introduction
1.1 Negative Priming
1.2 Computational Modeling of Negative Priming
1.3 Thesis Overview
1.4 Original Contributions

2 Negative Priming
2.1 A Paradigm to Access Selective Attention
2.2 A Showcase Negative Priming Experiment
2.3 The Negative Priming Effect
2.4 Theories of Negative Priming
2.4.1 Distractor Inhibition Theory
2.4.2 Episodic Retrieval Theory
2.4.3 Response Retrieval Theory
2.4.4 Feature Mismatch Theory
2.4.5 Temporal Discrimination Theory
2.4.6 Dual Mechanism Theory
2.4.7 Global Threshold Theory
2.5 Summary

3 Imago Semantic Action Model
3.1 Deriving Simple Activation Dynamics
3.1.1 Networks of Integrate-and-Fire Neurons
3.1.2 Network Response to Input Onset and Offset
3.1.3 Exponential Fixpoint Dynamics
3.2 Implementation of the ISAM
3.2.1 Representation Variables
3.2.2 Visual Input
3.2.3 Interference of Semantically Identical Objects
3.2.4 Adaptivity of the Threshold
3.2.5 Response Generation
3.3 Computational Results
3.3.1 Comparison to the Experimental Data
3.3.2 Dependence on the Response Stimulus Interval
3.3.3 Variation of Distractor Saliency
3.3.4 Predictions for Single-Object Trials
3.4 Discussion
3.4.1 Modeling Priming
3.4.2 Phenomenological and Neural Models
3.4.3 The Implementation of the Model
3.5 Summary

4 EEG Correlates of Negative Priming
4.1 Introduction to Electroencephalography
4.1.1 EEG Recording
4.1.2 Data Processing
4.2 Review: ERP Correlates of Negative Priming
4.2.1 N200 Component
4.2.2 P300 Component
4.2.3 Positive Slow Wave Component
4.2.4 Summary of ERP Correlates
4.3 Hypotheses
4.4 Experimental Setup
4.5 Data Analysis
4.6 Behavioral Results
4.7 ERP Results
4.8 Discussion
4.9 Conclusion
4.10 Summary

5 Interlude: Advanced EEG Analysis
5.1 EEG Analysis in Cognitive Research
5.2 Models for Event-Related Potentials
5.3 Dynamic Time Warping
5.4 Pyramidal Averaging Dynamic Time Warping
5.5 Trial Clustering for Cleaner Averages
5.6 Enhancing Averaging by Integrating Time Markers
5.7 Recurrence Plots to Obtain the Warping Function
5.7.1 Recurrence Plots
5.7.2 Phase-Space Reconstruction
5.7.3 Line-of-Synchrony Detection in Cross-Recurrence Plots
5.7.4 An Algorithm for Line-of-Synchrony Detection
5.7.5 Results
5.8 Summary

6 Perception or Selection Effect
6.1 Task Switch Paradigm
6.1.1 Sequence of Experiments
6.1.2 Task Switch and Negative Priming
6.1.3 Condition Set
6.2 Task Switch and the ISAM
6.2.1 Extension of the ISAM
6.2.2 Calibration
6.2.3 Pre-Cue Simulation
6.2.4 Post-Cue Simulation
6.3 Hypotheses
6.4 Preparatory Task Switch Experiments
6.4.1 Design
6.4.2 Participants
6.4.3 Procedure
6.4.4 Data Analysis
6.4.5 Results, Baseline Experiment
6.4.6 Results, Pre-Cue Experiment
6.4.7 Discussion
6.5 Post-Cue Task Switch Experiment
6.5.1 Design
6.5.2 Participants
6.5.3 Data Analysis
6.5.4 Results, Stimulus Identification Phase
6.5.5 Results, Target Selection Phase
6.5.6 Results, Comparison of Partial Reaction Times
6.5.7 Discussion
6.6 General Discussion
6.7 Summary

7 Selection or Response Effect
7.1 Gaze Shift Paradigm
7.2 Hypotheses
7.3 Gaze Shift Experiment
7.3.1 Design
7.3.2 Participants
7.3.3 Procedure
7.3.4 Extraction of Partial Reaction Times .
7.3.5 Analysis of Behavioral Data
7.3.6 EEG Data Analysis
7.4 Results
7.4.1 Response-Repetition Effect
7.4.2 Partial Reaction Times
7.4.3 EEG Correlates
7.5 Discussion
7.6 Summary

8 The General Model for Negative Priming
8.1 A Framework to Test all Negative Priming Theories
8.1.1 Different Paradigms
8.1.2 Inclusion of Theories
8.2 Characterizing System Components
8.2.1 Feature Layers
8.2.2 Semantic Representations
8.2.3 Episodic Memory
8.2.4 Memory Retrieval
8.2.5 Central Executive
8.3 Implementation of the General Model
8.3.1 Feature Variables
8.3.2 Feature Binding Mechanism
8.3.3 Semantic Variables
8.3.4 Short-Term Modulation of Connectivity .
8.3.5 The Adaptive Threshold in the Semantic Layer
8.3.6 Action Variables
8.3.7 Memory Processes
8.3.8 Connectivity Modulation
8.3.9 Generating Real World Reaction Times .
8.4 Defining Setscrews for the Theories .
8.5 Voicekey Paradigm
8.6 Word Picture Comparison Task
8.7 Discussion
8.8 Summary
8.9 Simulation Plots

9 Conclusion
9.1 Computational Modeling in Psychology
9.2 EEG Correlates
9.3 Behavioral Paradigms Beyond Response Latencies
9.4 The Time Course of Negative Priming
9.5 Summary and Outlook

Bibliography

A Tables
A.1 Tables of Chapter 2
A.2 Tables of Chapter 3
A.3 Tables of Chapter 4
A.4 Tables of Chapter 6
A.5 Tables of Chapter 7
A.6 Tables of Chapter 8

Publication List
Publications
Invited Talks
Talks at Conferences
Posters at Conferences
Theses
Awards

Acknowledgements

1 Introduction

Theoretical life in psychology seemed just a forever-long sequence of dichotomies.

(Newell, 1994)

The present thesis reports on an interdisciplinary attempt at explaining the negative priming effect, a characteristic of selective attention, by a combination of behavioral fundamental research, neuroimaging studies, theoretical psychology and computational modeling. The negative priming effect is one of a very few measures for ignoring. It is revealed as a slowing down of responses to stimuli that were ignored recently as compared to those that are new. Since the discovery of the negative priming effect in 1966 a vivid debate on the cognitive mechanisms underlying the deceleration has evolved, without arriving at a conclusive consensus.

Over the years, a large number of negative priming experiments have been conducted, mostly focusing on a special aspect of the effect by the use of a unique paradigm. Regrettably, the results of each of these studies show a unique pattern as well. Only the bare slowing of responses to previously ignored stimuli is found in most of the experiments, but virtually any manipulation of a paradigm also affects negative priming.

In the introduction we will explain the negative priming phenomenon in section 1.1, giving a condensed overview of the exhaustive presentation of the field of negative priming in chapter 2. We then expound the importance of computational modeling for theoretical psychology in section 1.2. The structure of the thesis is presented in section 1.3 which also describes our research on negative priming as a whole. Finally, in section 1.4 we will conclude the introduction by listing the original contributions included in this thesis.

1.1 Negative Priming

Selective attention enables goal-directed behavior despite the permanent, immense input to the sensory system. The downside of this ability involves the problem of how information is ignored. Contradicting early speculations of an active attending and passive ignoring, a special situation revealed the active nature of ignoring. In the original experiment by Dalrymple-Alford and Budayr (1966), subjects had to process lists of Stroop tasks. While in the original Stroop task no systematic repetition of color and color words was implemented, these authors composed the stimulus cards in a special way, namely the ignored meaning of a color word always became the to be named color in which the next word was shown on some of the lists, on others there was no relation between two succeeding words. The experiment showed that people were slower in responding to the related lists compared to unrelated stimulus colors. Even if the semantic meaning of the words has been ignored, it must have entered the cognitive system as it showed the characteristic interference.

Since then, several standard negative priming paradigms have emerged, each featuring a certain dimension on which priming happens, e.g. the identity of stimulus objects or their location on the presentation screen. The set of stimuli also varies enormously, e.g. pictures, shapes, words, letters, sounds, colored dots. The common denominator of all paradigms is the classification of stimuli in targets that have to be attended to, and distractors that are to be ignored. Stimulus repetitions are considered in dependence of the role of the repeated object as target or distractor in two related trials. Variations of this basal setting include the manipulation of experimental parameters like the time between two related trials, the number of distractors from zero in some trials to multiple over the entire experiment, and the saliency of the distractor. For a detailed listing of the sometimes contradictory results, see section 2.3.

Because of the controversial nature of the negative priming effect, a variety of different theo- retical accounts have been developed. But until now none of the theoretical accounts is able to explain all aspects of the negative priming effect, they all have their strong points as well as their shortcomings. All theories assume different mechanisms to be responsible for negative priming. In order to clarify the situation of diverging explanatory accounts, the time course of negative priming is crucial. The mechanisms postulated by the different theories act in different stages of trial processing.

1.2 Computational Modeling of Negative Priming

The theories to explain negative priming can be categorized roughly as memory-based and activation-based accounts. The first group assumes the memorization of a trial and eventually a retrieval of the information in the next trial. The latter group assumes negative priming to be caused by interference of trial processing with persistent activation from former trials. Both directions produced a variety of small branches, many limited to a single appearance in order to explain a certain, singular pattern of results. But due to the lack of comparability and concreteness, there is no solution of the debate on the level of argumentative theories in sight.

In the face of such a sensitive phenomenon it is understandable that no comprehensive expla- nation has been found so far. Because a satisfying theory should be less complex than the data it explains, it seems reasonable to focus on the interaction of the underlying causes instead on ad hoc defined data features. However, a main reduction of complexity is already achieved by the design of experiments. Nevertheless, a theoretical approach is based on the assumption that the complexity of experimental data can be further reduced by identifying repeating patterns in the data. Our first attempt, the implementation of a simplified but still biologically motivated model of target selection has proven too simple by our experiments. Although it provided us with a tangible account of several dependencies of negative priming. A crucial point in the specification of mech- anisms producing negative priming seems to be the exact time course of processing a trial where a previously ignored stimulus has to be attended in comparison with the processing of an unprimed stimulus. Therefore, we faced the problem to reveal temporal information about negative priming.

A first step in that direction is already our simplistic computational implementation as described by continuous nonlinear differential equations which themselves show a characteristic time course. In order to test the validity of the model we designed several experimental paradigms according to the objective of making statements about the inner temporal structure of a negative priming trial. Some of the experiments recorded EEG data which has shown to be a beneficial tool in identifying systematic differences in trial processing, both spatially and temporally. For two experiments we developed techniques to record additional time makers during trial processing, making it possible to temporally localize the emergence of reaction time effects. In order to tackle the problem of diverse paradigms and the incomparability of theoretical accounts, we designed a computational framework for perception based action selection by means of physiologically justified building blocks which each obey a biologically plausible dynamics.

Despite all concise and generally understandable theories that seem to have identified the causes of an observed phenomenon, it is very important to keep in mind that psychological fundamental research uses statistical properties of experimental data in order to interprete human behavior. On the one hand, behavioral experiments tend to produce largely varying results, caused by the complexity of the human cognitive system. On the other hand, the interpretation of results is usually not unambiguous. Both aspects provide a base for the arduous and controversial discourse that is necessary for clarifying a certain psychological phenomenon.

One possibility to proceed in the discussion is to solidify theories by mapping their assump- tions on measurable processes in the brain, thereby eliminating arbitrariness of the respective interpretations. A second way is to computationally implement theories. Clearly, the obtained implementation inherits the freedom of interpretation from the underlying theory. Yet, the imple- mentation adds further degrees of freedom. But the benefits of an implementation are obvious. It eliminates the risk of misinterpretation, as the source code can be made available for other research groups interested in working with the model instead of leaving them with wordy descriptions. A computational model may provide links to biological data, all the more if it is based on naturally observable processes.

Nevertheless, certain aspects have to be remembered when arguing on the level of implementa- tions. In order to reproduce the observed results, most models have to undergo a precise fitting of parameters, which is also a very subjective process. Therefore, great care has to be taken of the distinction between results due to parameter fits and extrapolations by the internal dynamics of the model itself. A different way to benefit from a computational model is to analyze the structural result after fitting, which carries a formalized version of the fitted data. Or, in the words of Hintz- man (1991): The measure of a model’s value lies not in its ability to fit data, but in how much we can learn from it.

We will comply with the necessity of quantification in two ways. First we will take up a single theory of negative priming, i.e. the imago semantic action model described in section 2.4.7, and build a minimal model producing realistic effects on the basis of the postulated mechanisms, see chapter 3. A detailed implementation was performed in close interaction with the originator of the theory. The presence of the cognitive representation of a certain object is modeled by a single variable, by which we obtain a rather clear dynamic system which is able to deal with realistic stimulus sequences and generates artificial reaction times. In chapter 6 we will show how the model can be extended to generate hypotheses in a more complex paradigm. The generalized model enables us to resolve contradictions arising in the initially attempted modeling approach. This is to be considered as a success of the modeling process, as we are able to falsify an essential assumption of the original theory by means of a straightforward implementation.

The second computational approach is more ambitious with respect to the discussion about the applicability of the theories of negative priming in specific situations. We build a computational model comprising most of the mechanisms suspected to play a role in the neural processing in negative priming. The outcome is not only a meta-model for negative priming, termed General Model, but in itself a simplified model of the brain as a framework for action selection based on perception. We addressed the tradeoff between biological realism and understandability by modeling each assumed mechanism separately but keeping the internal dynamics of each of the corresponding layers very simple by taking over the dynamical framework of our first model.

1.3 Thesis Overview

The present thesis will describe our multi-level approach to reveal the temporal structure of the negative priming effect. Accompanied by computational modeling, we run sophisticated psychological experiments and record and analyze EEG data. We will start with an overview of the phenomenon of negative priming in chapter 2.3 by reviewing the literature for behavioral results and theoretical explanations of the effect.

Based on one of the theoretical accounts introduced in chapter 2.3, namely the Imago Semantic Action Model, see section 2.4.7, we will implement our first computational model for negative priming, the ISAM. The basics of the modeling approach and the implementation will be the first part of chapter 3. The second part will be devoted to testing the ISAM by deriving predictions and reproducing several effects related to negative priming.

Adapting the voicekey paradigm described in section 2.2, we will describe an EEG experiment in chapter 4 that replicates findings from one of the few studies on event-related potentials related to negative priming. Beforehand we will give a detailed introduction to EEG recordings and the corresponding data analysis and thoroughly review hitherto existing findings of EEG correlates of negative priming.

During the preoccupation with EEG data analysis, we came upon an inconsistency in averaging event related potentials. Chapter 5 introduces our solution to the problem to reconstruct a very noisy signal that additionally is subject to erratic temporal fluctuations. As such a new technique first has to prove its validity in a broad discussion, we limit it in the current thesis to an interlude independent of the remainder.

Due to the additional source of uncertainty in EEG research, i.e. the interpretation of differ- ing event related potentials in the different experimental conditions, we determined ourselves to behavioral experiments and designed a paradigm which requires a button press between stimulus identification and target selection phase which is recorded as an additional reaction time. Chapter 6 describes the model based generation of hypotheses by the ISAM of chapter 3, the paradigm itself and finally the results that locate negative priming in the later part of a trial and that contradict the ISAM all along the line.

After separating the stimulus identification phase, the remainder of a trial still contains the two stages of processing of target selection and response generation. One theory predicts negative priming to be exclusively produced in the response generation phase. Therefore, we constructed a second trial splitting paradigm which now singles out the response generation phase. In chapter 7 we will describe the paradigm, go into expected side effects of the altered paradigm and finally display the results, the devotion of negative priming to the target selection phase of a trial.

Not only the nontrivial extension of our identity based priming paradigm given in chapter 7 to a comparison task, but also the counterevidence for the ISAM by the experiment in chapter 6 made us head for a less rigid computational model. Chapter 8 pictures the result in form of our General Model for negative priming which provides an implementation of each theory and the ability to respond in various different negative priming paradigms. Due to the complexity of the model chapter 8 can only be seen as the general introduction to a new framework which will possibly shed light on the questions why different paradigms produce such diverse result patterns, and how the theories can be compared on a par.

The previous chapters are concluded in chapter 9 which also collects all results and forms a complete picture of negative priming as we can give it by our research. This chapter contains also an outlook on future directions to finally conclude the main body of the thesis. Appended is a listing of experimental data in tables, which were excluded from the according chapters for the sake of readability.

1.4 Original Contributions

All work presented in the present thesis is carried out by a closely cooperating workgroup in the framework of section C4 of the Bernstein Center for Computational Neuroscience Göttingen. The results presented here would not have been possible without this collaboration. My personal contributions are not restricted to modeling but have had an increasing influence also on experimental design, data analysis, interpretation of results, and design of algorithms.

- Our main contributions to (but not limited to) negative priming research are listed in the follow- ing.
- We developed a simple model for the transient of the firing rate response of an integrate and fire network to constant input by the means of a nonlinear Langevin equation, section 3.1.
- We employed the resulting dynamics to build a minimalistic computational model, section 3.2, reproducing priming effects based on the mechanisms of the global threshold theory, section 2.4.7.
- With the good performance of the model, section 3.3, we quantitatively validated global threshold theory (Schrobsdorff et al., 2007b).
- We adapted our voicekey paradigm, section 2.2 to an EEG recording environment, section 4.4 and replicated some of the very sparse event related potential correlates for negative priming found in a rather different paradigm, section 4.6.
- We confirmed that processing in ignored repetition trials first benefits from stimulus repe- tition similar to the attended repetition condition, but only later in the trial both conditions diverge due to different demands on cognitive control, section 4.8, (Behrendt et al., 2009)
- We developed sophisticated signal processing methods, sections 5.4 and 5.7, which enhance the averaging of event related potentials, section 5.7.5, and provide a measure for the tem- poral variation in the processing between two trials, section 5.5, (Ihrke et al., 2008, 2009b).
- We designed an enhanced algorithm for line-of-synchrony detection in recurrence plots which outperforms established solutions, section 5.7.4, (Ihrke et al., 2009a).
- We introduced time markers in addition to the usual reaction time into negative priming paradigms, making it possible to investigate the temporal structure of the mechanisms causing negative priming by means of behavioral measures, section 6.1 and 7.1.
- By applying our technique of recording intermediate time markers, we have shown that the stimulus identification phase of a trial carries no negative priming, but only facilitation in the presence of repeated objects, section 6.6.
- By deriving predictions from our computational implementation of the global threshold theory to the task switch paradigm, section 6.2, we provided strong counterevidence for that theory as predicts negative priming to happen already in the identification phase, sections 6.3 and 6.6.
- We showed that negative priming happens in the target selection phase of a trial, section 7.5, by again isolating a part from trial processing, in this case the response generation phase, section 7.1.
- Finally we implemented a neurophysiological model, section 8.3, of the parts of the brain that are assumed to be involved in processing a priming trial, section 8.2. The General Model is able to cope with various paradigms, section 8.1.1, and implements the behavior assumed by any of the negative priming theories, section 8.1.2.

Although partially not yet published as articles, all points are documented by a series of conference contributions listed on page 162 ff. and are available at www.nld.ds.mpg.de/~hecke/research.html

2 Negative Priming

Priming is characterized by a sensitivity of reaction times to how stimuli have been encountered recently. A reduction of the reaction time, positive priming, is usually observed with repetitions of stimuli or responses and is well-known and experimentally understood (Scarborough et al., 1977). Our object of investigation, negative priming, a slowdown in the reaction time usually in response to previously ignored stimuli, is experimentally less tangible (Fox, 1995). The negative priming effect is sensitive on even subtle parameter changes, which poses many methodological and conceptual challenges, but bears exactly for this reason great potential for applications in research fields such as memory, selective attention, and aging effects.

The following chapter will thoroughly introduce the negative priming phenomenon. After a classification of negative priming and a description of the terminology used in negative priming studies in section 2.1, we will discuss a showcase study to give a feeling for what a negative priming experiment looks like in section 2.2. The diversity of findings concerning negative priming will be shown in section 2.3. Then we will give a detailed listing of theoretical accounts to the negative priming effect in section 2.4.

2.1 A Paradigm to Access Selective Attention

Selective attention is the process of extracting behaviorally relevant information from the environment. The focusing on particular stimuli brings along an ignoring of irrelevant information. The process of ignoring is investigated by systematic variation of irrelevant stimuli. Interesting effects like change blindness, the failure to perceive even striking changes in a visual scene that are not behaviorally relevant (McConkie and Currie, 1996), or inattentional blindness, the apparent insensitivity of the cognitive system to unattended stimuli (Simons and Chabris, 1999), demonstrate impressively that our feeling of perceptual accuracy is not objective.

It is still unclear how the selection of stimuli is done. Two classes of mechanisms are assumed, top-down and bottom-up processes (Anderson, 2001). The first process actively guides the attentional focus by highlighting particular features of current interest. The latter one describes selection due to perceptual saliency. In everyday tasks, both of them interact.

As selection and ignoring are two sides of the same medal, the nature of ignoring is crucial, as distracting information can easily be varied in experiments, and thus gives access to the act of selection itself. Even if early attempts assumed a passive ignoring, empirical evidence for an active process comes from the inhibition of return paradigm (Milliken and Tipper, 1998). A prolonged reaction time is observed if a location which has been in the focus shortly before is required to be attended to.

A general approach to the processing of distracting stimuli is provided by the negative priming paradigm. Negative priming is often considered the most direct approach to assess the selective aspect of attentional processing, as the ignored, distracting stimuli can be proven to be actively processed (Houghton and Tipper, 1994).

Selective attention has to permanently deal with distracting information. Most paradigms we will discuss in the following show two items in each trial. One is to be attended, called the tar- get, while the other one, the distractor, is behaviorally irrelevant and has to be ignored. One such selection trial primes the subject for the next trial. Therefore, a pair of two successive trials is labeled prime and probe respectively. Generally speaking, the repetition of a target stimulus in two successive trials leads to a faster response. This effect is called positive priming. In contrast the presentation of a prime distractor as a target in the probe trial may lead to a deteriorated per- formance compared to a target which has not been presented immediately before. The behavioral slowdown indicates that irrelevant information is not passively ignored, but actively processed, as no effect on reaction times in subsequent trials would have been found if the information about the distractor is not present.. Negative priming has been found in a wide variety of experimental contexts and is therefore thought to be a reproducible and general phenomenon, see section 2.3.

In the present thesis we will rely on the following definition: Negative priming is a slowdown in reaction time in an ignored repetition condition, where a former distractor has become relevant. As we associate the term negative priming with reaction time differences, we can not use it as a label for the ignored repetition condition but rather chose the condition labels according to the configuration of stimuli in a trial, see (Christie and Klein, 2001). The first letter in the sequence contains information about which part of the prime display is repeated in the probe display. A D represents the distractor, a T the target. The second letter indicates the role the particular object has in the probe display, see table 2.1. For example, the string DT refers to the condition in which the prime distractor (first letter D) is repeated in the probe trial as target (second letter T), hence it is a traditional negative priming trial. In case both objects are repeated there is a second pair of letters appended for the second object. Because a target and a distractor are shown in the prime and the probe display each, and target and distractor are never identical, seven relevant combinations of target-distractor relations are conceivable, see table 2.1.

Illustration not visible in this excerpt

Table 2.1: The seven possible priming conditions of a paradigm with one target and one distractor in each of the prime and probe display.

2.2 A Showcase Negative Priming Experiment

To get a first idea of the characteristics of negative priming, we will now discuss a rather straight- forward study which can be taken as starting point for all our experiments. The study was part of our publication (Schrobsdorff et al., 2007a). We use a visual identity priming task where the target is selected by means of its color and then responded to according to its identity. The paradigm was introduced by Tipper (1985) and has been used in the Göttingen gerontology group for some years and has been optimized in several ways. Since negative priming tends to disappear with problem complexity, identity priming with simple stimuli is an option that maximizes negative priming in this respect. For efficiency reasons we present the trials continuously, such that every trial primes the subject for the following trial.

Illustration not visible in this excerpt

Figure 2.1: Objects used as target and distractor stimuli with their German labels.

Stimuli are six different objects, represented by hand-drawn pictograms, see figure 2.1, that are either shown in green or in red color. We use voice recording together with a sound level threshold to determine the reaction time for every trial. As the experiment is run in German, the names begin with a plosive and consist of a single syllable (Bus, Ball, Baum, Buch, Bett, Bank) for a sharp, thus easily detectable onset of the sound signal. Object presentation is balanced in the different priming conditions as well as their appearance as target and distractor. Priming conditions CO, DT, TT, DDTT and DTTD are a repeated measures factor.

An exemplary sequence of displays of seven trials is shown in figure 2.2. One stimulus display consists of two overlapping line drawings, a green target and a red distractor object. Stimuli appeared entirely in the focal area. The subject is instructed to name the target objects aloud and ignore the superimposed red objects. They were told to answer as quickly and as accurately as possible. Then, after a blank screen period and the presentation of a fixation cross, the next display is presented.

Illustration not visible in this excerpt

Figure 2.2: Example of a sequence of stimuli. Consecutive screens are shown. Either stimuli or a blank screen followed by a fixation cross is displayed. The meaning of the acronyms is explained in section 2.1. E.g. in the sequence of the second and third stimulus displays, the tree switches from red distractor to green target, but the other two items are unrelated, a DT condition, see table 2.1.

An experimental session starts with a test of color discriminability and memory span, followed by a familiarization with the stimulus objects. Of the entire 420 experimental trials in 10 blocks of 42 trials, 400 trials are analyzed (80 trials of each priming condition), while the first two start trials of every block are excluded from analysis. Each trial consists of the following displays: a fixation cross, centered on the screen for 500 ms, a display containing superimposed pictures shown up to the response, but not longer than 2 seconds and a blank screen for 1000 ms, see figure 2.2. An error is registered when subjects failed to give a correct, clear answer. Participants are 12 adults, 4 male and 8 female, mean age 23.6 years, SD = 4.6.

Illustration not visible in this excerpt

Figure 2.3: Reaction times for the five experimental conditions. Note that the positive priming effects on the left side are larger than the negative priming effects on the right hand side.

Mean reaction time of the different priming conditions, standard deviation, and the effect strength, i.e. the difference to CO trials, are shown in figure 2.3 reported in table A.1 in ap- pendix A.1. Erroneous trials (2.4%) are excluded from analysis. Trials with response latencies less than 250 ms or more than two standard deviations above the individual mean of each prim- ing condition are excluded as well (4.7%). DTTD trials produce the slowest responses, followed by DT and CO trials, whereas the responses to TT trials are faster than control and DDTT trials produce the fastest responses.

For statistical analysis, a one way analysis of variance (ANOVA) was used to explore the effects of CO, DT, DTTD, TT and DDTT. The alpha level for all analyses was set at 0.05. Greenhouse-Geisser corrected degrees of freedom are used as the data violated the assumption of sphericity. Reaction times depend significantly on the priming condition F(1.45, 15.93) = 23.27, MSE= 1938.83, p < 0.001. Planned comparisons show that reaction times in DT and DTTD tri-

Illustration not visible in this excerpt

Figure 2.4: Four different paradigms for negative priming: a) location b) flanker task c) voicekey identification d) word-picture comparison task. In all examples green defines the tar- get. The location priming paradigm reveals negative priming in the encoding of space. The flanker task implements a rather difficult stimulus response mapping, whereas vo- calization in the voicekey paradigm is a very easy task. The word-picture comparison paradigm has the advantage of a disentanglement of target identity and response.

als were significantly increased, as compared to CO trials (CO vs. DT: t(11) = −3.57, p < 0.01; CO vs. DTTD: t(11) = −3.37, p < 0.01). As anticipated, the reaction time for trials in the at- tended repetition conditions TT and DDTT were significantly decreased (CO vs. TT: t(11) = 3.11, p < 0.01; CO vs. DDTT: t(11) = 4.74, p < 0.001). Directed comparison of attended repetition conditions reveal a difference of reaction time (TT vs. DDTT: t(11) = 6.11, p < 0.001), whereas the reaction time of ignored repetition conditions did not differ (DT vs. DTTD: t(11) = −0.60, p = 0.558).

The experiment shows how the repetition of stimuli can influence reaction times in a negative priming paradigm. A repetition of relevant stimuli leads to a prominent speedup of the trial processing, whereas a repetition of irrelevant stimuli as target causes a slowdown of the reaction. The results document the important fact: irrelevant stimuli are not filtered out by early perception mechanisms, but rather are subconsciously processed.

2.3 The Negative Priming Effect

Negative priming is subject to fervid discussions among cognitive theorists for several decades. The negative priming effect was discovered first in a Stroop task (Dalrymple-Alford and Budayr, 1966), subjects have to name the ink in which a color-word is written. A usually strong tendency to read the word has to be suppressed. Reaction times are delayed in trials where the color of the ink in the probe is identical to the word in the prime. The semantic meaning of the word serves as the distractor, because it has to be ignored in order to be able to correctly name the color of the ink. The results were replicated in similar settings by Neill (1977).

Negative priming is present in a wide variety of experimental contexts (for reviews see Fox, 1995; May et al., 1995; Tipper, 2001; Mayr and Buchner, 2007). For example, negative priming has been elicited using different stimuli such as line drawings (Tipper and Cranston, 1985), letters (Neill and Valdes, 1992; Neill et al., 1992), words (Grison and Strayer, 2001), auditory stimuli (Buchner and Steffens, 2001; Banks et al., 1995; Mayr and Buchner, 2006) and nonsense shapes (DeSchepper and Treisman, 1996). Negative priming has been obtained with manual (Neill and Valdes, 1992; Tipper et al., 1992) and verbal responses (Allport et al., 1985; Tipper and Cranston, 1985), as well as in situations where the mode of response changed between prime and probe (Chiappe and MacLeod, 1995). Furthermore, negative priming has also been found in various tasks including naming (Tipper, 1985), same-different matching (DeSchepper and Treisman, 1996), Stroop-like tasks (Neill, 1977) and spatial localization (Milliken et al., 1994; Park and Kanwisher, 1994; May et al., 1995; Kabisch, 2003).

In spite of the obvious universality, the negative priming effect is sensitive to a variety of pa- rameters. Most paradigms show their individual aspect of negative priming, but no global pattern of results exists (Fox, 1995). It has been shown that negative priming can depend on the length of the response stimulus interval (RSI) between prime and probe (Neill et al., 1992; Kabisch, 2003; Frings and Eder, 2009). But there are also studies reporting a constant negative priming effect for varied RSIs (Tipper et al., 1991; Hasher et al., 1991, 1996). Surprisingly for very short RSIs, a DT condition can produce a facilitatory (Lowe, 1985), or hampering effect (Frings and Wühr, 2007a). In the other extreme, an experiment revealed negative priming after a month using nonsense shapes which are very unlikely to be seen in other circumstances (DeSchepper and Treisman, 1996). In continuous presentation of trials, the proportion of preprime RSI and current RSI influences neg- ative priming (Neill and Valdes, 1992; Mayr and Buchner, 2006) but not reliably (Hasher et al., 1996; Conway, 1999).

In the absence of distractors in the probe trial during a DT condition, negative priming vanishes or even reverses to facilitation (Lowe, 1985; Tipper and Cranston, 1985; Allport et al., 1985; Moore, 1994). A more salient prime distractor increases the magnitude of the negative priming effect in DT conditions (Grison and Strayer, 2001; Tipper, 2001). Negative priming is reduced or even reversed to facilitation when the emphasis is put on speed rather than accuracy (Neumann and Deschepper, 1992). Negative priming occasionally depends on age (Spieler and Balota, 1996; Verhaeghen and De Meersman, 1998; Gamboz et al., 2002) and sex (Bermeitinger et al., 2008). Increasing the perceptual load, e.g. by raising the number of distractors presented in a single trial, leads to less negative priming (Lavie et al., 2004). In other settings a higher number of prime distractors causes an increase of negative priming (Neumann and Deschepper, 1992; Fox, 1995). The inclusion of TT trials or single target trials in the presentation sequence enhances negative priming (Neill and Westberry, 1987; Titz et al., 2008).

A short presentation time of prime and probe stimuli attenuates negative priming (Gibbons and Rammsayer, 2004). Negative priming vanishes if the target is presented a bit earlier than the distractor in the prime trial. On the other hand, if the prime distractor is shown simultaneously with the prime target but blanked after a short time, negative priming is observed (Moore, 1994). If the prime display contains a single stimulus that is masked, subjects reporting awareness of the prime object show positive priming, while subjects not aware of the object show a negative priming effect (Wentura and Frings, 2005). In subliminally primed trials the presence of a distractor in the probe leads to negative, the absence of a probe distractor to a positive priming effect (Neill and Kahan, 1999).

It should therefore be noted that negative priming is a universal effect which is sensitive to a variety of factors. Hence, it is not surprising that, although there is a lively theoretical discussion, a consistent explanation of the entire negative priming phenomenon is still lacking. Over the years, various theories have been developed to explain negative priming through a variety of mechanisms. The next section will give a thorough description of the theoretical accounts.

2.4 Theories of Negative Priming

Because of the controversial nature of the negative priming effect, a variety of different theoretical accounts have been developed since its discovery. The negative priming phenomenon has a very complex nature. None of the theoretical accounts is able to explain all aspects of the negative priming effect found in behavioral studies which have partly been outlined in section 2.3, they all have their strong points as well as their shortcomings. All theories assume different mechanisms to be responsible for negative priming. Their realization is also very diverse, from a purely descriptive formulation to detailed computational modeling. The most influential theories comprise the distractor inhibition account (Houghton and Tipper, 1994), episodic retrieval (Neill and Valdes, 1992) and response retrieval theory (Rothermund et al., 2005).

In the present section we will address each theoretical account by explaining the idea and point to experimental evidence as well as counterevidence. As some theories are derivates of others, we will highlight the differences. Many of the proposed concepts are settled in different areas of cognitive processing such that combinations are conceivable, which actually is one of the theoretical ideas, but some exclude each other.

2.4.1 Distractor Inhibition Theory

The first theoretical account in the context of negative priming was the inhibition hypothesis by Neill (1977) and Neill et al. (1990), before Neill and Valdes (1992) began to promote the episodic retrieval theory, see section 2.4.2. Meanwhile Tipper made himself the spokesperson of the dis- tractor inhibition theory (Tipper, 1985; Tipper and Baylis, 1987; Tipper et al., 1988; Tipper and McLaren, 1990; Tipper et al., 1991; Houghton and Tipper, 1994, 1996; Tipper, 2001; Tipper et al., 2002) accompanied by some early questioning work (Tipper and Cranston, 1985).

The basic idea of distractor inhibition theory is that irrelevant stimuli representations are actively suppressed to support the selection of the relevant target stimulus. The inhibition is assumed to persist for some time. If perceptual input is no longer present, the persisting inhibition drives the distractor representation below a baseline activation. The negative priming effect directly results from the time the probe target representation activation needs to reach baseline from below. There are two complementary processes involved in the attentional selection process: a direct feedforward excitation of the representation of perceived items by the visual pathway and another one that inhibits all irrelevant information. The slowdown of the reaction in the probe trial can be seen as a direct indicator of the amount of activation in the prime display. Distractor inhibition assumes selection to operate on a semantic or postcategorial level (Houghton and Tipper, 1994). It therefore also explains findings that report negative priming in semantic priming tasks (Tipper and Driver, 1988).

From a modeler’s perspective, the most important contribution to the domain of distractor in- hibition comes from Houghton and Tipper (1994). A computational implementation of an arti- ficial neural network qualitatively explains negative priming by an inhibitory rebound naturally emerging from the network connections between excitatory and inhibitory cells homeostatically balancing the state of a property unit. The initial version of the computational distractor inhibition model is very ambitious, as perception is split into the detection of single features, hardwiredly binding them into objects. The model has a very general connection scheme, to act in a variety of situations. Unfortunately, none of the further projects proposed by Houghton and Tipper (1994) has been realized since then. In order to investigate the time course of negative priming, the role of multiple distractors and different distractor salience, the model is later simplified by looking at only one isolated property unit for the target and one for the distractor, with connections only to their on and off cell (Houghton et al., 1996; Houghton and Tipper, 1998). The aim is to simplify the original model as much as possible while still observing the same dynamics. Regrettably, the generality of the first modeling approach is no longer present. On the one hand, simplifications of complex models are an adequate tool to understand the behavior of the entire system. On the

Illustration not visible in this excerpt

Figure 2.5: Schematic view of target and distractor representation during one trial. At stimulus onset both activations rise driven by the input. The two curves diverge due to the inhi- bition the distractor receives. As inhibition builds up, it balances the perceptual input to the distractor after some time. If a certain difference between target and distractor representation is reached, the target is assumed to be selected, and an action is taken. Then the input is switched off as the stimuli disappear. The target activation passively decays to zero, whereas the distractor activation is still subject to persisting inhibi- tion, driving the distractor representation below baseline in the so-called inhibitory rebound, being responsible for the negative priming effect in the next trial. Figure adapted from (Houghton and Tipper, 1994).

other hand, the reintegration of the single units into the bigger network always brings along vari- ous nonlinear effects that are inherent to the model and can not be neglected when deriving system behavior from results of looking at isolated units. The reintegration does not take place, which might be an indicator for too high complexity of the distractor inhibition model to learn something from it.

A strong point of distractor inhibition theory comes from the study of varied distractor saliency. The negative priming effect increases with growing saliency of the distractor (Lavie and Fox, 2000; Grison and Strayer, 2001; Tipper et al., 2002). This effect can be very well explained in terms of the inhibition model, since a stronger distractor would require more inhibition, causing a stronger inhibitory rebound, and thus leading to a more prolonged reaction time.

Distractor inhibition theory can directly explain the impact of the depth of processing (Craik and Lockhart, 1972; Craik, 2002). Processing on a deep conceptual level produces a bigger negative priming effect. Distractor inhibition theory can explain the results, as deeper processed items have a stronger activation and thus need more inhibition if characterized as distractor. Therefore, more deeply processed stimuli produce larger negative priming.

The original distractor inhibition theory fails to explain the dependency of negative priming on the response stimulus interval. If the representation of a distractor object is inhibited, the impact of inhibition should be strongest immediately after the selection, because the inhibition is assumed to decay to zero with time. Although there is a general trend of negative priming to decay with increasing time between prime and probe (Neill and Valdes, 1992), no negative priming is observed in several studies when the RSI is very short or nonexistent (Lowe, 1985; Houghton et al.,

Illustration not visible in this excerpt

Figure 2.6: Alternative view of the distractor inhibition theory accounting for negative priming effects in the absence of a response stimulus interval. If the situation requires very strong inhibition, the activation of the distractor can drop below baseline already be- fore the end of the trial. Sketch adapted from a talk by Christian Frings, May 30th 2007 in Göttingen, on (Frings and Wühr, 2007a).

1996). In the original model the equilibrium between perceptual input and inhibition is tuned such that the activation of the distractor stays positive. If then a new display is shown directly after the response, a facilitatory effect in the DT condition is expected. Unfortunately, already the study that brought negative priming to light of Dalrymple-Alford and Budayr (1966) shows negative priming without any delay between succeeding stimulus pairs, subjects held cards with several colored words which they processed in order. Therefore, Wentura and Rothermund (2003), Frings and Wentura (2006) and Frings and Wühr (2007a) proposed an extension of distractor inhibition by assuming that the amount of inhibition is proportional to task difficulty. In demanding paradigms like the Stroop task, inhibition may exceed excitatory input thus pushing the distractor activation below baseline even before a reaction, see figure 2.6.

Distractor inhibition is incompatible with target only probe displays. In the absence of a dis- tractor priming constellations that usually produce negative priming effects can show facilitatory priming as reported by Moore (1994). A suitable extension of distractor inhibition theory concerns the notion of what is actually inhibited. Neill (1977) suggests that the semantic representations of the distractors themselves are inhibited, which matches with spreading inhibition through se- mantical networks (Quillian, 1966). Tipper and Cranston (1985) propose inhibition to act on the link between semantic representation and the response system. More explicitly, they assume a selection state of the response system in which the time-consuming resolving of the inhibition of the link between representation and response produces negative priming. In situations where no such selection is necessary, the response may still be facilitated because of the residual activation of the distractor representation. Unfortunately, the response inhibition account was not integrated in later papers.

Distractor inhibition theory is also challenged by the empirical finding of long-term negative priming effects (DeSchepper and Treisman, 1996; Grison et al., 2005). Tipper (2001) integrates these findings by emphasizing that different mechanisms might underlie the behaviorally similar effect in different settings. It is also stated that a retrieval of an episode (as postulated by the

Illustration not visible in this excerpt

Figure 2.7: Episodic retrieval assumes figuratively a do-not-respond tag that is attached to the prime distractor. If a probe display contains matching information, the former episode is retrieved and with it the tag. Removing this tag in order to respond to the former distractor which has become target in DT trials takes time which is equivalent to the negative priming effect. Disadvantageous for the theoretical discussion, episodic retrieval theory was often reduced to the picture of the tag, which is only introduced as a metaphor in the original work.

episodic retrieval theory, described in the next section) might also retrieve the inhibitory status of the previously ignored distractor.

2.4.2 Episodic Retrieval Theory

In recent years the majority of negative priming studies have interpreted their results according to the episodic retrieval theory. It has originally been introduced by Neill and Valdes (1992). The theory builds on the instance theory of automatization by Logan (1988). If identical tasks have to be fulfilled over and over again, memories of past trials are more and more used to completing the current trial. Negative priming is assumed to be the result of conflicting information caused by automatic retrieval of the prime episode during probe processing. It is argued that the retrieval is triggered by the similarity of prime and probe episodes. Because the object information from the retrieved episode in a DT trial is inconsistent with the current role of the object as a target, retrieved and perceived information are in conflict. Resolving the conflict is time consuming and results in the slowdown of the reaction time. Some of the negative priming phenomena listed in section 2.3 can be more easily explained by episodic-retrieval than inhibition mechanisms, such as effects of prime-probe temporal discriminability (Neill and Valdes, 1992), prime-probe similarity (Fox, 1998) and long-term negative priming from single-trial presentations (DeSchepper and Treisman, 1996).

According to later extensions by Neill (1997), the main determinants of the strength of retrieval are the recency of the memory trace and the strength of the memory representation of the trial. It is assumed that more recent memory traces are more likely to be retrieved than older ones. This assumption allows for the interpretation of experimental settings with many repetitions of highly similar episodes. Recency as a relevant factor receives empirical support from studies that show a negative correlation between response stimulus interval and negative priming effect (Neill and Valdes, 1992). Neill et al. (1992) reports an influence of the interval preceding prime onset on the negative priming effect, which challenges the inhibition based accounts, but is easily explained in

Illustration not visible in this excerpt

Figure 2.8: Response retrieval assumes that just the response given in the prime trial is retrieved triggered by perceptual similarities of the two displays. If the appropriate response is repeated, a positive priming effect is expected, whereas a response switch results in a negative priming effect. Note that in the voicekey paradigm shown in the current pic- ture, TT trials are always associated with a response repetition, and DT trials always require a response switch. Only more complex paradigms allow for a disentanglement of priming condition and response relation.

terms of episodic retrieval. When the pre-prime response stimulus interval is larger than the one of the current probe, the prime-probe pair is more easily separable from the sequence of stimuli. Hence retrieval of the prime episode in the probe is enhanced. The memory trace produced by the prime episode is more elaborated with deeper processing of the stimuli. Therefore, the depth of processing of the stimuli can influence the strength of retrieval of an episode positively, the data by Yee et al. (2000) can thus be explained by episodic retrieval.

Strong support for episodic retrieval comes from a study by Stolz and Neely (2001). They found an increased negative priming effect when the contextual similarity between prime and probe situations is increased. Prime/probe episodes were more similar in terms of visual characteristics and the required response. On the contrary, a facilitated response at very short response stimulus intervals (Lowe, 1985) is also difficult to explain in terms of the episodic retrieval framework. Another weakness of episodic retrieval is semantic negative priming. The absence of perceptual similarity should eliminate retrieval, and therefore no effects are to be expected.

2.4.3 Response Retrieval Theory

Marczinski et al. (2003) investigate different priming effects for young and old subjects in a two- alternative forced-choice task, which can not be explained by episodic retrieval theory. Therefore she introduces the notions of specific and unspecific encoding or retrieval respectively. The reac- tion time difference between trials with a response switch and trials with repeated responses are called response repetition effect. The idea was borrowed by Rothermund et al. (2005), who points to the inherent entanglement of priming condition and response relation in most negative priming paradigms. Usually DT trials are accompanied by a response switch, whereas TT trials require the same response. The response retrieval approach postulates that every reaction time difference in priming paradigms is due to the retrieval of a past response depending on perceptual similarity between the two displays.

A letter-matching task initially developed by Neill et al. (1990) is adapted in order to test the hypothesis (Rothermund et al., 2005). Strings of five letters are presented to the subjects and they have to answer yes or no by an appropriate button press depending on whether the second and the fourth letter are identical. The remaining three letters are identical and function as distractors. The advantage of the paradigm is that the repetition or non-repetition of the response can be varied independently from priming condition. It is possible to require both a yes or a no answer in every priming condition because the answer does not depend on the identity of the target, but on its relation to the second target. In their series of experiments Rothermund et al. (2005) could provide solid justification of response retrieval.

The letter-matching paradigm is problematic in the view of episodic retrieval theory. The sim- ilarity of the prime and probe display is dependent on whether both targets match or not. In fact the highest similarity is achieved in a TT condition which requires to answer yes in both prime and probe trial. If the answer no is correct, only one of the targets is repeated, possibly shortening reaction times of yes answers. However, two other experiments described by Rothermund et al. (2005) implemented task-switching paradigms where the described problem does not occur.

2.4.4 Feature Mismatch Theory

In a series of four experiments Park and Kanwisher (1994) developed their feature mismatch the- ory. If, at a certain location, an object changes identity, a mismatch of features is detected which causes a slowdown of processing (Milliken et al., 1994). They presented two letters, a particular one coding for the target, a specific other one for the distractor, on two out of four possible loca- tions on a screen. Subjects indicated via button press the target location. In order to discriminate between distractor inhibition and feature mismatch, they varied the feature that identifies the target between prime and probe. In the prime trial, the letter x had to be responded to, and o had to be ignored. In the probe trial, subjects should indicate the position of the o ignoring the x. According to the distractor inhibition account, TT trials should lead to a positive effect, as the probe target location is already positively primed. But it turned out to produce a longer reaction time on TT tri- als, and a faster response in DT trials, contradicting distractor inhibition. Unfortunately the aspect of task switching effects (see chapter 6 for a discussion of those) was not addressed in the paper.

Feature mismatch theory received quite some attention, and as it is often listed among explanations for negative priming. Nonetheless, there has been no further development of the theory nor experiments in strong support of feature mismatch.

2.4.5 Temporal Discrimination Theory

Milliken et al. (1998) added temporal discrimination theory to the set of explanations for negative priming. Temporal discrimination assumes a classification of stimuli as old, where a response can be retrieved from memory, or new, where a response has to be generated from scratch. The classification takes time depending on the similarity between the current stimulus and a memory trace. The dependency is not monotonic: the classification as new is fast when prime and probe stimulus are very dissimilar. The classification as old is fast when the displays are identical. But intermediate similarities such as in DT trials where the prime distractor is repeated but not in its former role but as target, the decision whether the display is old or new takes more time. Experimental replications of the effect on which the initial paper by Milliken et al. (1998) is based, i.e. negative priming without selection in the prime trial, do not discuss the theory itself (Neill and Kahan, 1999; Healy and Burt, 2003).

Even if the temporal discrimination account is often cited, it is mostly cut down to the descrip- tion above. It is rarely theoretically addressed. Only Frings and Wentura (2005) and Frings and Wühr (2007b) develop the model further. They address the question whether the awareness of the prime distractor, which was masked in most of the experiments in favor of temporal discrimination, plays a role for negative priming or positive priming to occur in single prime distractor DT trials. If the masked prime distractor was supraliminal, a positive effect is expected as participants were not instructed to ignore items eventually contained as a flicker in the mask.

As both temporal discrimination and episodic retrieval theory rely on the question of if the response can be retrieved from memory or if it has to be computed directly, it is hard to delineate them from one another. The difference usually pointed out in the literature is the presence or absence of a do not respond-tag, which was introduced metaphorically in episodic retrieval theory, see section 2.4.2, but then falsely made a central part by commentators.

On a closer look, temporal discrimination tacitly involves two different processes. The direct computation of a response is completely different from a retrieval of the answer from memory. No clue exists that these processes take an equal amount of time. Nevertheless, most discussions of temporal discrimination seem to have been misdirected by an illustration Milliken et al. (1998) draws to explain his theory. He equates an experimental trial with the time a PhD student takes to finish his thesis depending on whether he found a project suitable to his interest. If he chose a not matching subject, the degree of match with his interest will determine how long it takes him to realize and switch the project. No match causes an early switch, and he barely loses time in comparison to a perfect match with his interests. But if there is some overlap, he will proceed for some time with the project, until he finally realizes he has chosen the wrong project, then change and again take a full period of PhD research. The analogy raises the following problems: the student always has to write a thesis. It is only one and the same process involved, but response retrieval or direct response computation are very different; and secondly, the old-new classification is always performed, whereas the student would have no switching time if the project is suitable. The acknowledgment of two independent processes renders most of the criticism on temporal discrimination less striking, as most critical arguments rely on the equality of processing time after the old/new-classification.

The weakest point of temporal discrimination theory is the assumption of a serial processing of classification and retrieval or direct generation of a response. Most processes in the brain work simultaneously, and therefore a parallel computation of the old/new signal together with a directly computed answer and the retrieval of past episodes is rather probable. Dropping the assumption of seriality would certainly bury the temporal discrimination account which mostly appeals due to its simplicity.

2.4.6 Dual Mechanism Theory

Obviously there is evidence in support of each of the theoretical accounts and no approach is clearly favored over the others. As distractor inhibition and episodic retrieval are by far domi- nant in the domain of explaining the negative priming effect, integrating accounts are called dual mechanism theories.

The original dual-mechanism account of negative priming by May et al. (1995) proposes that inhibition as well as memory retrieval can be the source of negative priming and the experimental context specifies which of the two mechanisms is expected to operate. Another attempt to integrate inhibition and retrieval perspectives was made by Tipper (2001). He argued that it is important to note that distractor inhibition and episodic retrieval theories are not mutually exclusive, and both inhibitory and retrieval processes could be involved in the emergence of negative priming. Although retrieval processes can be responsible for producing negative priming effects, inhibitory processes are still required in selecting information for goal directed behavior. Supposedly, in most tasks negative priming will be caused by a mixture of contributions from persisting inhibition

Illustration not visible in this excerpt

Figure 2.9: Time series of the activation variables during one simulated trial processed by the computational implementation of the ISAM, see chapter 3. Only the representations of objects of the current probe trial or the preceding prime trial have nonzero acti- vation. The threshold level (blue) is driven by the total activation. If only one su- perthreshold activation is left, a decision is made. The DT condition is shown on the left. The forced decay of activity of the former distractor variable is visible in a subtle kink of the red solid curve. Right is shown the TT condition. The activation of the former target green solid line has not yet decayed to zero when the variable is again subject to input, shortening the reaction time significantly.

and interference from retrieval. Because these processes may sometimes oppose each other, it is difficult to distinguish them by means of behavioral measures like reaction times and error rates (Gibbons, 2006). However, depending on contextual conditions and other experimental factors, the contributions of inhibitory and retrieval processes might vary considerably (Kane et al., 1997; Tipper, 2001). Gamboz et al. (2002) revealed in a meta analysis of age-related negative priming experiments that there is no significant evidence for a paradigm to produce patterns of results favoring either inhibition or retrieval theories.

These integrative concepts seem to provide the better framework to explain the various facets of negative priming, as they raise the level of complexity and the number of possibly contributing parameters. However, a more explicit formulation than the one given by the mentioned accounts would be desirable in many cases. For example, the determination whether encoding or retrieval processes play the main role or to what extent they contribute to the effect is widely left to inter- pretation and thus largely depends on the paradigm. A comprehensive model that allows for exact, ideally quantitative predictions for a wide variety of paradigms, tasks and other experimental pa- rameters would be desirable.

2.4.7 Global Threshold Theory

In the imago-semantic action model (ISAM), Kabisch (2003) developed the hypothesis of a thresh- old variable deciding which items from perceptual input can be responded to. The threshold is assumed to adapt according to the current average activation of representations of objects. Addi- tionally, the ISAM proposes a forced decay of activation if residual activity is partly overwritten by perceptual input of a new stimulus. These two mechanisms together can account for positive as well as negative priming. The ISAM differs from the distractor inhibition theory explained

Illustration not visible in this excerpt

Figure 2.10: Schematic view of the ISAM. The right blue box represents the preliminary rating of relevance for the perceived objects. The direct relevance of certain objects drives the factor of situational acuteness that controls speed and accuracy of the compu- tation. The level of global activity affects a threshold which truncates the list of perceived objects into a space of possible actions. A decision is achieved if in the set of applicable actions only a single one is left. The blue box on the left contains a semantic analysis of the perceived objects. It is able to project back to the posterior rating of relevance in a so-called semantic feedback loop. The interaction among the components leads to a decision about a reaction by a dynamical process.

section 2.4.1, by postulating only facilitative input and passive decay in the absence of input.

The assumed processing is sketched in figure 2.9 which actually shows data from our computa- tional simulations of the ISAM described in chapter 3. Perceptual input drives the corresponding representation variables to a high value. The target receives additional support, which drives target and distractor activation apart. When the threshold surpasses the second largest activation, the one of the distractor, the system has selected the target and is able to respond. In DT trials the decay of the former distractor is faster than for an unrepeated object. Therefore the overall activation is lower, resulting in a slower rise of the threshold, which then delays the reaction. In TT trials, target activation has not yet reached zero. Thus more activation is present, which speeds up threshold and subsequently the reaction.

The ISAM forms a comprehensive concept of action selection. The presented objects are assumed to undergo a pre-attentive processing and perception stage, resulting in an abstract cognitive representation of the objects. Formally, the decision between target and distractor is determined by the task instruction, which is made accessible to the model via the semantic feedback loop (left in figure 2.10). In contrast to the abstracted early visual processes, the decision is guided by attention and a conscious application of the task instruction.

The stimuli are assumed to be initially processed automatically according to a relevance rating based on low-level features such as motion or color. The stimuli are sorted hierarchically by their (automatically assigned) relevancy. The relation between stimulus and associated action incentive is that strong that we will speak of both synonymously. The subject has to decide which one of the current action incentives to follow.

Attention is modeled as trimming the perceived stimulus set by an adaptive threshold which is driven by the overall activation. If more than one option for actions exists, the threshold adapts such that only one option remains. During the adaptation of the threshold, representation variables can still be subject to input, be it top-down or bottom-up.

According to the dual-code hypothesis of Krause et al. (1997), assigning modified relevancy values to the respective objects happens in a semantic space where stimuli are processed jointly with the implied actions. The relative relevance of stimuli can be affected in a posterior rating in the semantic space. The activation corresponding to a target is amplified such that even if low-level perceptual features result in a higher saliency of the distractor, the target representation becomes significantly stronger than that of any distractor.

Kabisch (2003) even found a reversal of priming effects compared to the expected ordering shown in section 2.1 for certain response stimulus intervals. He argued that activation is subject to diffusion within its layer on a faster timescale than the one of the passive decay, which means that global activation is more persistent than specific activation of a certain object variable.

The dependence of negative priming effects on distractor saliency can easily be explained by the ISAM, as we will point out in section 3.3.3. If the distractor becomes more salient, the target and distractor activations split later in the trial processing, delaying a decision substantially.

2.5 Summary

Mechanisms of selective attention are made accessible in the framework of negative priming, a slowed reaction to previously ignored stimuli. In standard negative priming paradigms, an experi- mental trial consists of a target which requires a response and a distractor object that is irrelevant to the task. Experimental conditions are classified by the stimulus relations between two subse- quent trials, called prime and probe, respectively. Whenever a stimulus constellation occurs where a prime distractor becomes target in the probe, the DT condition, a negative priming effect is expected.

The variety of paradigms that showed a negative priming effect revealed a robust nature of the effect. But the overall pattern of results also points to a strong dependency on various parameters, often in an inconsistent manner. Therefore a large body of theoretical accounts have evolved, each based on a certain experimental setting. Due to the contradictory effects, a comprehensive theory is improbable.

3 Imago Semantic Action Model

The first mainstay in our quest to reveal the time course of negative priming is to implement a computational model that describes negative priming by the means of a dynamic system. The evolution of the model variables provides detailed temporal information. Characteristic differences between trials of different experimental conditions can be localized in time and thus give insight in the stage of processing where negative priming is produced.

We will present the computational implementation of global threshold theory, see section 2.4.7. The original work on the imago semantic action model by Kabisch (2003) describes the interaction of concrete representation activations which can be straightforwardly translated into model variables. We only had to specify the actual dynamics. Similar concretizations are necessary whenever a quantification is sought and only a qualitative description is available.

Our simulations nicely reproduced experimentally observed reaction times. Without further fitting several dependencies of negative priming are matched by our model. We thus show that the adaptive threshold mechanism proposed by global threshold theory is sufficient to explain both positive and negative priming effects. Additionally, the implementation provides testable predictions with respect to hitherto untested stimulus combinations, e.g. single object trials, see section 3.3.4.

We will start the following chapter with considerations on the transient behavior of an artificial neural network to the onset and offset of input, section 3.1. The resulting exponential fixpoint dynamics is then used to implement the ISAM. Implementation and resulting dynamics are de- scribed in section 3.2. Section 3.3 shows how the explanatory power of the ISAM in several as- pects, i.e. artificial reaction times, the sensitivity of negative priming to the length of the response stimulus interval, varying distractor saliency and the presentation of single target trials. The results are discussed in section 3.4. The present chapter is based on (Schrobsdorff et al., 2007b).

3.1 Deriving Simple Activation Dynamics

The simulation of the ISAM, as it will be discussed in the following, acts on a rather abstract level. It contains a single variable for the semantic representation of an object triggered by visual input. For the abstraction we make several assumptions. First, we consider representation of a concept as an increase of firing rate for a specific, densely coupled assembly embedded in a larger network. For simplicity we model the sensory presence of an object or a certain feature as a constant input to the corresponding cluster. The of input either present or absent. The variables of the ISAM subsume the firing rate of such an assembly as it is driven by input. In order to determine a realistic dynamics of this activation of the corresponding concept, we consider an isolated cluster as an all-to-all coupled network of integrate and fire neurons. We then average the firing rate of the network over many presentations and analyze the shape of rise and decay.

3.1.1 Networks of Integrate-and-Fire Neurons

The membrane potential hi of neuron i = 1 . . . N is driven by external input Ii(t) and recurrent connections which deliver spikes of adjacent neurons modulated by a synaptic strength wi,j.

Illustration not visible in this excerpt

If hi reaches the firing threshold θ = 1, it delivers a spike to its postsynaptic neurons and is reset

by the threshold value [Illustration not visible in this excerpt]

The external input Ii(t) is drawn independently in each time step from a Gaussian distribution, with a mean chosen such that a single neuron receives on average the same input equal to the difference of threshold θ and resting potential [Illustration not visible in this excerpt]. On average and without the recurrent coupling a neuron would fire once during presentation of the stimulus. The firing rate of the network was determined by summing up all spikes occurring in one timestep.

We simulated a network of N = 1000 neurons. A stimulus was shown for 50 timesteps, and the interstimulus interval was also 50 timesteps long. The total output of a neuron was fixed to α = 0.87. The stochasticity of the input and the sensitivity of the network for fluctuations result in a rather random single trial firing. But on average a coherent behavior becomes visible. For the results shown in figure 3.1 we averaged 10.000 trials in order to obtain a good estimation of the firing rate over time.

3.1.2 Network Response to Input Onset and Offset

Illustration not visible in this excerpt

Figure 3.1: Average firing rate of the network during input (gray shaded region) and no input. The fraction of two subsequent values is shown in red. Black lines, the averages of the respective fractions, are indicating that they remain rather constant over time.

In order to derive a computationally simple dynamics for the representation variables of the ISAM, we tried to fit the time course of rise and decay of the firing rate of the network. A good candidate seems to be an exponential fixpoint dynamics, i.e. a variable approaches its current input value by a fixed fraction of the distance in every timestep. This fraction is called time constant of the variable.

In figure 3.1 we show the averaged firing rate and plot the percentage of change from one timestep to the succeeding one in reference to the actual fixpoint, i.e. maximum firing rate or zero. The observed time constants are only marginally constant, but sufficient for a justification of the simplified dynamics we will use for the implementation of the ISAM.

Illustration not visible in this excerpt

Figure 3.2: Distribution of membrane potentials averaged over 10.000 trials. Note that the poten- tials are mostly equally distributed, as the colormap only covers values from 0.0098 to 0.0115. Nevertheless, the fine grained plot reveals the processes generating the firing rates analyzed in figure 3.1: Initially all neurons are pushed towards higher membrane potential by the input, leaving a relative gap that is propagated upwards. Then, bands of neurons with a certain membrane potential form as the recurrent in- put builds up. Finally, the system relaxes and the less regular spikes rebuild a more equally distributed picture until no further spikes are generated.

Besides the result of a dynamics to use in our implementation of the ISAM, the periodicity of the time constant, even after severe averaging, points to interesting behavior of the system. Therefore, as a short excursion, we will spend some time on it. Figure 3.2 shows the distribution of membrane potentials averaged over 10.000 trials as in figure 3.1. During input, all neurons are shifted in their membrane potential such that small potentials become very improbable, to the benefit of superthreshold potentials. Most potentials within the band that is shifted upwards during stimulus presentation have a probability of 0.0098 and 0.0115, which is near an equal distribution. But there is some inner structure that survives the averaging process, which is revealed when having a closer look. In the beginning, all units receive only external input. They are shifted upwards, leaving a relative gap which then propagates through the entire range of potentials. Neurons that spiked are not reset to zero but reset by 1 and still receive recurrent as well as external input, which results in virtually no neurons having membrane potentials between zero and 0.15. As recurrent input tends to settle at a certain value, there is a trend of jumping into the band between 0.18 and 0.28 after spiking. This band is now shifted upwards by the same amount of activation, being smeared out during travel to the threshold. In every timestep a neuron jumps from one band to the next one. After offset of input there is only remaining recurrent excitation which decays rather fast. As the number of spikes decreases smoothly, the bands are washed out. In the beginning of the decay, there are still jump bands visible, but from time step 60 there is just the small trend upwards until settlement in the absence of input.

[...]