In this section, we will review some aspects of what is known about the PFC and DA/NA involvement in a number of separate processes that may be considered to be part of cognitive flexibility as tested in the ARE task.
In every new phase, novelty detection takes place. Then, acquisition of a conditioned response is the essential step in goal-directed action. After every new phase, the newly formed associations will be consolidated, followed by retrieval of this information in later sessions or phases. Action monitoring and error detection are on-going processes and may be followed by subsequent corrective actions like inhibition. Finally, extinction or strategy switching is needed to adapt behavior to the altered conditions. It is beyond the scope of this review to discuss all these processes in detail, but we will review some that were recently studied in relation to the functions of the PFC of the rat and may be relevant to the findings obtained using the ARE- task.
6.1 Novelty Detection
Medial PFC is involved in the reactions to novelty in rats (Holson and Walker, 1986; Dias and Honey, 2002), as it is in human (Daffner et al., 2000). This is not only apparent from lesion studies, but from the studies with novelty exposure, which activates PFC neuronal activity (Handa et al, 1993) and causes DA and NA effluxes in the PFC (Feenstra et al., 1995, 2000). The DA responses remain when the stimulus is no longer novel but still salient (e.g. a reward): in this case, the response may shift to the first event that predicts the reward (Hollerman et al., 2000). NA is more specifically involved in novelty, as habituation of the NA responses develops rapidly (Vankov et al., 1995) and as behavioral habituation and novelty seeking may depend on NA (Mason and Fibiger, 1977; Sara et al., 1995), Based on these data, one may infer that novelty in general, including task novelty (Barceló et al., 2002), results in the activation of PFC-related circuits. Our findings using the ARE-task suggest, however, that not all task alterations require PFC involvement for adapting the behavior.
6.2 Acquisition of Conditioned Responses
The PFC is generally thought to be uninvolved in the acquisition of classical, Pavlovian conditioning or in operant learning. Numerous authors
report that rats with PFC lesions were not impaired in learning a wide variety of associations or discriminations, e.g. fear conditioning (Quirk et al., 2000), discrimination learning in active avoidance (Li and Shao, 1998) or odor-reward association (Schoenbaum et al., 2002), acquisition of spatial or visual-cued version of cheeseboard or cross-arm tasks (Ragozzino et al., 1999a,b), and operant discrimination learning (de Bruin et al, 2000).
In view of a number of recent data obtained with various experimental techniques, this view is challenged. Baldwin et al. (2000) observed impaired learning of lever-pressing after local injections of a NMDA-antagonist into medial PFC, while Izaki et al. (2000) found that medial PFC lesions had a similar effect. Both groups indicated a special role for DA as well: local injections of a potentiated the effects of the NMDA- antagonist (Baldwin et al., 2002), and DA was reported to be selectively activated during learning of the lever-press response (Izaki et al., 1998). This initial (shaping) phase generally precedes all other experimental tasks and is often not included in the test for lesion effects or, at least, is not included in report. Inclusion of the initial phase may be very important in relation to PFC function, because ‘motor set’ has been suggested to be one of the main properties of the PFC and because the ‘motor rule’ is the first rule to be learned in this sequence of task phases (see above). Moreover, in the studies on task-related neuronal activity, Mulder et al. (2000) described neurons in the medial PFC that develop sustained task-related activity during the acquisition phase, indicating that a neuronal substrate may be available in the PFC. It is well-known that DA is not only activated by appetitive stimuli but also by aversive stimuli (Bertolucci-D’Angio et al., 1990; Feenstra, 2000), and Stark et al. (1999, 2000) showed that DA efflux in the medial PFC of the gerbils performing an aversively motivated shuttle- box task is selectively activated during learning, i.e. during strategy formation. These examples suggest the involvement of the PFC in making operant or 'planned' responses in which the response may be thought of as separated from the outcome, unlike food-searching tasks where the response directly leads to the outcome.
Although at the moment, no further information is available regarding the involvement of the PFC or DA in the shaping phase of the ARE-task (Fig. 4), the activation of PFC- and DA-dependent mechanisms is not only in line with the ideas put forward by Franz (1907), but also with the recent theories by Passingham (1993, 1998) that new actions are supported by the PFC but that upon practice, brain activation is shifted to other cortical and cerebellar areas (Shadmehr and Holcomb, 1997). As this is what we observe in the course of serial reversals, a similar mechanism may be expected during the shaping phase.
6.3
Consolidation
PFC involvement in the consolidation phase of association formation has been found in appetitively motivated odor discrimination. Using c-fos as a marker, Tronel and Sara (2002a) reported neuronal activation in medial PFC during consolidation. In addition, impaired consolidation was observed after local blockade of NMDA-receptors in the same area (Tronel and Sara, 2002b). Consolidation of appetitive instrumental learning was reported to take place in the core subarea of the nucleus accumbens (ventral striatum) (Hernandez et al., 2002). As other studies indicated that a corticostriatal network mediates instrumental learning, an additional prefrontal contribution to consolidation might be expected (Baldwin et al., 2000). Aversively motivated learning has been used frequently in studies of memory consolidation, and in general, consolidation of this type of memory does not appear to depend on PFC areas (Ambrogi Lorenzini et al., 1999). Memory for inhibitory avoidance learning, however, was dependent on the precentral PFC area (FR2 at level A3.7, Fig. 1) (Mello e Souza et al., 2000).
Memory consolidation is under a strong modulatory influence of emotional processes, and the effects of both catecholamines have been described in many learning paradigms (McGaugh, 2000).
Consolidation has not been studied as a separate process in the ARE-task. However, some evidence is available for a role during extinction (see below).
6.4 Extinction
Response inhibition has often been studied in extinction trials, where a predicted presentation of a reinforcer is omitted. This can occur in a classical conditioning paradigm, where the CS is not followed by the US anymore, or in an operant paradigm, where the action is not followed by the expected outcome. Inhibition assessed in extinction trials is, however, different from the inhibition in e.g. Go-No Go paradigms. In the latter case, responding or not responding has different consequences, and active control of behavior is called for, while in extinction there are no consequences, and adaptation might be expected to be more non-committal. Interestingly, however, PFC has been suggested to be involved in extinction of conditioned fear. In rats with lesions in the dorsomedial or ventromedial PFC, but not ventrolateral PFC, Morgan et al. (1993) and Morgan and LeDoux (1995, 1999) reported enhanced freezing responses when the rats were re-exposed to an explicit CS 24 h after the acquisition of the conditioned fear. It may not be the immediate expression of extinction, i.e. the acute inhibition of the behavioral reaction, through which the PFC
controls the behavior. Rather, the PFC may be involved in the consolidation of this new association. This was shown by Quirk et al. (2000), who identified the infralimbic area as the site that is necessary for the consolidation of extinction of fear conditioning. Consolidation of extinction was observed only when LTP-like changes (spontaneous or artificially induced) were observed in PFC neurons (Herry and Garcia, 2002). In addition, stimulation of the infralimbic area at 24 h after the acquisition even accelerates the extinction (Millad and Quirk, 2002). These results suggest that PFC involvement is important during consolidation of a new association and in retrieval of this information. Studies that specifically relate prefrontal catecholamine activation to extinction are sparse, as extinction is difficult to separate from retrieval. However, Morrow et al. (1999) reported that catecholamine lesions in the medial PFC impaired extinction without affecting acquisition.
6.5 Retrieval
Re-exposure to a cue that was learned to have predictive properties in a behavioral situation leads to retrieval of the previously acquired information, maintenance of an active representation of that information, and, upon reinforcement, whether it is appetitive or aversive, to reconsolidation of the association (Sara, 2000a). Presentation of such a reminder cue may facilitate behavioral reaction and performance (Sara, 2000b). Using a paradigm in which presentation of the retrieval cue is separated from the task in which the information is to be used, Gisquet- Verrier et al. (1989) showed a similar reminder effect of cue presentation. Importantly, Botreau et al. (2001) observed that the medial PFC is required for using the information. The involvement of various arousal systems in the retrieval process may be suggested, since it was improved by the activation of NA-systems (Sara and Devauges, 1989). It is not yet known whether monoamines act in the PFC to support retrieval, or whether the PFC and monoamines control retrieval mechanisms taking place somewhere else in the brain.
A specific retrieval cue was not used in the ARE task (except for the shaping phase) where up to the present no inactivation studies have been carried out. However, retrieval of previously stored task-relevant information may be an important function of the PFC, as a working memory task learned before a PFC-lesion was more affected than the same task acquired after the lesion (Broersen, 2000; see also Becker et al., 1981). It is possible that the late impairment after inactivation of the lateral PFC would fit with these findings (see above), but they obviously need more experimental support.