Past and future, human and non-human, semantic/procedural and episodic.

James R Hurford, Molly Flaherty and Giorgis Argyropoulos

Peer commentary on ``The evolution of foresight: What is mental time travel and is it unique to humans?'' by Thomas Suddendorf and Michael C. Corballis, to appear in Behavioral and Brain Sciences.

Abstract

The overlap of representations of past and future is not a completely new idea. S&C usefully discuss the problems of testing the existence of such representations. Our taxonomy of memory differs from theirs, emphasizing the late evolutionary emergence of notions of time in memory.



The target article makes a useful contribution. We offer some reservations which do not undermine its central purpose.

S&C join company with the literature from the past three decades casting doubt on a radical psychological past/future (memory/planning) distinction. Whereas memory was once thought of as exclusively retrospective, it is now recognized that `prospective memory' (a term introduced by Meacham and Singer (1977)) shares features with retrospective memory. Cook et al. (1983) also demonstrated the overlap between retrospective memory and prospective memory in rats. Tulving (2005) identifies episodic memory closely with both past and future (i.e. planned) events.

We are not so convinced that non-humans lack any retrospective episodic memory. Sue Savage-Rumbaugh (personal communication) tells of promising Kanzi a treat tomorrow and then being reminded of the promise the next day. Kanzi's specific desire may not have waned overnight, so there was not necessarily any representation of a future mental state distinct from the present, but Kanzi did recall the specific event of being promised. For retrospective cases, the question is whether, when an animal observes something happening, it stores a stripped-down `episodic' memory of the particular event, or makes some inferences about the lasting state of the world, such as 'food behind tree'. King, the gorilla tested by Schwartz et al (2004), was shown events with no lasting consequence on the world, such as a man skipping. King could show that he recalled these events, as compared with distractor suggestions of other events which he might have (but had not) seen. But King was only tested 15 minutes after observing the event. The data are sparse, and we agree with S&C that there is a large quantitative gulf between humans and non-humans for retrospective episodic memory.

Designing experiments to test the distinction concerned is challenging. We suggest a sense in which considerations of parsimony can conceivably be applied. Obviously no animal, human or otherwise, stores all the information from an observed event. If an animal remembers a specific episode, how much selection of its details happens? If, on the other hand, the animal does not store the episodic information, but only certain inferences relevant to its own typical needs, how many such inferences does it make? S&C's itemization of the components of 'www' is useful. Hurley (2006) describes humans as `inferentially promiscuous'. Does there come a point when we have to list so many different types of 'w' inferred by an animal from an event that it becomes more parsimonious to assume that the animal just stored a stripped-down 'episodic' memory of the event itself?

We agree with S&C that Mulcahy and Call (2006) come close to demonstrating planning by animals for anticipated needs. Mulcahy and Call (2006) showed that some bonobos in their experiment collected a specific tool for use on a specific food-retrieving task as much as 14 hours later. S&C's scrupulous critique of this experiment is correct, and there indeed remains the possibility that the animals just got into the habit of selecting the right tool. A very similar procedure to S&C's suggested red and green light future-planning food caching experiment has in fact been successfully completed by scrub jays (Raby et al., 2007), but S&C would judge that this is also not sufficient, due to the limited scope and inflexibility of the behavior. If these objections are valid, then radically different criteria for thinking about MTT, far from any sort of www components, are in order. We commend S&C for setting out just such criteria.

S&C's theatre metaphor is a bold departure, and provides fruitful ground for new investigation. That said, several components show evidence of insufficient development. Most particularly, it is not clear that the `broadcaster' component is integral to their framework. Does episodic memory only become episodic when it is shared with others? It seems unlikely that S&C would wish to make this claim, but by including the `broadcaster' as one of their seven components, the reader is left to think that episodic memory requires verbal sharing. This precludes possible research on episodic, or episodic-like, memory in non-humans.

Although it plays no central role in S&C's main discussion of MTT, we take issue with their taxonomy of memory and prospection systems in their figure 1. We see the picture as radically different. We would bracket procedural and semantic memory together as timeless forms of memory, evolutionarily preceding the emergence of episodic memory, in both its retrospective and prospective forms. Indeed, when S&C appear to distinguish between `semantic memory and prospection', we find this hard to interpret. Semantic memory encodes tenseless facts like `sugar be sweet'. Likewise, procedural memory encodes instructions on what action to take whenever certain circumstances arise. Both procedural and semantic memories are laid down by experience, which is necessarily temporally prior to their formation, but they are not memories about particular events in the past, still about less events in the future. So we agree with S&C's statement that ``The mental reconstruction of past events and construction of future ones may have been responsible for the concept of time itself, and the understanding of a continuity between past and future.'' This entails that the emergence of past/future episodic memory was also the emergence, for the first time in evolution, of types of mental representation incorporating notions of past and future time. This is, of course, a different matter from semantic or procedural memories encoding facts about temporal order, e.g. thunder follow lightning.

There is now increasing evidence for co-involvement of motor and sensory components in memory for objects and actions, tending to conflate the semantic/procedural distinction. Hommel et al. (2001) give an overview pointing to the conclusion that ``Perceived events (perceptions) and to-be-produced events (actions) are equally represented by integrated, task-tuned networks of feature codes -- cognitive structures we call event codes.'' (849) Hurford (2007) gives a short survey of other evidence; see also Martin et al. (1996), who found that ``naming tools selectively activated a left premotor area also activated by imagined hand movements''. Similarly, Mecklinger et al. (2002) concluded that ``visual working memory for manipulable objects is based on motor programmes associated with their use'' (1115) Gibson's (1979) idea of affordances depend crucially on perception-action coupling. The discovery of mirror neurons also shows overlaps between sensory and motor representations of actions. In philosophy, a school of thought known as `Enactive Perception' also emphasizes the involvement of action in perception; see Noë (2004)

Finally and quite speculatively, we mention a kind of mental representation which, though essentially semantic/procedural, encodes information about typical types of complex events brought to mind by humans in either a retrospective or a prospective sense. This is the idea of `scripts', such as the complex structure of things to do when going to a restaurant (Schank and Abelson, 1977). Such scripts form a framework for our retrospective and prospective representations of events. The ability to conjure up such scripts is a kind of general mental space integration capacity.

References

Bod, R. (1998). Beyond Grammar: An Experience-Based Theory of Language. Cambridge: Cambridge University Press.

Cook, R., M. Brown, and D. Riley (1983). Flexible memory processing by rats: Use of prospective and retrospective information. Journal of Experimental Psychology. Animal Behavior Processes 11, 453-469.

Gibson, J. J. (1979). The Ecological Approach to Visual Perception. London: Houghton Mifflin.

Hommel, B., J. Müsseler, J., G. Ascherslebel and W. Prinz (2001). The Theory of Event Coding (TEC): a framework for perception and action planning. Behavioral and Brain Sciences 24(5), 849-878.

Hurley, S. (2006). Making sense of animals. In S. Hurley and M. Nudds (Eds.), Rational Animals?, pp. 139-171. Oxford: Oxford University Press.

Hurford, J. R. (2007) The Origins of Meaning Oxford: Oxford University Press.

Martin, A., C. L.Wiggs, L. G. Ungerleider, and J. V. Haxby (1996). Neural correlates of category-specific knowledge. Nature 379, 649-652.

Meacham, J. A. and J. Singer (1977). Incentive effects in prospective remembering. Journal of Psychology 97, 191-197.

Mecklinger, A., C. Gruenewald, M. Besson, M.-N. Magnie, and D. Y. V. Cramon (2002). Separable neuronal circuitries for manipulable and non-manipulable objects in working memory. Cerebral Cortex 12(11), 1115-1123.

Mulcahy, N. J. and J. Call (2006). Apes save tools for future use. Science 312, 1038-1040.

Noë, A. (2004). Action in Perception. Cambridge, MA: MIT Press.

Raby, C. R., Alexis, D. M., Dickinson, A., & Clayton, N. S. (2007). Planning for the future by western scrub-jays. Nature, 445, 919-921.

Schwartz, B. L., C. A. Meissner, M. L. Hoffman, S. Evans, and L. D. Frazier (2004). Event memory and misinformation effects in a gorilla (Gorilla gorilla gorilla). Animal Cognition 7, 93-100.

Schank, R. C., and R. Abelson (1977) Scripts, Plans, Goals, and Understanding. Hillsdale, N. J.: Lawrence Erlbaum.

Tulving, E. (2005). Episodic memory and autonoesis: uniquely human? In H. S. Terrace and J. Metcalfe (Eds.), The Missing Link in Cognition: Origins of Self-Reflective Consciousness, pp. 3-56. Oxford: Oxford University Press.