Abstract
Episodic memory depends upon activity distributed across the brain. However, the activity underlying memory has largely been examined within single tasks in isolation. Thus it is unclear to what extent prior findings reflect task-general rather than memory-specific cognitive processes. Here we address this question using data from 371 patients recorded intracranially who performed a free recall task with encoding and retrieval phases alongside an arithmetic distractor phase. We ask whether neural decoders fit to predict behavior from one phase transfer to the others. Encoding-retrieval transfer exceeds both arithmetic-encoding and arithmetic-retrieval transfer and therefore cannot be explained solely by processes supporting arithmetic. We further detect transfer between arithmetic and retrieval but not between arithmetic and encoding. The brain-behavioral relations observed in these tasks thus do not merely reflect a single task-general factor of activity. We propose cross-task decoding as a method for identifying the neural factor structure underlying distinct cognitive processes.