Development of a new artificial neural network architecture separating short-term and long-term memory

Background

Traditional recurrent neural networks compress information into a single vector state, creating a limitation where short-term computation and long-term preservation must share the same representation. Past expansion methods increased memory capacity or divided time units, but lacked the combination of fast neuron-level processing and long-term preservation seen in biology.

Method

The researchers introduced Continuous Memory Machines, a recurrent structure in matrix form where short-term and long-term memory states perform different roles. Short-term memory tracks recent neural activity, persistent long-term memory stores information for later use, and Transformer updates both memories together to enable bidirectional reading and writing.

Main results

In algorithmic, in-context learning, and iterative reasoning tasks, Continuous Memory Machines outperformed various baseline models. They showed stronger generalization capabilities than previous memory-augmented networks while maintaining the interpretable attention patterns of existing models.

Limitations & open points

The abstract does not specify concrete constraints or performance limitations for actual commercialization or industrial field applications.

Glossary

• 반복 신경망 — An artificial neural network architecture that utilizes previous memory in subsequent calculations when processing sequential data.
• 트랜스포머 — An artificial intelligence model architecture that excels at understanding context by grasping the relationships across the entire data at once.