A new Deepseek whitepaper has outlined a new form of long-term memory for AI models, named Engram. Engram-based models are more performant than their MoE counterparts, and decouple compute power from system RAM pools to improve results.