AwareLiquid

everest-an/o1

我们研发的 M1 是面向端侧与流式场景的高效液体神经网络架构，核心目标是大幅降低大模型的部署与训练成本，在长序列、不规则数据等真实商用场景下实现更稳定的推理效果。 不同于传统 Transformer 依赖 KV 缓存扩容的思路，M1 基于固定大小的连续状态实现记忆流转，推理内存不随序列长度增长，所有优势均有实测数据支撑： 长序列内存效率领先 8000 倍：百万字上下文场景下，内存占用仅为同参数 Transformer 的 1/8063。普通端侧芯片即可处理无限长的音频流、传感数据流，彻底解决长序列推理内存爆炸的行业痛点，硬件部署成本可降低 90% 以上。 跨窗口持续记忆能力：滑动窗口场景下关键信息保留得分 0.56，同尺寸 Transformer 跨窗口记忆能力为 0，可稳定承接长文档、长对话、连续监测等流式场景的信息承接。 不规则数据高鲁棒性：内置原生时间常数机制，非均匀采样数据下精度仅下降 7.7%，远优于 LSTM/GRU 31%~33% 的精度衰减，适配工业监测、可穿戴设备等真实世界数据场景。 训练成本大幅降低：训练 loss 波动幅度仅为同级循环模型的 1/4，显著降低训练试错成本与算力消耗，中小团队也可高效完成架构迭代。 原生持续学习能力：基于生成式重放机制实现 “学新不忘旧”，5 任务连续学习场景下所有种子实验均稳定优于基线，支持端侧模型在线迭代，无需频繁全量重训。 针对此前上下文推理的短板，我们已通过注意力机制优化完成修复：在标准指针追踪推理任务上，准确率达到 100%，反超同参数 Transformer 的 99.32%，补齐了效果层面的核心短板，实现「效率大幅领先 + 效果持平反超」的综合优势。 目前 M1 架构已完成全量实验验证，第二代 M2 架构正在迭代，重点优化混合注意力机制与递归思考深度，未来可广泛落地于消费电子端侧 AI、工业边缘计算、流式语音交互等商用场景。 Our M1 is a high-efficiency liquid neural network architecture designed for edge and streaming scenarios. Its core objective is to significantly reduce the deployment and training costs of large models, achieving more stable inference performance in real-world commercial scenarios with long sequences and irregular data. Unlike traditional Transformers that rely on key-value cache expansion, M1 uses a fixed-size continuous state for memory transfer. Inference memory does not increase with sequence length, and all these advantages are supported by real-world testing data: Long sequence memory efficiency is 8000 times faster: In scenarios with millions of words of context, memory usage is only 1/8063 of a Transformer with the same parameters. Ordinary edge chips can handle infinitely long audio streams and sensor data streams, completely solving the industry pain point of memory explosion in long sequence inference, reducing hardware deployment costs by more than 90%. Cross-window continuous memory capability: Key information retention score is 0.56 in sliding window scenarios, while the cross-window memory capability of a Transformer of the same size is 0. It can stably handle information processing in streaming scenarios such as long documents, long dialogues, and continuous monitoring. High robustness to irregular data: The built-in native time constant mechanism results in only a 7.7% accuracy drop under non-uniform sampling data, far superior to the 31%~33% accuracy decay of LSTM/GRU, making it suitable for real-world data scenarios such as industrial monitoring and wearable devices. Significantly reduced training costs: The training loss fluctuation is only 1/4 of that of comparable recurrent models, significantly reducing training trial-and-error costs and computational consumption, allowing small and medium-sized teams to efficiently complete architecture iterations. Native continuous learning capability: Based on a generative replay mechanism, it achieves "learning new things without forgetting old ones." In continuous learning scenarios across 5 tasks, all seed experiments consistently outperform the baseline, supporting online iteration of edge models without frequent full retraining. Addressing the previous weakness in contextual reasoning, we have optimized and repaired it through an attention mechanism: On the standard pointer tracking reasoning task, the accuracy reaches 100%, surpassing the 99.32% of Transformers with the same parameters, addressing the core shortcomings in performance and achieving a comprehensive advantage of "significantly superior efficiency + comparable or even surpassing performance." The M1 architecture has completed full-scale experimental verification, and the second-generation M2 architecture is currently iterating, focusing on optimizing the hybrid attention mechanism and recursive thinking depth. In the future, it can be widely applied to commercial scenarios such as consumer electronics edge AI, industrial edge computing, and streaming voice interaction.