←搜索结果

搜索“RAG”

按匹配度 · 共 289 条结果 清除搜索

星期二

  1. Prime Intellect精选

    Prime Intellect 发布 INTELLECT-2：首个全球分布式强化学习训练的 32B 参数模型

    Prime Intellect 发布 INTELLECT-2，称其为首个 32B 参数全球去中心化强化学习训练，任何人都可无许可贡献异构算力。该训练以 QwQ-32B 为基座模型，沿用 DeepSeek-R1 的 GRPO 与可验证奖励方法，并配套开源 prime-RL 异步分布式 RL 库、Shardcast 权重分发库和 TOPLOC 可验证推理方案。推理节点可用消费级 GPU，例如 4×RTX 3090 即可参与训练，训练数据已发布在 Hugging Face。

    INTELLECT-2: The First Globally Distributed Reinforcement Learning Training of a 32B Parameter Model

    xt round of data collection This infrastructure enables a decentralized training setup with the following unprecedented features: Completely removes communication overhead: By leveraging asynchronous reinforcement learning, the broadcast of new policy models is fully overlapped with ongoing inference and training—eliminating communication as a bottleneck; Supports heterogeneous inference nodes: Anyone can generate reasoning traces at their own pace—there’s no requirement for uniform processing s

    匹配字段正文

    #company#模型发布#数据/训练#开源生态

星期五

  1. Mistral AI精选

    Mistral OCR：面向复杂文档理解的多语言多模态 OCR

    Mistral AI 的 Mistral OCR 可解析图像和 PDF 中的文本、图片、表格及公式，但该模型现已停止维护，并由更强的新款 OCR 模型取代。其 API 曾以 mistral-ocr-latest 提供，价格为每美元 1000 页，单节点处理速度最高可达每分钟 2000 页，并支持结构化输出及选择性自托管。

    3 Exemple

    ognition. It takes images and PDFs as input and extracts content in an ordered interleaved text and images. As a result, Mistral OCR is an ideal model to use in combination with a RAG system taking multimodal documents (such as slides or complex PDFs) as input. We have made Mistral OCR as the default model for document understanding across millions of users on Le Chat, and are releasing the API mistral-ocr-latest at 1000 pages / $ (and approximately double the pages per dollar with batch inferen

    匹配字段正文

    #lab#模型发布#多模态#RAG

星期三

  1. Cartesia 产品与研究精选

    Cartesia 发布 Llamba 系列模型，用架构蒸馏将 Llama 转换为高效 Mamba-2 变体

    Cartesia 发布 Llamba-1B、3B、8B 模型，采用 MOHAWK 架构蒸馏方法将 Llama-3.X 转换为 Mamba-2 架构，训练数据量比从头预训练少 1000 倍。Llamba-8B 在 NVIDIA H100 上生成 8192 tokens 时吞吐量比 Llama-3.1-8B 高 12 倍，并已针对 Apple Silicon 和 MLX 框架优化。

    Llamba: scaling distilled recurrent models for efficient language processing

    mba-2 backbone using multiple stages. This multi-stage process allows us to preserve the core competencies of the teacher model while converting its architecture into one that leverages the efficiency of Mamba-2 layers. We make a few modifications to the original architecture, and apply this multi-stage distillation recipe. Alternating MLP Blocks: By interleaving Llama’s gated MLP components with Mamba-2 mixing layers, we not only maintain performance but also reduce the number of temporal mixin

    匹配字段正文

    #company#模型发布#开源/仓库#端侧

星期一

  1. Prime Intellect

    Prime Intellect 等发布 7B 宏基因组基础模型 METAGENE-1

    USC、Prime Intellect 与 Nucleic Acid Observatory 合作发布 7B 参数自回归 Transformer 宏基因组基础模型 METAGENE-1，在超过 1.5 万亿碱基对的污水宏基因组 DNA 与 RNA 序列上预训练，用于疫情监测、病原体检测和新兴健康威胁的早期发现。模型采用针对宏基因组序列的 BPE 分词，在病原体检测和宏基因组嵌入基准上取得 SOTA，并演示了病原体与异常检测场景。团队称其 512 token 上下文长度限制了复杂序列设计任务，当前版本风险较低，但未来更大模型开源前需严格安全评估。

    METAGENE-1: Metagenomic Foundation Model

    ic types. We show the metagenomic data collection and sequencing pipeline in the figure above. The process begins with the collection of wastewater (left), which contains genomic fragments from a diverse collection of species (center). This is processed via high-throughput metagenomic sequencing to produce millions of paired-end reads (right), each consisting of hundreds of base pairs. The complete dataset comprises over 1.5 trillion base pairs of metagenomic sequences used for model pretraining

    匹配字段正文

    #company#模型发布#数据/训练#开源生态

星期四

  1. Answer.AI 官方研发博客

    Answer.AI 与 LightOn 发布 ModernBERT，为 BERT 提供替代方案

    Answer.AI 与 LightOn 发布 ModernBERT，一个编码器专用模型系列，提供 base（149M 参数）和 large（395M 参数）两个版本，可作为 BERT 类模型的直接替换。它把上下文长度从多数编码器的 512 提升到 8192 token，并首次在编码器训练数据中加入大量代码，在检索、自然语言理解和代码检索任务上整体优于主流模型，速度最高可达 DeBERTa 的 4 倍。模型将随 transformers v4.48.0 发布，在此之前需从 main 分支安装，官方建议搭配 Flash Attention 2 使用。

    odel fine-tuned for retrieval. That’s because its encoder-only architecture makes it ideal for the kinds of real-world problems that come up every day, like retrieval (such as for RAG), classification (such as content moderation), and entity extraction (such as for privacy and regulatory compliance). Finally, 6 years later, we have a replacement! Today, we at Answer.AI and LightOn (and friends!) are releasing ModernBERT. ModernBERT is a new model series that is a Pareto improvement over BERT and

    匹配字段正文

    #company#模型发布#开源生态#数据/训练

星期一

  1. Zyphra Research

    Zyphra 发布开源小模型 Zamba2-7B

    Zyphra 发布 7B 规模小语言模型 Zamba2-7B，采用 Mamba2 与两个共享注意力块交替的混合架构，并以 Apache 2.0 开源权重。官方称其在评测基准上超过 Mistral-7B、Gemma-7B 和 Llama3-8B，首 token 时间快 25%、每秒 token 数提升 20%，内存占用也明显降低。模型基于 3 万亿 token 预训练数据，并在 128 张 H100 上训练约 50 天。

    s still reachable and surpassable with a small team and moderate budget. Zamba2-7B is released under an open source license, allowing researchers, developers, and companies to leverage its capabilities. We invite the broader AI community to explore Zamba's unique architecture and continue pushing the boundaries of efficient foundation models. Zyphra's team is committed to democratizing advanced AI systems, exploring novel architectures on the frontier of performance, and advancing the scientific

    匹配字段正文

    #lab#模型发布#开源生态#推理

星期二

  1. Zyphra Research

    Zyphra 发布 1.2B 小模型 Zamba2-mini，权重以 Apache 2.0 开源

    Zyphra 发布 1.2B 小语言模型 Zamba2-mini，称其达到 Llama2 7B 级别的性能，4bit 量化下内存占用不足 700MB，并以 Apache 2.0 协议开源模型权重。该模型采用 Mamba2 与共享注意力层交替的混合架构，相比 Phi1.5-1.3B 首 token 时间快 1.67 倍、内存开销降低 23.3%。模型在约 3T token 的 Zyda 及公开数据集上预训练，并在 100B 高质量 token 上退火，已提供 Hugging Face 集成与纯 PyTorch 实现。

    re quality), then annealed on 100B of the highest-quality tokens. Zamba2-1.2B will be released under an open source license, allowing researchers, developers, and companies to leverage its capabilities. We invite the broader AI community to explore Zamba's unique architecture and continue pushing the boundaries of efficient foundation models. A Huggingface integration is available here, and a pure-pytorch implementation is available here. Zyphra's team is committed to democratizing advanced AI s

    匹配字段正文

    #lab#模型发布#开源生态#端侧

星期日

  1. Zyphra Research

    Zyphra 发布 2.7B 端侧模型 Zamba2-small

    Zyphra 发布 Zamba2-small，一个面向端侧应用的 2.7B 小语言模型，权重以 Apache 2.0 开源。该模型将 Zamba-7B 的 Mamba1 块替换为 Mamba2 块，并采用两个 ABAB 交错共享注意力块，对每个共享 MLP 块应用 LoRA projector。相比 Phi3-3.8B，其首 token 时间快 2 倍、内存开销降低 27%、生成延迟低 1.29 倍，并保持 3-4B 稠密 Transformer 的质量而只需 1-2B 的推理算力与内存。

    re quality), then annealed on 100B of the highest-quality tokens. Zamba2-2.7B will be released under an open source license, allowing researchers, developers, and companies to leverage its capabilities. We invite the broader AI community to explore Zamba's unique architecture and continue pushing the boundaries of efficient foundation models. A Huggingface integration is available, and a pure-pytorch implementation is available. Zyphra's team is committed to democratizing advanced AI systems, ex

    匹配字段正文

    #lab#模型发布#端侧#开源生态

星期二

  1. Zyphra Research精选

    Zyphra 发布 7B 基础模型 Zamba，开源全部训练检查点

    Zyphra 发布 7B 参数基础模型 Zamba，采用 Mamba 块与每 6 层一个全局共享注意力层结合的混合架构，训练和推理比标准 Transformer 更省算力。模型分两阶段训练，先用约 1 万亿 token 开放网页数据，再用约 500 亿 token 高质量数据退火，在多项基准上超过 LLaMA-2 7B 和 OLMo-7B，且训练数据用量不到后者一半。全部检查点以 Apache 2.0 协议开源，由 7 人团队在 128 张 H100 上耗时 30 天完成，Hugging Face 集成和完整技术报告将随后发布。

    Quality vs. Inference Speed

    g paper to ensure that Zamba is a fully reproducible research artifact. Zamba will be released under an open source license, allowing researchers, developers, and companies to leverage its capabilities. We invite the broader AI community to explore Zamba's unique architecture and continue pushing the boundaries of efficient foundation models. A Huggingface integration and a full technical report will be released shortly. Zyphra's team is committed to democratizing advanced AI systems, exploring

    匹配字段正文

    #lab#模型发布#开源生态#数据/训练