←Search results

Search “RAG”

By relevance · Total 3407 results Clear search

Wednesday

  2. Latent SpaceSelected

    DeepSeek open-sources the DeepInfer inference framework, processing 155 trillion tokens per day in a single cluster

    DeepSeek has open-sourced DeepInfer, its self-developed inference framework that supports DeepSeek-V4.1-Flash, and published production data: a single cluster processes 155 trillion tokens per day, 96% input cache hit rate, 400ms first-token latency, and 230 tokens/s generation speed. The framework is customized based on vLLM and supports distributed inference, speculative decoding, SWA Bounded Replay, and structured output, with target platforms NVIDIA SM100 and Huawei Ascend 950. DeepSeek also states that components such as KV cache management and load balancing are not yet fully open-sourced, while the vLLM team says that under a matching GB200 configuration, decoding throughput is within 6% of the results published by DeepInfer.

    DeepSeek has open-sourced DeepInfer, its self-developed inference framework that supports DeepSeek-V4.1-Flash, and published production data: a single cluster processes 155 trillion tokens per day, 96% input cache hit rate, 400ms first-token latency, and 230 tokens/s generation speed. The framework is customized based on vLLM and supports distributed inference, speculative decoding, SWA Bounded Replay, and structured output, with target platforms NVIDIA SM100 and Huawei Ascend 950. DeepSeek also

    Matched fieldsBody

    #newsletter#English text is being prepared.#Reasoning#English text is being prepared.
  3. Latent Space

    DeepSeek open-sources the DeepInfer inference framework, processing 155 trillion tokens per day on a single cluster

    DeepSeek has open-sourced DeepInfer, the production inference framework supporting V4.1 Flash, and published production data including 155 trillion tokens processed per day on a single cluster, a 96% input cache hit rate, an average 400ms time to first token, and 230 tokens/s generation. The framework is customized based on vLLM and uses the Apache 2.0 license, supports distributed inference, speculative decoding, SWA Bounded Replay and structured output, and targets NVIDIA SM100 and Huawei Ascend 950. DeepSeek says components such as KV cache management and load balancing are not fully open-sourced because they are coupled with internal infrastructure, while the vLLM team reports that under matched GB200 configurations, decoding throughput differs from DeepInfer's published results by less than 6%.

    ion inference framework supporting V4.1 Flash, and published production data including 155 trillion tokens processed per day on a single cluster, a 96% input cache hit rate, an average 400ms time to first token, and 230 tokens/s generation. The framework is customized based on vLLM and uses the Apache 2.0 license, supports distributed inference, speculative decoding, SWA Bounded Replay and structured output, and targets NVIDIA SM100 and Huawei Ascend 950. DeepSeek says components such as KV cach

    Matched fieldsBody

    #newsletter#English text is being prepared.#Reasoning#English text is being prepared.
  18. AWS Machine Learning

    How Octave built a secure multi-tenant AI foundation on Amazon Bedrock

    Octave built a three-tier multi-tenant AI foundation on Amazon Bedrock for the quality management platform Reliance, cutting deployment time per use case by 50%. The foundation uses Cohere Embed v4 to embed 2 billion documents, executes Anthropic Claude models through AgentCore Gateway and isolated microVMs, and Guardrails filters content on both the input and output sides. Reliance serves more than 650 enterprise customers across 180 countries.

    High-level architecture of Octave’s AI foundation

    Matched fieldsBody

    #lab#Tutorials / practice#RAG#English text is being prepared.

Tuesday

  1. QbitAI

    Memory Tensor Open-Sources MemT 1.0 Series Memory-Native Models

    Memory Tensor has open-sourced and released the MemT 1.0 series of memory-native models, including MemT-1.0-Nano with 36B parameters and 3B activated (based on Qwen3.6-35B-A3B) and MemT-1.0-Flash with 288B parameters and 17B activated (based on DeepSeek-V4-Flash), adding a memory computation pathway parallel to attention in selected Transformer modules. The team built a 34-billion-token memory training corpus, of which 20 billion tokens were used for continued pre-training and 14 billion for post-training. In a comprehensive evaluation across 11 memory benchmarks, Flash and Nano achieved average scores of 50.6 and 47.0, respectively, ranking in the top two among the parametric memory methods evaluated; on 500 SWE-bench Verified tasks, the proportion of tasks that Nano succeeded on all three attempts rose from 54.00% for the no-experience baseline to 62.80%.

    t a 34-billion-token memory training corpus, of which 20 billion tokens were used for continued pre-training and 14 billion for post-training. In a comprehensive evaluation across 11 memory benchmarks, Flash and Nano achieved average scores of 50.6 and 47.0, respectively, ranking in the top two among the parametric memory methods evaluated; on 500 SWE-bench Verified tasks, the proportion of tasks that Nano succeeded on all three attempts rose from 54.00% for the no-experience baseline to 62.80%.

    Matched fieldsBody

    #chinese#Model releases#Open-source ecosystem#English text is being prepared.