Liquid AI 发布实时推理引擎 RIPL，端到端延迟低于 50 ms
Liquid AI Launches RIPL, a Real-Time Inference Engine for Sub-50 ms LFMs - MarkTechPost
Liquid AI 发布实时推理引擎 RIPL（Real-time Inference Pipeline），用于在实时请求路径中服务 LFM2.5 系列模型（230M 至 2.6B 参数），端到端生成响应低于 50 ms。按 Liquid AI 自测数据，在单张 B200 上其 p95 延迟比运行在 vLLM 上的 Qwen3.5 低 5.6 至 28.4 倍，其中 LFM2.5-1.2B 为 155.8 ms、Qwen3.5-0.8B 为 4431 ms。该引擎仅面向企业部署，支持 NVIDIA H100、H200、B200 和 RTX PRO 6000，Shopify 已在搜索场景以低于 20 ms 的推理延迟运行 LFM。
正文
Add as a preferredsource on Google
Liquid AI just announced Liquid RIPL, short for Real-time Inference Pipeline. RIPL is a low-latency engine that serves Liquid Foundation Models (LFMs) inside a live request path. It returns complete generated responses end-to-end in under 50 ms. It also claims that latency holds while traffic arrives continuously at peak load. For developers, the message is simple. Swap gradient-boosted trees for a generative model, and keep the latency budget.
TL;DR
Engine: Serves LFM2.5 models from 230M to 2.6B parameters
Runs on: Enterprise deployments on NVIDIA H100, H200, B200 and RTX PRO 6000 GPUs. Benchmarks used a single B200.
Performance: 5.6 to 28.4x lower p95 latency than Qwen3.5 on vLLM, per Liquid’s own tests.
Best: LFM2.5-1.2B hit 155.8 ms p95 vs 4,431 ms for Qwen3.5-0.8B, a 28.4x gap (256 in, 128 out).
Worst: LFM2.5-2.6B gained only 5.6x (449.1 ms vs 2,499 ms). Sub-50 ms fails on longer requests.
What problem does Liquid RIPL solve?
Most serving stacks target large batches and long agentic jobs. A few seconds per request is acceptable there. Search ranking, fraud checks and payment authorization need answers in milliseconds. Liquid AI argues teams default to classical models for these paths because LLM serving is too slow. RIPL is built to remove that trade-off.
Open Source OmniParseBench · by Datalab · Apache 2.0
Benchmark any document parser on tables, math, forms and handwriting. Yes/no tests per page across text, table and layout families. Score vendor parsers on the Hugging Face dataset or your own PDFs with score and predict primitives.
★ Star on GitHub Multi-column, rotated text, degraded scans · tagged by script and language
github.com/datalab-to/omni_parse_bench Fork it · Add your parser
How does RIPL work?
Liquid AI describes RIPL as optimizing the full path, from request arrival to completed response. In several workloads, Liquid AI plots the same model, LFM2.5-350M, on vLLM and on RIPL. RIPL’s p95 line sits well below the vLLM line in each case. That isolates part of the gain to the engine, not just the model architecture.
The LFM2.5 models themselves are hybrids. LFM2.5-1.2B-Instruct has 16 layers: 10 double-gated convolution blocks and 6 GQA blocks. LFM2.5-2.6B has 30 layers: 22 convolution blocks and 8 GQA blocks.
RIPL benchmarks: how fast is it vs vLLM?
Liquid measured p95 end-to-end latency under open-loop Poisson arrivals on one B200. LFMs ran on RIPL. Baselines, including MiniCPM5 and Granite 4.2, ran on vLLM.
WorkloadLoadLFM2.5 on RIPL (p95)Qwen3.5 on vLLM (p95)Gap
1B class, 32 in → 32 out1,000 req/s34.6 ms (1.2B)498 ms (0.8B)14.4x
1B class, 256 in → 128 out400 req/s155.8 ms (1.2B)4,431 ms (0.8B)28.4x
2B class, 32 in → 32 out1,000 req/s76.4 ms (2.6B)586.8 ms (2B)7.7x
2B class, 256 in → 128 out300 req/s449.1 ms (2.6B)2,499 ms (2B)5.6x
Source: Liquid AI blog, Figure 2. Sealf-reported.
Liquid AI also mapped 6 real-time workloads against fixed budgets:
Adtech query rewriting: 32 in, 32 out, 50 ms budget, LFM2.5-350M.
Search intent classification: 128 in, 1 out, 20 ms budget.
Search reranking: 1k in, 1 out, 30 ms budget.
Payments risk scoring: 512 in, 16 out, 50 ms budget, LFM2.5-1.2B.
Credit decisioning: 8k in, 64 out, 150 ms budget.
Image alt text: 512×512 image and 64 tokens in, 16 out, 100 ms budget, LFM2.5-VL-450M.
Competitor Comparison
FeatureLFM2.5-1.2B on RIPLQwen3.5-0.8B on vLLMLFM2.5-2.6B on RIPLQwen3.5-2B on vLLM
Parameters1.17B0.8B2.69B2B
Context32,768262,144131,072262,144
License (weights)LFM Open License v1.0Apache 2.0LFM Open License v1.0Apache 2.0
ModalityTextText, image, videoTextText, image, video
Default modeInstructNon-thinkingAlways reasonsNon-thinking
Serving engineRIPL (enterprise)vLLM (open source)RIPL (enterprise)vLLM (open source)
p95, 32→3234.6 ms498 ms76.4 ms586.8 ms
p95, 256→128155.8 ms4,431 ms449.1 ms2,499 ms
Is RIPL used in production?
Yes, according to Liquid AI. Shopify is deploying LFMs across latency-sensitive commerce workloads. In search enhancement, LFMs run with sub-20 ms inference on production traffic. According to Liquid AI’s blog, they use about 50% fewer parameters than alternatives and deliver 2 to 10x faster inference.
Key Takeaways
RIPL is Liquid AI’s engine for LLMs inside live request paths.
Liquid claims 5.6 to 28.4x lower p95 latency than Qwen3.5 on vLLM.
Same-model tests (LFM2.5-350M) show RIPL itself beats vLLM.
Shopify runs LFMs at sub-20 ms in production search.
Enterprise-only, closed, and all figures are self-reported.
Asif RazzaqWebsite | + postsBio ⮌
Asif RazzaqGradium Open-Sources Phonon, a 31M-Parameter On-Device TTS Model Under Apache 2.0
Asif RazzaqIBM AI Releases Granite Speech 5.0 2B for Multilingual Transcription and Translation
RELATED ARTICLESMORE FROM AUTHOR
Gradium Open-Sources Phonon, a 31M-Parameter On-Device TTS Model Under Apache 2.0
IBM AI Releases Granite Speech 5.0 2B for Multilingual Transcription and Translation
A Coding Guide to Cactus Whistle: Does a 16.9 MB Speech Model Hold Up Against Whisper Base? Word Error Rate, Speed, Streaming, Timestamps, Keywords,...