OpenAI releases Pando: a Git backend rebuilt for agent-scale parallel pushes, with median push roughly 5x faster
English text is being prepared.
OpenAI has released Pando, its self-developed Git backend, compatible with the Git protocol and clients. In production, median push latency has dropped to about 1/5 of the original, and it can serve nearly 1 PB of Git data per day. The multi-branch concurrent writes and massive concurrent reads brought by Codex once made the old system slow to push and pull, with reliability at one point below 90%. Pando instead makes individual refs serializable, maintains only causal consistency across the whole repository, and places packs in object storage for reuse across clients. The project went from inception to production in about 10 weeks using Codex, was rewritten in Rust, and rolled out via a dual-write proxy followed by phased traffic migration.
Why read this
AI score 69
The original article breaks down Pando's trade-offs of trading causal consistency for write parallelism and reusable packs for read scaling, offering insight into large-scale repository backend design.
Topics
View original postWe built Pando, a new Git backend for our monorepo, to handle more pushes and fetches without changing how our engineers and agents use Git. Pando cut push latency by ~80% and now serves nearly a petabyte of Git data a day. We can grow Git capacity alongside agent usage. https://t.co/4c0YXOfKqK
View original
We built Pando, a new Git backend for our monorepo, to handle more pushes and fetches without changing how our engineers and agents use Git. Pando cut push latency by ~80% and now serves nearly a petabyte of Git data a day. We can grow Git capacity alongside agent usage. https://t.co/4c0YXOfKqK
Body
OpenAI 发布 Pando：给 Agent 规模并行推送重做的 Git 后端，中位推送约快 5 倍
Codex 把 OpenAI 单体仓推到「多分支并发写 + 海量并发读」的极限后，他们自研 Pando，兼容 Git 协议与客户端的新后端：分支级因果一致性、跨客户端复用 packfile；生产上中位推送延迟约降到原来的 1/5，日均可吐近 1 PB Git 数据。 https://t.co/lj8LXUD8nJ
Codex 让并行改动、分支、推送、CI 拉取同时放大，旧系统去年底起慢推慢拉、合并失败，可靠性一度低于 90%。他们没换掉 Git 工作流，而是换后端：无关分支可并行更新；读侧尽量用同一份已存 pack 吃满带宽。
对比过两条常见路：三阶段提交靠多副本多数派锁 ref，全仓更新偏串行，磁盘与协调压在推送路径；共享 WAL/清单能并行上传 pack，但无关分支仍抢同一 compare-and-swap，副本跟进建索引跟不上就拖「刚推完就能拉」。Pando 松掉两点假设（全仓某时刻一致、以及按客户端现算专用 pack）改成：单条 ref 可串行化，全仓只保因果一致；pack 进对象存储，元数据记 tip、依赖与包指针。推不同分支互不堵；读多条分支时，不必假设同一快照。
写入默认只广告主干作基，广告不随百万级分支膨胀，pack 更易复用；代价是叠分支可能多传。他们仓里中位 pack <20 KiB，P99 <3 MiB。读侧把 have/want 映射成已存 ref version，靠 packfile-uri 下发可缓存包（Express 命中率 >99%），标准机可吃满 10 Gbps。后台按 LSM 思路把 L0 合到更高等级（至 L4），既压缩又当跳跃索引，规划约 O(log Δ)。
约 10 周用 Codex 从立项到生产（约 4 月起接流量），Rust 重写，先双写代理再分期切流。效果含合并更稳、单小时曾送出 >57 TB（>99% 本地缓存）。改进回馈上游（并行下包、explicit haves 等）。仿真与生产口径需分开看；核心取舍是用因果一致换写并行、用可复用 pack 换读扩展，同时保住 Git 客户端兼容。