inclusionAI 开源 buildkit-service：面向智能体基础设施的分布式镜像构建与包镜像服务

inclusionAI; buildkit-service

Repository navigation

NotificationsYou must be signed in to change notification settings; Fork4 (4); Star2 (2)You must be signed in to star a repository

Distributed image build and package mirror services for agentic infrastructure.

Repository files navigation

README; Apache-2.0 license

Distributed image build and package mirror services for agentic infrastructure.It batch-builds Dockerfiles into OCI / Nydus images, pushes them to any OCI registry, and ships an in-cluster package cache (pip / npm / apt / yum / apk / git / container registry) that dramatically speeds up dependency installation during builds.

Features

Batch builds: buildctl-batch CLI drives large Dockerfile build sets across a horizontally scalable pool of buildkitd workers, with retries, failure export, and consistent-hash worker scheduling; HTTP build API: buildctl-daemon accepts a single Dockerfile context zip per request, supports OCI / Nydus / dual-format output, FIFO build queues (modes), and rendezvous-hash registry routing for sharding pushes across multiple registries; Package mirror: in-cluster caches for pip, npm, apt/yum/dnf, apk, git, and pull-through container registry mirrors — with graceful fallback to upstreams; Nydus support: built on Nydus, enabling lazy-pulling container images; Cloud neutral: plain Kubernetes + Helm; works on any cloud or on-premises cluster. Any OCI registry (Docker Hub, ghcr.io, Harbor, ACR, ECR, GAR, ...) can be a build target or mirror upstream; P2P preheating: optional Dragonfly image preheating (docs/preheat.md)

Kubernetes with support for privileged Pods (required by buildkitd); Helm 3, kubectl, curl, zip, and openssl; Cluster egress to GHCR, Docker Hub, and the target registry; About 4 CPU, 8 GiB memory, and 20 GiB ephemeral storage for this evaluation

The chart defaults to one replica per workload, node-local caches, no PVC, and no dedicated-node selectors. chart/values-quickstart.yaml adds tighter API and cache limits for a small development cluster. Size and isolate production workers using the guidance in docs/buildctl-batch.md.

The default service image is published by the release workflow. Verify that it is available before installing:

For a fork or before its first release, build and push the image to a registry your cluster can pull from, then set SERVICE_IMAGE:

Otherwise use the published default:

Install the build service without the optional package mirrors:

Keep this port-forward running in a second terminal. The API is intentionally ClusterIP-only by default:

Create a minimal context and push an OCI image to a unique, anonymous ttl.sh repository (the image expires after one hour):

Production targets should use an authenticated OCI registry and immutable image tags. Install the package mirrors separately when needed; see docs/package-mirror.md.

For a private target registry, create a Docker config Secret instead of putting credentials in Helm values or shell arguments:

The referenced Docker config controls target push and manifest-check credentials. Keep service-image pull credentials in the same config or provide another compatible pull Secret.

Documentation

Repository layout

Standalone local development

Use make dev-setup and make dev-up for a persistent environment built from this checkout. After source changes, run make dev-rebuild; make dev-down stops applications and preserves data. See local development for requirements, recovery, diagnostics and explicit destructive reset.

Building from source

Requires Go 1.23 or newer. The batch CLI requires Linux, a C compiler and LMDB development files (liblmdb-dev on Debian/Ubuntu); its result store is unavailable without CGO. Cross-compiling it requires a matching Linux C toolchain and LMDB library. On macOS, build and run the container image or use the local E2E suite. Docker with Buildx is required to build the container images.

Release workflows publish buildctl-daemon, apt-cacher-ng, git-cache, and preheat images to ghcr.io/ /buildkit-service/.

Local end-to-end tests

Requires Docker with Linux containers, Git, Make and uv.

The suite builds current sources and deploys them into an isolated kind cluster with a local registry. It targets linux/amd64: Apple Silicon uses emulation; Linux amd64 runs natively. See the E2E guide for resource requirements, lifecycle commands, diagnostics and coverage boundaries.

buildctl-daemon accepts source archives and controls image builds. Keep it on a trusted network. The chart disables its public LoadBalancer by default; use port-forwarding for development or place an HTTPS ingress/reverse proxy in front of it for remote access. Never send the Bearer token over public plain HTTP, and never expose the pprof port publicly.

Distributed image build and package mirror services for agentic infrastructure.