Spirit AI 开源 Spirit-v1.5 机器人 VLA 基础模型

Spirit-v1.5 A Robotic Foundation Model by Spirit AI

This repository contains the official implementation of the **Spirit-v1.5 VLA model**, as well as the runtime wrapper required to reproduce our results on the RoboChallenge benchmark.

As of Jan 11, 2026, Spirit-v1.5 ranks **#1** on the RoboChallenge Table30 benchmark. 📰 News **[2026-01]** Initial Release: Technical blog, inference code, base model checkpoint, and a fine-tuned checkpoint are now available. **[2026-4]** Fine-tuning code released! Directory Structure

Installation & Setup System Requirements **Hardware**: Tested on NVIDIA A100 80GB GPU. **Software**: Python 3.10+. Installation Options Option 1: uv (recommended) For inference only:** For training:** Option 2: pip For inference only:** For training:** **Note**: `flash-attn` requires matching CUDA and PyTorch versions. If installation fails, refer to the flash-attn installation guide. Dependency Files `requirements-base.txt` - Core dependencies for inference (backward compatible) `requirements-train.txt` - Additional dependencies for training `requirements.txt` - Complete dependencies (base + training) `pyproject.toml` - Project configuration with optional `[train]` extra Key Dependencies Base (Inference):** `torch==2.8.0` - PyTorch deep learning framework `torchvision==0.23.0` - Computer vision utilities `transformers==4.57.1` - Hugging Face transformers library `diffusers==0.35.2` - Diffusion models library `safetensors==0.5.3` - Safe tensor serialization `numpy==2.2.6` - Numerical computing `pillow==10.4.0` - Image processing `requests==2.32.5` - HTTP library `scipy==1.15.2` - Scientific computing Training (Additional):** `opencv-python>=4.8.0` - Image processing for data augmentation `wandb>=0.16.0` - Experiment tracking and logging `tqdm>=4.66.0` - Progress bars `einops==0.8.1` - Tensor operations (required by flash-attn) `flash-attn==2.8.3` - Flash Attention 2 for efficient training Model Checkpoints | Model | Type | Link | |----------|-------------|-------------| | Spirit-v1.5 | Base Model | | | Spirit-v1.5-move-objects-into-box | Fine-tuned Model| | Run RoboChallenge We provide a minimal launcher script located at `scripts/run_robochallenge.sh`. Environment Configuration The script requires the following environment variables to be set. Note that `USED_CHUNK_SIZE` defaults to 60 if not specified; all other variables are mandatory.

| Variable | Description | |----------|-------------| | `TASK_NAME` | Must correspond to a task defined in `robochallenge/runner/task_info.py`. | | `ROBOCHALLENGE_JOB_ID` | The unique ID for the job collection. | | `USER_TOKEN` | Your authentication token. | | `CKPT_PATH` | Directory containing the `model.safetensors` file. | | `USED_CHUNK_SIZE` | Action chunk size (Default: 60). | Execution Example:

Below is an example for the RoboChallenge task `move_objects_into_box`.

Finetune Guide Prerequisites Pretrained Spirit-v1.5 base model checkpoint Training dataset (see Dataset section below) Multi-GPU setup (recommended: 8x A100 80GB) Training Script

The training script is located at `scripts/run_finetune.sh`. It uses PyTorch's `torchrun` for distributed training. Environment Variables

Set the following environment variables before training:

| Variable | Description | Required | |----------|-------------|----------| | `DATA_ROOT` | Path to training dataset directory | Yes | | `PRETRAINED_PATH` | Path to pretrained model checkpoint (must contain `model.safetensors` and `config.json`) | Yes | | `OUTPUT_DIR` | Directory for saving checkpoints (default: `./outputs`) | No | | `NUM_GPUS` | Number of GPUs to use (default: 8) | No | | `BATCH_SIZE` | Training batch size per GPU (default: 32) | No | | `MAX_TRAIN_STEPS` | Maximum training steps (default: 40000) | No | | `LOG_INTERVAL` | Logging interval in steps (default: 25) | No | | `SAVE_STEPS` | Checkpoint saving interval (default: 2500) | No | | `NUM_WORKERS` | Number of data loading workers (default: 32) | No | | `PREFETCH_FACTOR` | Data prefetch factor (default: 8) | No | | `WANDB_MODE` | Weights & Biases logging mode (default: `disable`) | No | Training Example

Training Output

Training outputs will be saved to the `OUTPUT_DIR`: **Checkpoints**: Saved every `SAVE_STEPS` steps **W&B Logs**: If enabled, metrics are logged to Weights & Biases Dataset

Download the training dataset from Hugging Face:

Set `DATA_ROOT` to the downloaded dataset directory when running training. Distributed Training

The training script uses PyTorch's Fully Sharded Data Parallel (FSDP) for efficient multi-GPU training: Automatically shards model parameters across GPUs Reduces memory footprint for large models Supports gradient checkpointing for memory efficiency Intended Uses

Spirit-v1.5 is a Vision-Language-Action (VLA) model designed specifically for robotic control. The model accepts current observations and textual descriptions as input and generates the next action chunk for the robot to execute. Out-of-scope Uses

Our models are not specifically designed for any tasks or scenarios other than robotic manipulations.

Developers should expect failures in generation results regarding the out-of-scope scenarios.

Developers should be aware of and adhere to applicable laws or regulations (including privacy, trade compliance laws, etc.) that are relevant to their use case, and evaluate and mitigate for privacy, safety, and fairness before using within a specific downstream use case, particularly for high-risk scenarios. Bibtex Acknowledgments This codebase borrows code from openpi, qwen-vl and RoboChallengeInference. We thank them for their efforts and innovations, which have made the development process more efficient and convenient.

Thank you to everyone who contributed their wisdom and efforts to this project. Contact