An Open, Modular Exploration Towards Systematic World–Action Model Pretraining

What is OpenWAM

OpenWAM is an open research stack for systematically developing World-Action Models (WAMs). It turns tightly coupled design choices into modular components and controlled experiments. It consists of:

• OpenWAM-Infra: A modular infrastructure for composing and comparing model, representation, training, inference, deployment, and evaluation choices.
• OpenWAM-Study: Controlled studies that derive practical principles for inheriting world knowledge, coupling world and action learning, and scaling across domains.
• OpenWAM-α: An open pretrained WAM that applies these principles at scale, trained on 518.5M frames (about 6,400 hours) of egocentric human and robot data.

News

• [2026/11/05] 🚀 Thanks to ResearchBoost-Agent (RBA) for accelerating OpenWAM-α training! Validation on four H200 GPUs across seven SFT benchmarks shows 2.54–2.77× training speedups over the original main. See training acceleration for the measurement conditions and current configuration.
• [2026/10/25] 🤗 We release all cleaned robot datasets used for OpenWAM pretraining: AgiBotWorld-Beta, RoboCOIN, InternData-A1, and DROID.
• [2026/10/10] 🌟 OpenWAM reaches 1K GitHub stars! Thank you for your support!
• [2026/09/09] 📄 OpenWAM Paper is released on arXiv.
• [2026/09/06] 🤖 OpenWAM is integrated into XPolicyLab.
• [2026/09/06] 🤗 We release all pretrained and finetuned models on huggingface.
• [2026/09/06] 🔥 OpenWAM Codebase Release！

Installation

Choose one installation route:

Route	Use it when you want to…
Docker	run a CUDA policy server, train, develop in a container, or deploy to an offline GPU host
Native Python environment	manage Python and CUDA dependencies directly on your machine

Docker includes the project's Python environment. GPU hosts still need an NVIDIA driver and NVIDIA Container Toolkit; see the guide's host requirements.

Native Python environment

Create an environment with conda:

# Requires Python >= 3.10
conda create -n openwam python=3.10
conda activate openwam

or with venv:

# Requires Python >= 3.10 (check with `python3 --version`)
python3 -m venv .venv
source .venv/bin/activate

OpenWAM requires PyTorch >= 2.7.1 and < 3. We recommend PyTorch 2.7.1 with CUDA 12.8:

pip install torch==2.7.1 torchvision==0.22.1 torchaudio==2.7.1 --index-url https://download.pytorch.org/whl/cu128

Then install OpenWAM:

pip install -e .

deepspeed ships as a source distribution and is compiled during pip install, so a C compiler (gcc) must be on the PATH.

Cosmos-Predict2.5 Extras (Optional) — needed only for experiments with the cosmos_predict25_2b video backbone

With your environment activated:

git submodule update --init third_party/cosmos-predict2.5
bash scripts/install_cosmos_predict25.sh

The script installs the upstream cosmos packages into the active environment and compiles transformer-engine (CUDA toolkit with nvcc required), then automatically restores the package versions OpenWAM pins.

Assets Preparation

The downloaders are interactive by default; every menu step also has a flag (--name, --source, --root, --yes, see --help) so they can run unattended, and the default storage location is resolved relative to the repository regardless of the working directory. Component downloaders store assets under assets/ and update the matching YAML path; the released-checkpoint downloader keeps each checkpoint's self-contained config unchanged.

1. Video Backbone

python scripts/download_assets/download_video_backbone.py

Supported video backbones

Wan2.2-TI2V-5B ✅	Wan2.1-VACE-1.3B ✅	Wan2.1-I2V-14B-480P ✅
Cosmos-Predict2.5-2B ✅	Cosmos3-Edge ✅

Weights are saved under assets/video_backbone_ckpt/ and the selected configs/model/video_backbone/ file is updated with the downloaded path.

2. Benchmark Data

python scripts/download_assets/download_benchmark_data.py

Supported benchmarks

RoboTwin2.0 ✅	RoboDojo ✅	RoboDojo-Real ✅
LIBERO ✅	VLABench ✅	EBench ✅
RoboCasa365 ✅	RoboCasa_GR1 ✅

Data is saved under assets/benchmark_data/<benchmark>/. Normalization statistics are prepared when needed, and the selected dataloader configuration is updated.

3. VLM Backbone (Optional)

Required only by tri_system:

python scripts/download_assets/download_vlm_backbone.py

Supported VLM backbones

Qwen3-VL-2B-Instruct ✅

Weights are saved under assets/vlm_backbone_ckpt/, and the selected configuration is updated.

4. Visual Encoders (Optional)

Required only for video backbones that use an external encoder:

python scripts/download_assets/download_visual_encoder.py

Supported visual encoders

DINOv3 ViT-B/16 ✅	V-JEPA 2.1 ViT-G/16 ✅
Wan2.2 VAE ✅	FLUX.2 VAE ✅

Weights are saved under assets/visual_encoder_ckpt/, and the selected encoder configuration is updated.

5. Released OpenWAM Checkpoints

Use this downloader to obtain OpenWAM-Alpha releases or OpenWAM-Study checkpoints from the OpenWAM collection:

python scripts/download_assets/download_openwam_checkpoints.py

Checkpoints are saved under assets/openwam_ckpt/openwam_alpha/ or assets/openwam_ckpt/openwam_study/<type>/. Each checkpoint directory contains its own config and can be deployed directly with:

bash scripts/deploy.sh <ckpt_dir_path>

For fine-tuning, set training.finetune_ckpt_path in configs/train.yaml to the downloaded checkpoint directory. Benchmark data is still required.

Quick Start

Quick Start provides a minimal end-to-end example: prepare the assets, train or fine-tune a policy, deploy its checkpoint, and run a first inference check.

The example uses the DualSystem JointSelfAttention architecture, the Wan2.2-TI2V-5B video backbone, and the Mutual attention mask.

Component	Selection	Configuration
Architecture	dual_system / joint_self_attn	configs/model/dual_system.yaml
Video backbone	wan22_ti2v_5b	configs/model/video_backbone/wan22_ti2v_5b.yaml
Attention mask	mutual	model.architecture.attention_mask_mode
Dataset	libero	configs/dataloader/libero.yaml

Resource recommendation: We recommend 8 GPUs with 80 GB VRAM each for training. This configuration supports normal training for all architectures using Wan2.2-5B or smaller video backbones. More GPUs are better when available and can further improve training throughput.

From Scratch Training

1. Download LIBERO and let the downloader update its dataloader configuration:

   python scripts/download_assets/download_benchmark_data.py

   Select LIBERO in the interactive menu.

2. Download Wan2.2-TI2V-5B:

   python scripts/download_assets/download_video_backbone.py

   Select Wan2.2-TI2V-5B and the desired model source.

3. Start a debug run with the complete model selection:

   bash scripts/train.sh \
     dataloader=libero \
     model=dual_system \
     model/video_backbone=wan22_ti2v_5b \
     model.architecture.variant=joint_self_attn \
     model.architecture.attention_mask_mode=mutual \
     training.debug=true

   training.debug=true runs 20 steps, saves at steps 10 and 20, and uses a constant learning rate. Check the run output, then set training.debug=false for normal training. Training defaults and CLI overrides are defined in configs/train.yaml. Debug outputs use training.output_path, whose default is outputs/openwam_checkpoints.

4. Deploy the debug checkpoint and inspect one input-output cycle:

   bash scripts/deploy.sh <debug_ckpt_dir_path>

   Keep the server running, then open another terminal and run the two inference helpers. Deployment enables compile by default, so the first inference may take longer while compilation warms up; later requests are typically faster:

   python scripts/inference_test/inference_single_test.py \
     --server ws://127.0.0.1:8848 --test --state-dim 10
   
   python scripts/inference_test/inference_continuous_test.py \
     --server ws://127.0.0.1:8848 --test --state-dim 10

   The single-request helper checks ping, one prediction, and reset. The continuous helper sends repeated predictions over one connection and reports the returned action dimension and latency. Stop the deployment process after the checks.

Four-Source Pretraining

Use the four cleaned LeRobot v3 datasets with the shared mixture configuration:

bash scripts/train.sh dataloader=mixture

configs/dataloader/mixture.yaml enables AgiBotWorld-Beta, RoboCOIN, DROID, and InternData-A1 with proportional sampling by default. Their standalone configurations are in configs/dataloader/ and point to /mnt/data/wangyuran/pretrain_data_final. Override each source's dataset_dir when moving the datasets, keeping the supplied meta statistics with them. Readers use the existing statistics directly and require no trim CSV, exclusion list, or statistics generation step.

OpenWAM-α Fine-Tuning

1. Download LIBERO as shown above.

2. Download the OpenWAM-Alpha foundation checkpoint:

   python scripts/download_assets/download_openwam_checkpoints.py

   Select OpenWAM_Alpha and OpenWAM-Alpha-Pretrain-Foundation-Model. Keep the resulting directory as <foundation_ckpt_dir_path>.

3. Start fine-tuning from that directory:

   bash scripts/train.sh \
     dataloader=libero \
     training.finetune_ckpt_path=<foundation_ckpt_dir_path>

   The default model configuration already matches the required setup above. You can set the same field in configs/train.yaml instead of passing it on the command line. num_frames=33 and video_stride=4 in configs/dataloader/libero.yaml produce the 32-step action horizon expected by the sampler; no separate action_chunk override is needed.

4. Deploy the resulting checkpoint directory:

   bash scripts/deploy.sh <ckpt_dir_path>

   Install and run the LIBERO client according to the LIBERO evaluation guide.

OpenWAM Usage Guidance

OpenWAM is configured through composable Hydra YAML files. Select an architecture, backbone, dataloader, and runtime behavior by changing configuration values or overriding them on the command line. The task guides below are the maintained entry points for using and extending the repository:

Guide	Use it when you want to…
Training acceleration	use the default RBA speed preset and configure gradient checkpointing or text-encoder offload
Docker and offline deployment	build an image, run a first policy, train, develop with Compose, or deploy to an offline GPU host
Training and deployment	choose a model/dataloader, prepare assets, train, fine-tune, resume, or deploy a policy
Architecture extension	extend a video, visual, VLM, action backbone, or WAM architecture
Benchmark integration	extend a dataloader and connect a benchmark client to the WebSocket protocol
OpenWAM-α fine-tuning	fine-tune the released foundation checkpoint to execute downstream task

💡 Agent tip: Pick the guide that matches your task and feed it directly to your agent — less explaining, more building.

Installation and Assets Preparation above cover environment setup and model or dataset downloads. Benchmark-specific environment and evaluation details remain in the benchmarks directory.

Training acceleration

Existing bash scripts/train.sh ... commands, including the OpenWAM-α fine-tuning example above, load presets/fast.env by default through training.speed_preset. The optimizations are included in OpenWAM; no separate RBA package or environment-variable prefix is needed.

Configure the preset and memory options in configs/train.yaml or override them on the command line:

training:
  speed_preset: presets/fast.env
  use_gradient_checkpointing: true
  gradient_checkpointing_layers: null
  text_encoder_offload: false

• gradient_checkpointing_layers: null recomputes all layers; a non-negative integer recomputes the first N layers, capped at the model depth (0: none). This layer limit applies when use_gradient_checkpointing=true to dual-system joint_self_attn and IDM's standard joint-loop fallback. IDM teacher-forcing, single-system, and tri-system paths retain their existing checkpointing policies.
• text_encoder_offload: defaults to false. Set it to true to keep frozen, eval-mode Wan text-encoder weights on the CPU between text-cache misses. This requires the text cache enabled by fast.env and is independent of gradient checkpointing. Cache misses add CPU-to-GPU weight transfers. The separate use_gradient_checkpointing_offload option controls activation offload.

To skip preset loading, add training.speed_preset=null to your training command; manually exported optimization switches remain in effect. Explicit YAML/CLI preset selection takes precedence over the legacy OPENWAM_PRESET selector, and loaded preset values override matching environment variables.

Choose batch size and memory options for your hardware. Initial iterations include compilation warmup. The speedups in News come from warmed-up training-throughput measurements on four H200 GPUs across seven SFT benchmarks, without an 80 GiB memory cap. The original main used all-layer checkpointing, while the tested optimized configuration used no checkpointing. These speedups include the benefit of avoiding recomputation and do not represent measurements of the current all-layer-checkpointing defaults above.

Projects Based on OpenWAM

Community projects and integrations that extend, deploy, or help optimize OpenWAM:

• LD4WAM — LD4WAM: Learning Latent Dynamics from Human Videos for World Action Models (arXiv:2608.22403). Learns a Latent Dynamics Model (LDM) from human videos and uses it to build a World Dynamics Action Model (WDAM) — a Wan2.2 video DiT coupled with a latent-dynamics expert and an action expert — on top of the OpenWAM framework, evaluated on RoboTwin 2.0.
• ResearchBoost-Agent (RBA) — An autonomous agent that profiles, optimizes, and validates AI training code. RBA supported the development of OpenWAM-α training acceleration.
• ManiMux — A real-robot inference, deployment, and experiment-management framework that integrates OpenWAM through XPolicyLab, connecting policy servers to robot hardware with a shared configuration workflow and the RoboGUI operator interface.

If you build on OpenWAM, feel free to open a PR to add your project here.

Star History

License

OpenWAM is released under the Apache License 2.0.

Citation

If you use OpenWAM, please cite:

@article{wang2026openwam,
  title   = {OpenWAM: An Open, Modular Exploration Towards Systematic World-Action Model Pretraining},
  author  = {Yuran Wang and Siqiao Huang and Mingleyang Li and Chenhao Zhang and Jiaqi Liang and Weiyang Jin and Yue Chen and Xuemin Chi and Donghao Zhou and Qize Yu and Yu-Kai Wang and Yuhan Rui and Shenzhe Yao and Zhen Yuan and Zhenhao Shen and Kefei Zhu and Zijie Zhu and Ning Gao and Xiaowei Chi and Guanqi He and Shanghang Zhang and Hao Dong and Lin Shao and Hang Zhao},
  year    = {2026},
  journal = {arXiv preprint arXiv: 2609.07398}
}

Contact

Have a question, want to share results, or just follow development? Join us:

WeChat Group Scan to join

Discord discord.gg/yRSqhpNXu