TACO: TActile World Model as a Self-COrrector for Scalable VLA Post-Training

Shengbang Liu^1,3,*   Yueru Jia^1,2,*   Yuyang Yan^1,*   Jiaming Liu^1,*,†   Xinran Zhang^1,2,*   Qiuxuan Feng¹   Yandong Guo²   Shiji Zhou⁴   Boxin Shi¹   Shanghang Zhang^1,📧

¹State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University
²AI2 Robotics   ³Sun Yat-sen University   ⁴Beihang University

_{^*Equal Contribution  ^†Project Lead  ^📧Corresponding Author}

---

TACO is a tactile-aware world-model-driven framework for scalable VLA post-training in contact-rich robot manipulation. Given real-world rollouts, TACO follows a Recognize-Imagine-Label loop: it recognizes failure-adjacent contact states, imagines local visuo-tactile correction segments, and labels corrective actions for policy post-training.

The resulting corrective supervision is used with knowledge-insulated tactile adaptation, allowing the policy to learn contact recovery behaviors without degrading pretrained visual-language priors.

This repository now includes staged releases of the TACO tactile-aware world model and tactile-aware VLA. The released code covers visuo-tactile joint denoising for Wan-style video world models, tactile/force sequence loading, tactile-aware VLA model and training code with advantage conditioning and knowledge insulation, training/cache runners, inference support, and example configs. Model checkpoints, datasets, and the full post-training loop will be released progressively.

📋 Table of Contents

• Roadmap
• Repository Structure
• World Model
• Citation

🗓 Roadmap

• Open-source visuo-tactile world model
• Open-source tactile-aware VLA model
• Open-source the full TACO framework

📁 Repository Structure

TACO/
├── assets/                         # Figures, videos, and project media
├── visuo_tactile_world_model/      # First staged release: tactile-aware world model
│   ├── configs/                    # Accelerate / distributed runtime configs
│   ├── examples/                   # Minimal YAML example
│   ├── scripts/                    # Data preparation and tactile utility scripts
│   ├── visuo_tactile_world_model/
│   │   └── world_model/            # Visuo-tactile world-model Python package
│   ├── run.py                      # YAML config launcher
│   └── pyproject.toml              # Editable install metadata
├── tactile_aware_vla/              # Tactile-aware VLA training and inference code
│   ├── examples/taco/              # HDF5-to-LeRobot tactile data conversion
│   ├── scripts/                    # Norm stats, training, and policy serving
│   ├── src/openpi/                 # pi0.5 tactile + advantage + KI implementation
│   ├── README.md                   # Module setup and training guide
│   └── pyproject.toml              # Editable install metadata
├── README.md                       # Project overview
├── LICENSE                         # Apache-2.0 license
└── .gitignore

📚 Each released module keeps its setup notes in the module README.

🌐 World Model

The visuo-tactile world model release lives in visuo_tactile_world_model. See that README for setup, examples, and model-specific notes.

🤖 Tactile-Aware VLA

The tactile-aware VLA release lives in tactile_aware_vla. It provides pi0.5-style flow-matching training with force-history conditioning, scalar advantage conditioning, and knowledge-insulated adaptation. See the module README for data format, conversion, training, and inference instructions.

🙏 Acknowledgement

We thank the LightEWM project for its valuable codebase and engineering foundation, which informed the development of the visuo-tactile world model release.

📄 Citation

@article{liu2026taco,
  title={TACO: TActile World Model as a Self-COrrector for Scalable VLA Post-Training},
  author={Liu, Shengbang and Jia, Yueru and Yan, Yuyang and Liu, Jiaming and Zhang, Xinran and Feng, Qiuxuan and Guo, Yandong and Zhou, Shiji and Shi, Boxin and Zhang, Shanghang},
  journal={arXiv preprint},
  year={2026}
}