BeamNG-AI-Driver

BeamNG.tech connection and full-driver-network baseline for future AI-driving experiments.

The project is being reset. Previous driving work proved useful pieces of the workflow, but the active project is now intentionally stripped back to:

• local BeamNG.tech configuration
• BeamNGpy connection validation
• launch and scenario smoke testing
• basic vehicle telemetry and camera polling
• full neural-network architecture definition
• training scaffold and one-step live policy-environment validation
• live rollout and PPO readiness checks
• first curriculum definitions for starting RL carefully
• layered long-term architecture for actuator control, driving skills, role behavior, and mission tasks

The AI-driving architecture is being redesigned from this clean baseline. The long-term direction is layered: learn vehicle controls first, then reusable driving skills, then role-aware behavior such as police officer or getaway driver tasks.

Current Status

Active status: first constrained full-driver PPO training path is ready; actuator-control work is support work, not the main blocker.

Completed:

• BeamNG.tech path/config validation.
• BeamNGpy import/config validation.
• Live BeamNG.tech launch through BeamNGpy.
• Small scenario load on smallgrid.
• Vehicle spawn.
• Basic Electrics and Damage polling.
• Basic camera creation and colour-frame polling.
• Full PyTorch driver-network skeleton with realistic sensor branches and BeamNGpy vehicle-control/light outputs.
• Observation and action adapters for the full driver network.
• Checkpoint save/load and rollout storage scaffolding.
• One-step live BeamNG.tech environment wrapper that polls a realistic observation, runs the policy, decodes an action, and keeps supervisor info outside the policy observation.
• Short live rollout runner for repeated observation-to-action steps.
• PPO action sampling, log-prob scoring, and update plumbing.
• Basic supervisor/reward module kept separate from policy observations.
• Safe live control-application validator.
• First tiny curriculum definitions for stationary rollout, low-speed control wiring, and the first RL rollout.
• Gearbox and parking-brake curriculum validator for teaching reverse, neutral, drive, brake, parking brake, and light-throttle outcomes before movement training.
• First live PPO trainer for the gearbox, brake, parking-brake, and throttle curriculum with steering/lights/clutch clamped out.
• Compact NEAT actuator-controller scaffold for gearbox, throttle, brake, and parking-brake evolution.
• Strict NEAT cleanliness gate that checks full-sequence coverage, sustained phase outcomes, and actuator stability before any live evaluation is accepted.
• Staged NEAT actuator lessons validated offline and live for Drive movement, Drive-to-stop, neutral parked throttle, and Reverse movement.
• Reverse transition bridge validated offline and live from neutral parked throttle into Reverse movement and final parked neutral.
• First constrained full-driver PPO trainer added for low-speed driving with Drive, steering, throttle, and brake active.
• Layered architecture registry for perception, actuator control, driving skills, role behavior, and mission/task execution.

Not active:

• training-center generation
• green-gate stopping
• low-speed circle training
• obstacle-stop training
• active reinforcement-learning runs
• trained model checkpoints
• trained driving behavior
• completed NEAT actuator policy

Those are preserved as prototype history, not the active architecture.

Quick Start

Install Python dependencies:

pip install -r requirements.txt

Create local BeamNG.tech settings:

Copy-Item systems\local_config.example.py systems\local_config.py

Edit systems\local_config.py for your machine. At minimum, set:

• BEAMNG_HOME
• BEAMNG_BINARY
• BEAMNG_USER
• BEAMNG_TECH_DOWNLOAD_DIR, optional
• BEAMNG_TECH_ZIP, optional

Run local validation:

python scripts\stage1_validate_beamngtech.py

Run the live smoke test:

python scripts\stage1_live_smoke_test.py

Validate the full driver-network architecture without training:

python scripts\define_full_driver_network.py

Validate the training scaffold without launching BeamNG.tech:

python scripts\prepare_training_scaffold.py

Validate PPO training readiness without launching BeamNG.tech:

python scripts\validate_training_readiness.py

Validate the layered architecture direction:

python scripts\validate_layered_architecture.py

Validate the live policy-environment wrapper:

python scripts\validate_live_training_env.py

This launches BeamNG.tech, loads a simple smallgrid scenario, captures one selected camera view plus basic vehicle telemetry, runs one network inference pass, decodes the action, writes an ignored report under logs/, and closes. It does not apply controls unless --apply-controls is passed.

Run a short observe-only live rollout:

python scripts\run_live_rollout.py

Add --sample-actions to exercise the PPO sampling path. Add --apply-controls only when you intentionally want the untrained policy to send controls to the vehicle.

Validate safe live control application:

python scripts\validate_live_control_application.py

This applies a short low-risk brake/throttle/brake sequence only to prove the control path. It is not a driver and not training.

Validate the first drivetrain-control curriculum:

python scripts\validate_gearbox_parking_brake_curriculum.py

This checks the sequence needed before driving training: neutral with parking brake, drive with light throttle, braking to stop, neutral/parked throttle, reverse with light throttle, and final neutral with parking brake.

Run the first live RL trainer:

python scripts\train_gearbox_parking_brake.py --headless --episodes 50

This trains only the first drivetrain-control lesson. It does not train general driving behavior. Training stops early when the configured rolling reward target is reached.

Run fast offline RL warm-up for the same lesson:

python scripts\train_gearbox_parking_brake_offline.py --episodes 150

This uses a small local drivetrain simulator for faster reward learning before live BeamNG evaluation. It is reinforcement learning from reward, not behavior cloning.

Train the compact NEAT actuator controller. Use staged lessons first:

python scripts\train_gearbox_neat.py --lesson drive --generations 80
python scripts\train_gearbox_neat.py --lesson stop --generations 100
python scripts\train_gearbox_neat.py --lesson neutral --generations 40
python scripts\train_gearbox_neat.py --lesson reverse --generations 100
python scripts\train_gearbox_neat.py --lesson reverse_bridge --generations 220

Evaluate the winning NEAT controller live:

python scripts\evaluate_gearbox_neat_live.py logs\gearbox_neat\<run_id>\winner.pkl --lesson drive --headless

NEAT is currently scoped to the actuator-control layer only. It evolves a small controller for reverse, neutral, drive, throttle, brake, and parking brake. The current full-sequence lesson is intentionally not marked complete. The staged lessons and the reverse transition bridge have passed offline and live validation, but the combined sequence still needs a clean merged policy. Current staged status:

1. release parking brake and move in Drive: passed offline and live
2. brake to a controlled stop: passed offline and live
3. neutral with parking brake set and throttle response: passed offline and live
4. release parking brake and move in Reverse: passed offline and live
5. transition from neutral-throttle check into Reverse and final parked neutral: passed offline and live
6. combine the staged lessons into one full-sequence policy: next

Do not live-evaluate a NEAT winner unless its report has clean_learned: true and winner_cleanliness.passed: true.

Run the first direct driving-NN trainer:

python scripts\train_basic_driving.py --headless --episodes 5

This uses the full driver network and realistic policy observations. The first driving curriculum constrains gear to Drive, caps steering, disables brake by default, and clamps lights, clutch, and parking brake so PPO can first learn low-speed forward movement on a track. Supervisor telemetry may grade reward, but it is not copied into the policy observation.

Run the first visible track smoke test on Hirochi Raceway:

python scripts\train_basic_driving.py --map-name hirochi_raceway --scenario-name basic_driving_hirochi_startline --spawn-x -408.480164 --spawn-y 260.231537 --spawn-z 25.4231434 --spawn-rot-quat 0 0 -0.2799072306 0.9600270529 --episodes 1 --max-steps 35 --step-time 0.2 --max-steering 0.04 --max-throttle 0.08 --max-speed-mps 8 --target-speed-mps 4 --keep-open

Observed default etk800 automatic gear command mapping:

• -1 = reverse
• 0 = neutral
• 1 = park
• 2 = drive
• 3 = sport/first gear

Active Files

• scripts/stage1_validate_beamngtech.py - checks Python, BeamNGpy, local BeamNG.tech paths, tech.key, user directory, and API host settings.
• scripts/stage1_live_smoke_test.py - launches BeamNG.tech, loads a simple scenario, spawns one vehicle, polls telemetry, polls a camera frame, and closes the session.
• scripts/define_full_driver_network.py - instantiates the full PyTorch driver network and writes an ignored architecture report.
• scripts/prepare_training_scaffold.py - validates observation adapters, action decoding/sampling, checkpoint save/load, and rollout storage without training or opening BeamNG.tech.
• scripts/validate_training_readiness.py - runs a synthetic rollout through PPO update plumbing without launching BeamNG.tech.
• scripts/validate_layered_architecture.py - validates the long-term layered architecture and writes an ignored architecture report.
• scripts/validate_live_training_env.py - launches BeamNG.tech and validates one live observation-to-policy-to-action pass without training.
• scripts/run_live_rollout.py - runs a short repeated live policy loop, observe-only by default.
• scripts/validate_live_control_application.py - safely proves BeamNG.tech accepts applied vehicle controls.
• scripts/validate_gearbox_parking_brake_curriculum.py - validates the first gearbox, brake, parking-brake, and throttle curriculum sequence.
• scripts/train_gearbox_parking_brake.py - first live PPO trainer for the drivetrain-control curriculum.
• scripts/train_gearbox_parking_brake_offline.py - fast offline PPO warm-up for the same gearbox/parking-brake curriculum.
• scripts/train_gearbox_neat.py - evolves a compact NEAT actuator controller for the gearbox/parking-brake curriculum.
• scripts/evaluate_gearbox_neat_live.py - evaluates a winning NEAT actuator controller in live BeamNG.tech.
• scripts/train_basic_driving.py - first constrained live PPO trainer for the full driver network on low-speed driving.
• launch_beamngtech_api.bat - convenience launcher for BeamNG.tech with the API enabled.
• systems/config.py - default config values.
• systems/local_config.example.py - local override template.
• systems/architecture.py - layered architecture and role definitions for actuator control, driving skills, police behavior, getaway behavior, and future mission tasks.
• systems/driver_io.py - realistic observation/action contract for the full driver policy.
• systems/driver_network.py - full PyTorch driver policy skeleton.
• systems/driver_runtime.py - converts realistic sensor data into policy tensors, decodes policy outputs into BeamNGpy control/light commands, and handles checkpoints.
• systems/beamng_training_env.py - minimal live BeamNG.tech environment wrapper for polling observations, running inference, and optionally applying decoded controls.
• systems/supervisor.py - supervisor-only reward and termination helpers.
• systems/curriculum.py - first small curriculum definitions for starting RL without drifting into scripted driving.
• systems/gearbox_training.py - action filtering and phase rewards for the first gearbox/parking-brake RL curriculum.
• systems/neat_gearbox.py - compact NEAT observation/action/simulation logic for the actuator-control layer.
• systems/neat_gearbox_config.ini - NEAT configuration for the gearbox actuator controller.
• systems/driving_training.py - action constraints for the first direct full-driver driving curriculum.
• systems/training.py - rollout buffer, action sampler, and small PPO trainer for first controlled experiments.

Prototype History

Older AI-driving scripts and systems live under:

prototypes/

These files are reference material only. They should not be expanded as the current architecture unless deliberately reintroduced.

The old work includes:

• BeamNG connection helpers
• scripted control tests
• camera/perception probes
• training-center experiments
• green-gate stopping
• low-speed circle experiments
• tiny MLP model tooling
• old LLM/navigation/traffic prototypes

Driver Network Direction

The current driver network is a full-policy scaffold, but the long-term system is layered rather than one monolithic brain. Low-level actuator control should be learned first, then reusable driving skills, then role behavior, then mission tasks.

Layer order:

1. Perception - sensor features from human-equivalent inputs.
2. Actuator control - gear, throttle, brake, parking brake, clutch, and steering basics.
3. Driving skill - move, stop, reverse, turn, park, follow roads, avoid obstacles.
4. Role behavior - police officer, getaway driver, patrol, pursuit, escort, or other role-specific tactics.
5. Mission/task execution - convert a goal into role-aware objectives and success conditions.

NEAT is a candidate for the small actuator-control layer because it can evolve neurons/connections in a compact control network. The full camera/perception stack should not be evolved directly with classic NEAT; it should feed features into smaller skill or role policies.

The first active learning target is actuator control, not open-road driving.

Allowed observation branches:

• stacked RGB camera frames from one active view
• optional depth-camera frames from one active view
• active camera view selector: forward, left, right, or back
• vehicle electrics telemetry
• IMU
• GPS
• damage/vehicle condition indicators
• G-forces
• powertrain telemetry
• radar
• LiDAR
• ultrasonic
• optional external audio feature vector
• local timing features

Outputs:

• steering
• throttle
• brake
• parking brake
• clutch
• gear selection
• left signal
• right signal
• hazard signal
• headlights
• fog lights
• lightbar

The network also emits a value estimate for future reinforcement learning.

Excluded from driver observations:

• camera annotations
• instance masks
• ideal radar object truth
• road/lane metadata
• world mesh geometry truth
• exact object lists
• route progress
• lane-center or waypoint truth
• collision/off-road/wrong-way metadata as policy input
• teleport, recover, or reset controls

BeamNG metadata may still be used later by a separate supervisor for reward, validation, or debugging, but it must not be part of the policy observation.

The camera contract restores the previous four-view setup while keeping the human-equivalent constraint: the policy receives only one active view at a time. The allowed view names are forward, left, right, and back.

Training Readiness

The project is now prepared for live environment work, but not actual driving training yet.

Ready:

• full policy/value network
• realistic observation contract
• active-view camera input
• action decoder for BeamNGpy vehicle controls and lights
• checkpoint save/load
• rollout storage
• action sampling and log-prob scoring
• PPO update plumbing
• basic supervisor/reward helper
• first tiny curriculum definitions
• gearbox and parking-brake curriculum validation
• first live gearbox/parking-brake PPO trainer
• compact NEAT actuator-controller scaffold
• strict NEAT cleanliness validation for complete phase outcomes and stability
• staged NEAT Drive, Stop, Neutral, and Reverse actuator lessons validated offline and live
• reverse transition bridge validated offline and live
• constrained full-driver PPO training entry point for low-speed driving
• one-step live BeamNG.tech observation-to-action validation
• short live observe-only rollout validation
• controlled live action-application validation
• layered architecture validation

Still needed to make driving training productive:

• live sensor expansion beyond camera, electrics, and damage
• episode reset/start policy that does not use meta recovery as policy behavior
• one merged full-sequence NEAT actuator policy from the validated staged lessons
• reward tuning for the first direct driving curriculum
• checkpoint/resume workflow for long training runs