Football_MDP

Gym RL Environment for Football Skill Game.

Football Skills Environment

Overview

An RL environment featuring deterministic movement and stochastic shooting mechanics. The agent navigates from a fixed start position (0, 10), avoids obstacles, and attempts to score goals.

Environment Specifications

Component	Specification
State Space	(row, column, has_shot) - 800 total states (20×20×2)
Action Space	7 discrete actions: 4 deterministic movements + 3 stochastic shots
Grid Size	20×20, origin at (0,0) (top-left)
Start Position	(0, 10) (fixed)

Actions

Movement (0-3): Deterministic single-step movements (Up/Down/Left/Right)
Shooting (4-6): Stochastic, episode-terminating actions

• 4 Short Straight (~2 rows, high accuracy)
• 5 Long Straight (~4 rows, lower accuracy)
• 6 Long Right Curl (~4 rows, curves right)

Transition Dynamics

Supports stationary and non-stationary ("degraded pitch") modes. Always use:

transitions = env.get_transitions_at_time(state, action, time_step)  # Non-stationary
transitions = env.get_transitions_at_time(state, action)              # Stationary

Returns: [(probability, next_state), ...] ⚠️ Never access env.T directly

Rewards & Termination

Event	Reward	Notes
Goal	High + distance bonus	Shooting distance: 2 or 4 rows only
Missed Shot	-30	Ball misses goal positions
Cone Collision	-10	Hit training cone
Cutout Collision	-20	Hit defender cutout
Movement	-1	Per action (encourages efficiency)

Termination: Episode ends immediately upon any shooting action (4/5/6).

Installation & Usage

pip install gymnasium numpy

python3 pi_vi.py