slimSAC - simple, minimal and flexible implementation of Soft Actor-Critic

slimSAC provides a concise and customizable implementation of Soft Actor-Critic (SAC) algorithm in Reinforcement Learning⛳ for MuJoCo and DeepMind Control Suite environments. It enables to quickly code and run proof-of-concept type of experiments in off-policy Deep RL settings.

User installation

GPU installation:

python3 -m venv env
source env/bin/activate
pip install --upgrade pip setuptools wheel
pip install -e .[dev,gpu]

To verify the installation, run the tests as:pytest

Running experiments

Training

To train a SAC agent on DMC task dog-walk on your local system, run:
launch_job/dmc/local_sac.sh --experiment_name test_run_dog-walk --first_seed 0 --last_seed 0 --disable_wandb

It trains a SAC agent with 2 hidden layers of size 256 in both policy and critic network, for 1_000_000 steps.

• To see the stage of training, you can check the logs in experiments/dmc/logs/test_run_dog-walk/sac folder
• The models and episodic returns are stored in experiments/dmc/exp_output/test_run_dog-walk/sac folder

To train on cluster:
launch_job/dmc/cluster_sac.sh --experiment_name test_run_dog-walk --first_seed 0 --last_seed 0 --disable_wandb