Principia: Experiments in POP using PPO.

Currently able to get the sword!

demo.mp4

How was the environment created?

There's this cool repo SDLPoP, basically some guy just decided to disassemble and open source the OG Prince of Persia!

To create this project, i trimmed the repo to only contain the data (assets of the game) and src (logic part of game) folders along with the SDLPOP.ini (configurations eg. turning off cutscenes) file

Then using the make shared command the game was compiled on arch linux, btw (note: you'll have to recompile the game in src/ folder if you want to run it on your machine; see the SDLPoP Documentation)

Thereafter, the POP_Env.py file was created. To wrap the game around a gymnasium environment, ctypes was used to access the global variables (things like hitpoints current level, raw pixel frame, etc).

the things the agent 'sees'

The agent has access to the 84x84 grayscaled frames and states like current level, current hitpoints, max hitpoints, possession of sword, current room index and current guard hp.

The agent makes decisions with a frame skip of 4 (since 4 is the minimum number of frames any action takes)

The agent can perform 5 actions: Up, Down, Left, Right along with one Null action.

the rewards the agent receives

Action	Reward
discovering new rooms	+4
picking up the sword	+7
dying	-10
health increase	+1
health decrease	-1
decreasing guard's hp	+2
killing guard	+3
every time step (makes it kill itself if it ever gets stuck)	-0.01

How to run it on your machine?

Docker (recommended):

Using the docker image is much simpler, available at: Docker Hub

docker pull oceanthunder/principia

xhost +local:docker
docker run --rm -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix --name principia --gpus all --ipc=host -p 6006:6006 oceanthunder/principia

after which run multippo.py to start the training [change the num_cpu param in multippo.py file to inc/dec the no. of environments in parallel]

Command	Mode
python multippo.py 1	Single Window (Only 1 GUI opens)
python multippo.py 2	All Windows (GUI for every environment)
python multippo.py 3	Headless (No windows, fastest)

Example:

python multippo.py 1

To access logs, open a new terminal and then:

docker exec -it principia /bin/bash

now, inside the container, type:

tensorboard --logdir logs --port 6006 --bind_all

Then, open up a browser and visit http://localhost:6006 to see tensorboard logs!

Manual (if you are real serious about this):

Recompile the game as mentioned above.

Use python version 3.12.12 (might work on other python versions, haven't tested)

pip install the requirements.txt file

run the multippo.py file to start training

Notes for meself

nm -D ./SDLPoP/src/libSDLPoP.so [very useful command to find what methods are available in libSDLPop.so file, just pipe em up!]