BSS-Batch-Manager

Setup

The project has been setup using uv

You can also define the following config options in config.csv

• username - The username used to SSH into the host when submitting batches (defaults to your current system username)

• hostname - The server address the program submits batches to (required)

You must have an SSH key for the host added to your SSH agent

You must also be able to access the host either via local network or via a VPN

git clone git@github.com:MarshTheBacca/BSS-Batch-Manager.git --recurse-submodules
uv run main.py

Interface

The program uses an integer-based menu system. Your inital options are:

Create a batch

You will be prompted to choose the following:

• A starter network from networks

• A LAMMPS potential file from potentials

• Variables of the Bond Switch Simulator parameters to vary (maximum of 9)

After choosing your variables, for each one, you will be prompted to choose a method of variation, which changes depending on the variable

Variables that are not chosen will remain as defined in the batch template for all jobs in the batch

The batch will be saved as a zip file in batches

Delete a batch

The program will scan all batches in the batches directory and prompt the user to delete one

Edit the batch template

The program will prompt the user to choose a variable to edit in the batch template and give a new value until they exit

Submit batch to host

The program will prompt the user to choose a batch to submit, while also displaying information such as the number of jobs and date submitted (sorted by latest submission first)

Upon confirmation of submission, a daemon process is initiated which does the following:

• Checks to see if you have the relevent files in your home directory on the host, if not, copies over your local copy

• Sends over the batch zip and executes the remote python script on the host

• Executes check_file_exists.sh on the host every 5 seconds until the output batch zip has been detected

• Copies over the batch output zip, deletes it on the host and unzips it into output_files

The log of this script is placed in /utils/batch_submit.log

The remote python script does the following:

• Unzips the batch

• Copies in all relevant files in the correct directories, ie, the initial network, potential files and bss_parameters.txt

• Creates a unique job script for each job

• Submits each job script using qsub (with a maximum of 200 jobs allowed in parallel)

• Creates a batch output zip, including the necessary output files and logs (not the lammps.log or executable files for storage reasons)

• Deletes all other batch files

PLEASE NOTE that if for whatever reason the batch output zip is not created or otherwise not detectable, the daemon process will continue to check for the next 5 months. Should you wish to terminate the daemon process, use ps -ef | grep batch_submit.py and note the process ID (1st 5 digit number on line on Linux systems). Then use kill [process ID]. You could also shutdown your computer.

Troubleshooting

Make sure you’ve configued your username and hostname in config.csv correctly

Make sure you have an SSH key for the host and you can access it (test with commands like ping [hostname])

If you’re successfully transferring over batches, but they aren’t executing properly, you may need to do the following:

• Re-compile the executable on your host

• Replace your existing local copy

• Delete the BSS-Batch-Manager folder in your host’s home directory

• Re-run the program.