diff --git a/.gitignore b/.gitignore index 51b23b39..927e0aa0 100644 --- a/.gitignore +++ b/.gitignore @@ -1,8 +1,7 @@ .DS_Store -# RL training artifacts (training/ code is committed; outputs are not) +# RL training: checkpoints and logs ARE committed (training results must +# survive any single machine); only the env and scratch files are not. training/.venv/ -training/logs/ -training/checkpoints/ training/smoke_run.log training/__pycache__/ diff --git a/TRAINING_LINUX.md b/TRAINING_LINUX.md index bec0b53a..a5cd080b 100644 --- a/TRAINING_LINUX.md +++ b/TRAINING_LINUX.md @@ -1,44 +1,44 @@ # Training on the Linux / RTX 3090 box -Remote-training workflow: run long training sessions on the Linux machine, -watch the dashboard from any machine on the network, and ship the trained -model back to the Mac mini automatically when the run finishes. General -training concepts and the export/evaluate workflow live in -[TRAINING.md](TRAINING.md) — this doc is only what differs on the Linux box. +Remote-training workflow: run long training sessions on the Linux machine and +watch the dashboard from any machine on the network. **All training artifacts +(checkpoints, TensorBoard logs, exported bots) are committed to git** — no +result ever depends on a single machine, and moving models between the box +and the Mac is just `git pull`. General training concepts and the +export/evaluate workflow live in [TRAINING.md](TRAINING.md) — this doc is +only what differs on the Linux box. + +Both workflows below are wrapped in idempotent scripts in `training/` — +re-running either is always safe. ## One-time setup +GitHub auth first (git-over-HTTPS no longer accepts account passwords, so +clone over SSH — this key also lets `run_training.sh` push results): + ```bash -# GitHub auth (one-time): git-over-HTTPS no longer accepts account passwords, -# so clone over SSH. Generate a key, then add the printed public key at -# github.com/settings/keys → "New SSH key". ssh-keygen -t ed25519 # accept the defaults -cat ~/.ssh/id_ed25519.pub +cat ~/.ssh/id_ed25519.pub # add at github.com/settings/keys → "New SSH key" cd ~/ai-training git clone git@github.com:jcreek/CosmicClash.git # (submodules are editor tooling only — training doesn't need them) -# Godot 4.7.1 Linux binary -mkdir -p ~/ai-training/godot && cd ~/ai-training/godot -wget https://github.com/godotengine/godot/releases/download/4.7.1-stable/Godot_v4.7.1-stable_linux.x86_64.zip -unzip Godot_v4.7.1-stable_linux.x86_64.zip -echo 'export GODOT_BIN=~/ai-training/godot/Godot_v4.7.1-stable_linux.x86_64' >> ~/.bashrc && source ~/.bashrc - -# Python env -cd ~/ai-training/CosmicClash/training -python3 -m venv .venv -.venv/bin/pip install -r requirements.txt - -# CUDA sanity check — should print True -.venv/bin/python -c "import torch; print(torch.cuda.is_available())" - -# Headless smoke test — the game must boot without rendering -$GODOT_BIN --headless --path ../Game res://scenes/free_play.tscn --quit-after 300 +~/ai-training/CosmicClash/training/setup_linux.sh ``` -If the CUDA check prints `False`, reinstall torch from the CUDA index -(`pip install torch --index-url https://download.pytorch.org/whl/cu121`). +`setup_linux.sh` is safe to re-run any time (after a Godot upgrade, a broken +venv, a fresh clone — it checks each step before acting). It: + +- downloads the Godot 4.7.1 Linux binary to `~/ai-training/godot/` if missing + (override the location by exporting `GODOT_BIN`); +- creates `training/.venv` if missing and installs requirements; +- verifies CUDA torch, reinstalling from the CUDA wheel index if the box got + a CPU-only build; +- runs the Godot import pass (fresh clones have no `.godot/` cache, so + `class_name` scripts aren't registered until the project imports once); +- finishes with the headless smoke test — the game must boot without + rendering. ## Maximising throughput @@ -60,22 +60,27 @@ Tune by watching `time/fps`: run a 2-minute smoke run per setting and keep the best. Reference: 6 instances × speedup 16 ≈ 1,385 steps/s on an M4 Mac mini — a 20M-step run in ~4 h. Doubling fps halves that. -## Start a run +## Run training + +`run_training.sh [train.py args...]` wraps the whole cycle: +`git pull` → train → export the policy JSON → commit and push checkpoints, +logs, and the exported bot. Run it inside `tmux`/`screen` so an SSH +disconnect doesn't kill training. Ctrl-C is safe: the trainer writes +`final.zip` on the way out, and the script still exports, commits, and +pushes what it has. Fresh run: ```bash cd ~/ai-training/CosmicClash/training -.venv/bin/python train.py --experiment run03 --timesteps 20000000 \ - --n-parallel 14 --speedup 24 +./run_training.sh run03 --timesteps 20000000 --n-parallel 14 --speedup 24 ``` Resuming a previous policy (continues its timestep counter; `--timesteps` is *additional* steps). Lessons from run01/run02 hard-coded into flags: ```bash -.venv/bin/python train.py --experiment run03 --timesteps 20000000 \ - --n-parallel 14 --speedup 24 \ +./run_training.sh run03 --timesteps 20000000 --n-parallel 14 --speedup 24 \ --resume checkpoints/run02/final.zip --ent-coef 0.001 --reset-std 0.3 ``` @@ -85,32 +90,21 @@ Resuming a previous policy (continues its timestep counter; `--timesteps` is ~1.0 — check it 30–45 min in before committing to a long run. - `--reset-std` — on resume, restores exploration a collapsed checkpoint lost. -Run inside `tmux`/`screen` so an SSH disconnect doesn't kill training. -Ctrl-C is safe: `final.zip` is written on the way out. +## Results travel via git -## Auto-copy the result to the Mac mini when training finishes +`run_training.sh` commits and pushes everything a run produces: -One-time: enable **System Settings → General → Sharing → Remote Login** on -the Mac mini, and `ssh-copy-id jcreek@Joshs-Mac-mini.local` from the Linux -box so rsync runs unattended. +- `training/checkpoints//` — periodic checkpoints + `final.zip`, for + future `--resume`, evaluation, and difficulty tiers (an early checkpoint + *is* an easy bot); +- `training/logs/` — TensorBoard history; +- `Game/bots/.json` — the exported policy, immediately playable (point + Match or Spectate mode at `res://bots/.json`); +- `training/eval_history.json` — if evaluations ran. -Chain export + copy onto the training command (`;` not `&&`, so the copy -still happens after a Ctrl-C — `final.zip` exists either way): - -```bash -EXP=run03 -.venv/bin/python train.py --experiment $EXP --timesteps 20000000 --n-parallel 14 --speedup 24 ; \ -.venv/bin/python export_policy.py checkpoints/$EXP/final.zip ../Game/bots/$EXP.json && \ -rsync -av checkpoints/$EXP/final.zip \ - jcreek@Joshs-Mac-mini.local:~/Documents/repos/GitHub/CosmicClash/training/checkpoints/$EXP/ && \ -rsync -av ../Game/bots/$EXP.json \ - jcreek@Joshs-Mac-mini.local:~/Documents/repos/GitHub/CosmicClash/Game/bots/ -``` - -That lands both the raw checkpoint (for future `--resume` / evaluation on the -Mac) and the exported JSON policy (immediately playable — point Match or -Spectate mode at `res://bots/.json`). Add a third rsync of `logs/` if -you also want the TensorBoard history archived on the Mac. +On the Mac (or anywhere), collecting the results is just `git pull`. A +20M-step run adds roughly 40 MB of checkpoints — acceptable growth for the +guarantee that training is never lost with a machine. ## Dashboard over the network diff --git a/training/checkpoints/run01/final.zip b/training/checkpoints/run01/final.zip new file mode 100644 index 00000000..84157e19 Binary files /dev/null and b/training/checkpoints/run01/final.zip differ diff --git a/training/checkpoints/run01/ppo_10099596_steps.zip b/training/checkpoints/run01/ppo_10099596_steps.zip new file mode 100644 index 00000000..fa8634cb Binary files /dev/null and b/training/checkpoints/run01/ppo_10099596_steps.zip differ diff --git a/training/checkpoints/run01/ppo_10199592_steps.zip b/training/checkpoints/run01/ppo_10199592_steps.zip new file mode 100644 index 00000000..db626688 Binary files /dev/null and b/training/checkpoints/run01/ppo_10199592_steps.zip differ diff --git a/training/checkpoints/run01/ppo_10299588_steps.zip b/training/checkpoints/run01/ppo_10299588_steps.zip new file mode 100644 index 00000000..fb94ca87 Binary files /dev/null and b/training/checkpoints/run01/ppo_10299588_steps.zip differ diff --git a/training/checkpoints/run01/ppo_10399584_steps.zip b/training/checkpoints/run01/ppo_10399584_steps.zip new file mode 100644 index 00000000..0c1689c9 Binary files /dev/null and b/training/checkpoints/run01/ppo_10399584_steps.zip differ diff --git a/training/checkpoints/run01/ppo_10499580_steps.zip b/training/checkpoints/run01/ppo_10499580_steps.zip new file mode 100644 index 00000000..5d30024f Binary files /dev/null and b/training/checkpoints/run01/ppo_10499580_steps.zip differ diff --git a/training/checkpoints/run01/ppo_10599576_steps.zip b/training/checkpoints/run01/ppo_10599576_steps.zip new file mode 100644 index 00000000..8489b144 Binary files /dev/null and b/training/checkpoints/run01/ppo_10599576_steps.zip differ diff --git a/training/checkpoints/run01/ppo_10699572_steps.zip b/training/checkpoints/run01/ppo_10699572_steps.zip new file mode 100644 index 00000000..b2929992 Binary files /dev/null and b/training/checkpoints/run01/ppo_10699572_steps.zip differ diff --git a/training/checkpoints/run01/ppo_10799568_steps.zip b/training/checkpoints/run01/ppo_10799568_steps.zip new file mode 100644 index 00000000..51cd4705 Binary files /dev/null and b/training/checkpoints/run01/ppo_10799568_steps.zip differ diff --git a/training/checkpoints/run01/ppo_10899564_steps.zip b/training/checkpoints/run01/ppo_10899564_steps.zip new file mode 100644 index 00000000..7ca936b7 Binary files /dev/null and b/training/checkpoints/run01/ppo_10899564_steps.zip differ diff --git a/training/checkpoints/run01/ppo_10999560_steps.zip b/training/checkpoints/run01/ppo_10999560_steps.zip new file mode 100644 index 00000000..fb827769 Binary files /dev/null and b/training/checkpoints/run01/ppo_10999560_steps.zip differ diff --git a/training/checkpoints/run01/ppo_1099956_steps.zip b/training/checkpoints/run01/ppo_1099956_steps.zip new file mode 100644 index 00000000..8b68ff20 Binary files /dev/null and b/training/checkpoints/run01/ppo_1099956_steps.zip differ diff --git a/training/checkpoints/run01/ppo_11099556_steps.zip b/training/checkpoints/run01/ppo_11099556_steps.zip new file mode 100644 index 00000000..5ef70ef4 Binary files /dev/null and b/training/checkpoints/run01/ppo_11099556_steps.zip differ diff --git a/training/checkpoints/run01/ppo_11199552_steps.zip b/training/checkpoints/run01/ppo_11199552_steps.zip new file mode 100644 index 00000000..f0c33355 Binary files /dev/null and b/training/checkpoints/run01/ppo_11199552_steps.zip differ diff --git a/training/checkpoints/run01/ppo_11299548_steps.zip b/training/checkpoints/run01/ppo_11299548_steps.zip new file mode 100644 index 00000000..f1661803 Binary files /dev/null and b/training/checkpoints/run01/ppo_11299548_steps.zip differ diff --git a/training/checkpoints/run01/ppo_11399544_steps.zip b/training/checkpoints/run01/ppo_11399544_steps.zip new file mode 100644 index 00000000..45063f84 Binary files /dev/null and b/training/checkpoints/run01/ppo_11399544_steps.zip differ diff --git a/training/checkpoints/run01/ppo_11499540_steps.zip b/training/checkpoints/run01/ppo_11499540_steps.zip new file mode 100644 index 00000000..8f30cab8 Binary files /dev/null and b/training/checkpoints/run01/ppo_11499540_steps.zip differ diff --git a/training/checkpoints/run01/ppo_11599536_steps.zip b/training/checkpoints/run01/ppo_11599536_steps.zip new file mode 100644 index 00000000..dfc4d71b Binary files /dev/null and b/training/checkpoints/run01/ppo_11599536_steps.zip differ diff --git a/training/checkpoints/run01/ppo_11699532_steps.zip b/training/checkpoints/run01/ppo_11699532_steps.zip new file mode 100644 index 00000000..59250042 Binary files /dev/null and b/training/checkpoints/run01/ppo_11699532_steps.zip differ diff --git a/training/checkpoints/run01/ppo_11799528_steps.zip b/training/checkpoints/run01/ppo_11799528_steps.zip new file mode 100644 index 00000000..fd853c51 Binary files /dev/null and b/training/checkpoints/run01/ppo_11799528_steps.zip differ diff --git a/training/checkpoints/run01/ppo_11899524_steps.zip b/training/checkpoints/run01/ppo_11899524_steps.zip new file mode 100644 index 00000000..6d761611 Binary files /dev/null and b/training/checkpoints/run01/ppo_11899524_steps.zip differ diff --git a/training/checkpoints/run01/ppo_11999520_steps.zip b/training/checkpoints/run01/ppo_11999520_steps.zip new file mode 100644 index 00000000..dc4278c7 Binary files /dev/null and b/training/checkpoints/run01/ppo_11999520_steps.zip differ diff --git a/training/checkpoints/run01/ppo_1199952_steps.zip b/training/checkpoints/run01/ppo_1199952_steps.zip new file mode 100644 index 00000000..3646a6dd Binary files /dev/null and b/training/checkpoints/run01/ppo_1199952_steps.zip differ diff --git a/training/checkpoints/run01/ppo_12099516_steps.zip b/training/checkpoints/run01/ppo_12099516_steps.zip new file mode 100644 index 00000000..f88966bb Binary files /dev/null and b/training/checkpoints/run01/ppo_12099516_steps.zip differ diff --git a/training/checkpoints/run01/ppo_12199512_steps.zip b/training/checkpoints/run01/ppo_12199512_steps.zip new file mode 100644 index 00000000..1a63bd67 Binary files /dev/null and b/training/checkpoints/run01/ppo_12199512_steps.zip differ diff --git a/training/checkpoints/run01/ppo_12299508_steps.zip b/training/checkpoints/run01/ppo_12299508_steps.zip new file mode 100644 index 00000000..8a1c976e Binary files /dev/null and b/training/checkpoints/run01/ppo_12299508_steps.zip differ diff --git a/training/checkpoints/run01/ppo_12399504_steps.zip b/training/checkpoints/run01/ppo_12399504_steps.zip new file mode 100644 index 00000000..9066d1f7 Binary files /dev/null and b/training/checkpoints/run01/ppo_12399504_steps.zip differ diff --git a/training/checkpoints/run01/ppo_12499500_steps.zip b/training/checkpoints/run01/ppo_12499500_steps.zip new file mode 100644 index 00000000..33b95b34 Binary files /dev/null and b/training/checkpoints/run01/ppo_12499500_steps.zip differ diff --git a/training/checkpoints/run01/ppo_12599496_steps.zip b/training/checkpoints/run01/ppo_12599496_steps.zip new file mode 100644 index 00000000..8f4b2ba6 Binary files /dev/null and b/training/checkpoints/run01/ppo_12599496_steps.zip differ diff --git a/training/checkpoints/run01/ppo_12699492_steps.zip b/training/checkpoints/run01/ppo_12699492_steps.zip new file mode 100644 index 00000000..dc3fd43d Binary files /dev/null and b/training/checkpoints/run01/ppo_12699492_steps.zip differ diff --git a/training/checkpoints/run01/ppo_12799488_steps.zip b/training/checkpoints/run01/ppo_12799488_steps.zip new file mode 100644 index 00000000..96d68f86 Binary files /dev/null and b/training/checkpoints/run01/ppo_12799488_steps.zip differ diff --git a/training/checkpoints/run01/ppo_12899484_steps.zip b/training/checkpoints/run01/ppo_12899484_steps.zip new file mode 100644 index 00000000..c066f9cf Binary files /dev/null and b/training/checkpoints/run01/ppo_12899484_steps.zip differ diff --git a/training/checkpoints/run01/ppo_12999480_steps.zip b/training/checkpoints/run01/ppo_12999480_steps.zip new file mode 100644 index 00000000..34a6e497 Binary files /dev/null and b/training/checkpoints/run01/ppo_12999480_steps.zip differ diff --git a/training/checkpoints/run01/ppo_1299948_steps.zip b/training/checkpoints/run01/ppo_1299948_steps.zip new file mode 100644 index 00000000..651436a9 Binary files /dev/null and b/training/checkpoints/run01/ppo_1299948_steps.zip differ diff --git a/training/checkpoints/run01/ppo_13099476_steps.zip b/training/checkpoints/run01/ppo_13099476_steps.zip new file mode 100644 index 00000000..fba16bd1 Binary files /dev/null and b/training/checkpoints/run01/ppo_13099476_steps.zip differ diff --git a/training/checkpoints/run01/ppo_13199472_steps.zip b/training/checkpoints/run01/ppo_13199472_steps.zip new file mode 100644 index 00000000..481dd30c Binary files /dev/null and b/training/checkpoints/run01/ppo_13199472_steps.zip differ diff --git a/training/checkpoints/run01/ppo_13299468_steps.zip b/training/checkpoints/run01/ppo_13299468_steps.zip new file mode 100644 index 00000000..f0d99eb2 Binary files /dev/null and b/training/checkpoints/run01/ppo_13299468_steps.zip differ diff --git a/training/checkpoints/run01/ppo_13399464_steps.zip b/training/checkpoints/run01/ppo_13399464_steps.zip new file mode 100644 index 00000000..f82ec99d Binary files /dev/null and b/training/checkpoints/run01/ppo_13399464_steps.zip differ diff --git a/training/checkpoints/run01/ppo_13499460_steps.zip b/training/checkpoints/run01/ppo_13499460_steps.zip new file mode 100644 index 00000000..c5195ff1 Binary files /dev/null and b/training/checkpoints/run01/ppo_13499460_steps.zip differ diff --git a/training/checkpoints/run01/ppo_13599456_steps.zip b/training/checkpoints/run01/ppo_13599456_steps.zip new file mode 100644 index 00000000..cd1a6519 Binary files /dev/null and b/training/checkpoints/run01/ppo_13599456_steps.zip differ diff --git a/training/checkpoints/run01/ppo_13699452_steps.zip b/training/checkpoints/run01/ppo_13699452_steps.zip new file mode 100644 index 00000000..7d74fcee Binary files /dev/null and b/training/checkpoints/run01/ppo_13699452_steps.zip differ diff --git a/training/checkpoints/run01/ppo_13799448_steps.zip b/training/checkpoints/run01/ppo_13799448_steps.zip new file mode 100644 index 00000000..49abb7a9 Binary files /dev/null and b/training/checkpoints/run01/ppo_13799448_steps.zip differ diff --git a/training/checkpoints/run01/ppo_13899444_steps.zip b/training/checkpoints/run01/ppo_13899444_steps.zip new file mode 100644 index 00000000..744f9e3f Binary files /dev/null and b/training/checkpoints/run01/ppo_13899444_steps.zip differ diff --git a/training/checkpoints/run01/ppo_13999440_steps.zip b/training/checkpoints/run01/ppo_13999440_steps.zip new file mode 100644 index 00000000..1d328f34 Binary files /dev/null and b/training/checkpoints/run01/ppo_13999440_steps.zip differ diff --git a/training/checkpoints/run01/ppo_1399944_steps.zip b/training/checkpoints/run01/ppo_1399944_steps.zip new file mode 100644 index 00000000..c89b7e40 Binary files /dev/null and b/training/checkpoints/run01/ppo_1399944_steps.zip differ diff --git a/training/checkpoints/run01/ppo_14099436_steps.zip b/training/checkpoints/run01/ppo_14099436_steps.zip new file mode 100644 index 00000000..332d0399 Binary files /dev/null and b/training/checkpoints/run01/ppo_14099436_steps.zip differ diff --git a/training/checkpoints/run01/ppo_14199432_steps.zip b/training/checkpoints/run01/ppo_14199432_steps.zip new file mode 100644 index 00000000..a578ca4b Binary files /dev/null and b/training/checkpoints/run01/ppo_14199432_steps.zip differ diff --git a/training/checkpoints/run01/ppo_14299428_steps.zip b/training/checkpoints/run01/ppo_14299428_steps.zip new file mode 100644 index 00000000..7b400f41 Binary files /dev/null and b/training/checkpoints/run01/ppo_14299428_steps.zip differ diff --git a/training/checkpoints/run01/ppo_14399424_steps.zip b/training/checkpoints/run01/ppo_14399424_steps.zip new file mode 100644 index 00000000..596b3323 Binary files /dev/null and b/training/checkpoints/run01/ppo_14399424_steps.zip differ diff --git a/training/checkpoints/run01/ppo_14499420_steps.zip b/training/checkpoints/run01/ppo_14499420_steps.zip new file mode 100644 index 00000000..66dde8de Binary files /dev/null and b/training/checkpoints/run01/ppo_14499420_steps.zip differ diff --git a/training/checkpoints/run01/ppo_14599416_steps.zip b/training/checkpoints/run01/ppo_14599416_steps.zip new file mode 100644 index 00000000..266cc990 Binary files /dev/null and b/training/checkpoints/run01/ppo_14599416_steps.zip differ diff --git a/training/checkpoints/run01/ppo_14699412_steps.zip b/training/checkpoints/run01/ppo_14699412_steps.zip new file mode 100644 index 00000000..74d0e457 Binary files /dev/null and b/training/checkpoints/run01/ppo_14699412_steps.zip differ diff --git a/training/checkpoints/run01/ppo_14799408_steps.zip b/training/checkpoints/run01/ppo_14799408_steps.zip new file mode 100644 index 00000000..2ea849d4 Binary files /dev/null and b/training/checkpoints/run01/ppo_14799408_steps.zip differ diff --git a/training/checkpoints/run01/ppo_14899404_steps.zip b/training/checkpoints/run01/ppo_14899404_steps.zip new file mode 100644 index 00000000..8afc4849 Binary files /dev/null and b/training/checkpoints/run01/ppo_14899404_steps.zip differ diff --git a/training/checkpoints/run01/ppo_14999400_steps.zip b/training/checkpoints/run01/ppo_14999400_steps.zip new file mode 100644 index 00000000..ee625382 Binary files /dev/null and b/training/checkpoints/run01/ppo_14999400_steps.zip differ diff --git a/training/checkpoints/run01/ppo_1499940_steps.zip b/training/checkpoints/run01/ppo_1499940_steps.zip new file mode 100644 index 00000000..d9b8375f Binary files /dev/null and b/training/checkpoints/run01/ppo_1499940_steps.zip differ diff --git a/training/checkpoints/run01/ppo_15099396_steps.zip b/training/checkpoints/run01/ppo_15099396_steps.zip new file mode 100644 index 00000000..2ec71e11 Binary files /dev/null and b/training/checkpoints/run01/ppo_15099396_steps.zip differ diff --git a/training/checkpoints/run01/ppo_15199392_steps.zip b/training/checkpoints/run01/ppo_15199392_steps.zip new file mode 100644 index 00000000..dcb13e09 Binary files /dev/null and b/training/checkpoints/run01/ppo_15199392_steps.zip differ diff --git a/training/checkpoints/run01/ppo_15299388_steps.zip b/training/checkpoints/run01/ppo_15299388_steps.zip new file mode 100644 index 00000000..eaad50df Binary files /dev/null and b/training/checkpoints/run01/ppo_15299388_steps.zip differ diff --git a/training/checkpoints/run01/ppo_15399384_steps.zip b/training/checkpoints/run01/ppo_15399384_steps.zip new file mode 100644 index 00000000..c061e08b Binary files /dev/null and b/training/checkpoints/run01/ppo_15399384_steps.zip differ diff --git a/training/checkpoints/run01/ppo_15499380_steps.zip b/training/checkpoints/run01/ppo_15499380_steps.zip new file mode 100644 index 00000000..a50f6c49 Binary files /dev/null and b/training/checkpoints/run01/ppo_15499380_steps.zip differ diff --git a/training/checkpoints/run01/ppo_15599376_steps.zip b/training/checkpoints/run01/ppo_15599376_steps.zip new file mode 100644 index 00000000..525f585b Binary files /dev/null and b/training/checkpoints/run01/ppo_15599376_steps.zip differ diff --git a/training/checkpoints/run01/ppo_15699372_steps.zip b/training/checkpoints/run01/ppo_15699372_steps.zip new file mode 100644 index 00000000..5f704dcf Binary files /dev/null and b/training/checkpoints/run01/ppo_15699372_steps.zip differ diff --git a/training/checkpoints/run01/ppo_15799368_steps.zip b/training/checkpoints/run01/ppo_15799368_steps.zip new file mode 100644 index 00000000..5881eec3 Binary files /dev/null and b/training/checkpoints/run01/ppo_15799368_steps.zip differ diff --git a/training/checkpoints/run01/ppo_15899364_steps.zip b/training/checkpoints/run01/ppo_15899364_steps.zip new file mode 100644 index 00000000..d35dae30 Binary files /dev/null and b/training/checkpoints/run01/ppo_15899364_steps.zip differ diff --git a/training/checkpoints/run01/ppo_15999360_steps.zip b/training/checkpoints/run01/ppo_15999360_steps.zip new file mode 100644 index 00000000..43e49566 Binary files /dev/null and b/training/checkpoints/run01/ppo_15999360_steps.zip differ diff --git a/training/checkpoints/run01/ppo_1599936_steps.zip b/training/checkpoints/run01/ppo_1599936_steps.zip new file mode 100644 index 00000000..f56e1c74 Binary files /dev/null and b/training/checkpoints/run01/ppo_1599936_steps.zip differ diff --git a/training/checkpoints/run01/ppo_16099356_steps.zip b/training/checkpoints/run01/ppo_16099356_steps.zip new file mode 100644 index 00000000..5d07e63f Binary files /dev/null and b/training/checkpoints/run01/ppo_16099356_steps.zip differ diff --git a/training/checkpoints/run01/ppo_16199352_steps.zip b/training/checkpoints/run01/ppo_16199352_steps.zip new file mode 100644 index 00000000..62ff6e74 Binary files /dev/null and b/training/checkpoints/run01/ppo_16199352_steps.zip differ diff --git a/training/checkpoints/run01/ppo_16299348_steps.zip b/training/checkpoints/run01/ppo_16299348_steps.zip new file mode 100644 index 00000000..4eb4c32c Binary files /dev/null and b/training/checkpoints/run01/ppo_16299348_steps.zip differ diff --git a/training/checkpoints/run01/ppo_16399344_steps.zip b/training/checkpoints/run01/ppo_16399344_steps.zip new file mode 100644 index 00000000..453ae1f3 Binary files /dev/null and b/training/checkpoints/run01/ppo_16399344_steps.zip differ diff --git a/training/checkpoints/run01/ppo_16499340_steps.zip b/training/checkpoints/run01/ppo_16499340_steps.zip new file mode 100644 index 00000000..87d13530 Binary files /dev/null and b/training/checkpoints/run01/ppo_16499340_steps.zip differ diff --git a/training/checkpoints/run01/ppo_16599336_steps.zip b/training/checkpoints/run01/ppo_16599336_steps.zip new file mode 100644 index 00000000..6bc759d1 Binary files /dev/null and b/training/checkpoints/run01/ppo_16599336_steps.zip differ diff --git a/training/checkpoints/run01/ppo_16699332_steps.zip b/training/checkpoints/run01/ppo_16699332_steps.zip new file mode 100644 index 00000000..1816fd99 Binary files /dev/null and b/training/checkpoints/run01/ppo_16699332_steps.zip differ diff --git a/training/checkpoints/run01/ppo_16799328_steps.zip b/training/checkpoints/run01/ppo_16799328_steps.zip new file mode 100644 index 00000000..ef47e1b0 Binary files /dev/null and b/training/checkpoints/run01/ppo_16799328_steps.zip differ diff --git a/training/checkpoints/run01/ppo_16899324_steps.zip b/training/checkpoints/run01/ppo_16899324_steps.zip new file mode 100644 index 00000000..83f3a903 Binary files /dev/null and b/training/checkpoints/run01/ppo_16899324_steps.zip differ diff --git a/training/checkpoints/run01/ppo_16999320_steps.zip b/training/checkpoints/run01/ppo_16999320_steps.zip new file mode 100644 index 00000000..c3ac93fd Binary files /dev/null and b/training/checkpoints/run01/ppo_16999320_steps.zip differ diff --git a/training/checkpoints/run01/ppo_1699932_steps.zip b/training/checkpoints/run01/ppo_1699932_steps.zip new file mode 100644 index 00000000..8a96dedf Binary files /dev/null and b/training/checkpoints/run01/ppo_1699932_steps.zip differ diff --git a/training/checkpoints/run01/ppo_17099316_steps.zip b/training/checkpoints/run01/ppo_17099316_steps.zip new file mode 100644 index 00000000..3b94b24e Binary files /dev/null and b/training/checkpoints/run01/ppo_17099316_steps.zip differ diff --git a/training/checkpoints/run01/ppo_17199312_steps.zip b/training/checkpoints/run01/ppo_17199312_steps.zip new file mode 100644 index 00000000..b6a8c4a7 Binary files /dev/null and b/training/checkpoints/run01/ppo_17199312_steps.zip differ diff --git a/training/checkpoints/run01/ppo_17299308_steps.zip b/training/checkpoints/run01/ppo_17299308_steps.zip new file mode 100644 index 00000000..4673b2e9 Binary files /dev/null and b/training/checkpoints/run01/ppo_17299308_steps.zip differ diff --git a/training/checkpoints/run01/ppo_17399304_steps.zip b/training/checkpoints/run01/ppo_17399304_steps.zip new file mode 100644 index 00000000..117f0c70 Binary files /dev/null and b/training/checkpoints/run01/ppo_17399304_steps.zip differ diff --git a/training/checkpoints/run01/ppo_17499300_steps.zip b/training/checkpoints/run01/ppo_17499300_steps.zip new file mode 100644 index 00000000..74d3ebf8 Binary files /dev/null and b/training/checkpoints/run01/ppo_17499300_steps.zip differ diff --git a/training/checkpoints/run01/ppo_17599296_steps.zip b/training/checkpoints/run01/ppo_17599296_steps.zip new file mode 100644 index 00000000..62805c18 Binary files /dev/null and b/training/checkpoints/run01/ppo_17599296_steps.zip differ diff --git a/training/checkpoints/run01/ppo_17699292_steps.zip b/training/checkpoints/run01/ppo_17699292_steps.zip new file mode 100644 index 00000000..5bde589a Binary files /dev/null and b/training/checkpoints/run01/ppo_17699292_steps.zip differ diff --git a/training/checkpoints/run01/ppo_17799288_steps.zip b/training/checkpoints/run01/ppo_17799288_steps.zip new file mode 100644 index 00000000..4311cd13 Binary files /dev/null and b/training/checkpoints/run01/ppo_17799288_steps.zip differ diff --git a/training/checkpoints/run01/ppo_17899284_steps.zip b/training/checkpoints/run01/ppo_17899284_steps.zip new file mode 100644 index 00000000..092f67d4 Binary files /dev/null and b/training/checkpoints/run01/ppo_17899284_steps.zip differ diff --git a/training/checkpoints/run01/ppo_17999280_steps.zip b/training/checkpoints/run01/ppo_17999280_steps.zip new file mode 100644 index 00000000..f1a42432 Binary files /dev/null and b/training/checkpoints/run01/ppo_17999280_steps.zip differ diff --git a/training/checkpoints/run01/ppo_1799928_steps.zip b/training/checkpoints/run01/ppo_1799928_steps.zip new file mode 100644 index 00000000..bdd91f23 Binary files /dev/null and b/training/checkpoints/run01/ppo_1799928_steps.zip differ diff --git a/training/checkpoints/run01/ppo_18099276_steps.zip b/training/checkpoints/run01/ppo_18099276_steps.zip new file mode 100644 index 00000000..b654b27e Binary files /dev/null and b/training/checkpoints/run01/ppo_18099276_steps.zip differ diff --git a/training/checkpoints/run01/ppo_18199272_steps.zip b/training/checkpoints/run01/ppo_18199272_steps.zip new file mode 100644 index 00000000..39c9eeed Binary files /dev/null and b/training/checkpoints/run01/ppo_18199272_steps.zip differ diff --git a/training/checkpoints/run01/ppo_18299268_steps.zip b/training/checkpoints/run01/ppo_18299268_steps.zip new file mode 100644 index 00000000..5eb4ca14 Binary files /dev/null and b/training/checkpoints/run01/ppo_18299268_steps.zip differ diff --git a/training/checkpoints/run01/ppo_18399264_steps.zip b/training/checkpoints/run01/ppo_18399264_steps.zip new file mode 100644 index 00000000..957594ad Binary files /dev/null and b/training/checkpoints/run01/ppo_18399264_steps.zip differ diff --git a/training/checkpoints/run01/ppo_18499260_steps.zip b/training/checkpoints/run01/ppo_18499260_steps.zip new file mode 100644 index 00000000..94ac3034 Binary files /dev/null and b/training/checkpoints/run01/ppo_18499260_steps.zip differ diff --git a/training/checkpoints/run01/ppo_18599256_steps.zip b/training/checkpoints/run01/ppo_18599256_steps.zip new file mode 100644 index 00000000..c1b0d8b5 Binary files /dev/null and b/training/checkpoints/run01/ppo_18599256_steps.zip differ diff --git a/training/checkpoints/run01/ppo_18699252_steps.zip b/training/checkpoints/run01/ppo_18699252_steps.zip new file mode 100644 index 00000000..f0d923f9 Binary files /dev/null and b/training/checkpoints/run01/ppo_18699252_steps.zip differ diff --git a/training/checkpoints/run01/ppo_18799248_steps.zip b/training/checkpoints/run01/ppo_18799248_steps.zip new file mode 100644 index 00000000..ac9d2def Binary files /dev/null and b/training/checkpoints/run01/ppo_18799248_steps.zip differ diff --git a/training/checkpoints/run01/ppo_18899244_steps.zip b/training/checkpoints/run01/ppo_18899244_steps.zip new file mode 100644 index 00000000..9e83f875 Binary files /dev/null and b/training/checkpoints/run01/ppo_18899244_steps.zip differ diff --git a/training/checkpoints/run01/ppo_18999240_steps.zip b/training/checkpoints/run01/ppo_18999240_steps.zip new file mode 100644 index 00000000..57a9dbf3 Binary files /dev/null and b/training/checkpoints/run01/ppo_18999240_steps.zip differ diff --git a/training/checkpoints/run01/ppo_1899924_steps.zip b/training/checkpoints/run01/ppo_1899924_steps.zip new file mode 100644 index 00000000..a553680b Binary files /dev/null and b/training/checkpoints/run01/ppo_1899924_steps.zip differ diff --git a/training/checkpoints/run01/ppo_19099236_steps.zip b/training/checkpoints/run01/ppo_19099236_steps.zip new file mode 100644 index 00000000..0c0f0b31 Binary files /dev/null and b/training/checkpoints/run01/ppo_19099236_steps.zip differ diff --git a/training/checkpoints/run01/ppo_19199232_steps.zip b/training/checkpoints/run01/ppo_19199232_steps.zip new file mode 100644 index 00000000..dfa03b79 Binary files /dev/null and b/training/checkpoints/run01/ppo_19199232_steps.zip differ diff --git a/training/checkpoints/run01/ppo_19299228_steps.zip b/training/checkpoints/run01/ppo_19299228_steps.zip new file mode 100644 index 00000000..a635f2dc Binary files /dev/null and b/training/checkpoints/run01/ppo_19299228_steps.zip differ diff --git a/training/checkpoints/run01/ppo_19399224_steps.zip b/training/checkpoints/run01/ppo_19399224_steps.zip new file mode 100644 index 00000000..dd9dfc35 Binary files /dev/null and b/training/checkpoints/run01/ppo_19399224_steps.zip differ diff --git a/training/checkpoints/run01/ppo_19499220_steps.zip b/training/checkpoints/run01/ppo_19499220_steps.zip new file mode 100644 index 00000000..7ea99781 Binary files /dev/null and b/training/checkpoints/run01/ppo_19499220_steps.zip differ diff --git a/training/checkpoints/run01/ppo_19599216_steps.zip b/training/checkpoints/run01/ppo_19599216_steps.zip new file mode 100644 index 00000000..d3796f64 Binary files /dev/null and b/training/checkpoints/run01/ppo_19599216_steps.zip differ diff --git a/training/checkpoints/run01/ppo_19699212_steps.zip b/training/checkpoints/run01/ppo_19699212_steps.zip new file mode 100644 index 00000000..8140e3d3 Binary files /dev/null and b/training/checkpoints/run01/ppo_19699212_steps.zip differ diff --git a/training/checkpoints/run01/ppo_19799208_steps.zip b/training/checkpoints/run01/ppo_19799208_steps.zip new file mode 100644 index 00000000..70e22419 Binary files /dev/null and b/training/checkpoints/run01/ppo_19799208_steps.zip differ diff --git a/training/checkpoints/run01/ppo_19899204_steps.zip b/training/checkpoints/run01/ppo_19899204_steps.zip new file mode 100644 index 00000000..8b8006b7 Binary files /dev/null and b/training/checkpoints/run01/ppo_19899204_steps.zip differ diff --git a/training/checkpoints/run01/ppo_19999200_steps.zip b/training/checkpoints/run01/ppo_19999200_steps.zip new file mode 100644 index 00000000..82f0e0d5 Binary files /dev/null and b/training/checkpoints/run01/ppo_19999200_steps.zip differ diff --git a/training/checkpoints/run01/ppo_1999920_steps.zip b/training/checkpoints/run01/ppo_1999920_steps.zip new file mode 100644 index 00000000..e656c039 Binary files /dev/null and b/training/checkpoints/run01/ppo_1999920_steps.zip differ diff --git a/training/checkpoints/run01/ppo_199992_steps.zip b/training/checkpoints/run01/ppo_199992_steps.zip new file mode 100644 index 00000000..069828be Binary files /dev/null and b/training/checkpoints/run01/ppo_199992_steps.zip differ diff --git a/training/checkpoints/run01/ppo_2099916_steps.zip b/training/checkpoints/run01/ppo_2099916_steps.zip new file mode 100644 index 00000000..eba2d9cd Binary files /dev/null and b/training/checkpoints/run01/ppo_2099916_steps.zip differ diff --git a/training/checkpoints/run01/ppo_2199912_steps.zip b/training/checkpoints/run01/ppo_2199912_steps.zip new file mode 100644 index 00000000..420a5a7e Binary files /dev/null and b/training/checkpoints/run01/ppo_2199912_steps.zip differ diff --git a/training/checkpoints/run01/ppo_2299908_steps.zip b/training/checkpoints/run01/ppo_2299908_steps.zip new file mode 100644 index 00000000..a3dbb0e8 Binary files /dev/null and b/training/checkpoints/run01/ppo_2299908_steps.zip differ diff --git a/training/checkpoints/run01/ppo_2399904_steps.zip b/training/checkpoints/run01/ppo_2399904_steps.zip new file mode 100644 index 00000000..aca3bc39 Binary files /dev/null and b/training/checkpoints/run01/ppo_2399904_steps.zip differ diff --git a/training/checkpoints/run01/ppo_2499900_steps.zip b/training/checkpoints/run01/ppo_2499900_steps.zip new file mode 100644 index 00000000..a0841f18 Binary files /dev/null and b/training/checkpoints/run01/ppo_2499900_steps.zip differ diff --git a/training/checkpoints/run01/ppo_2599896_steps.zip b/training/checkpoints/run01/ppo_2599896_steps.zip new file mode 100644 index 00000000..425ec37e Binary files /dev/null and b/training/checkpoints/run01/ppo_2599896_steps.zip differ diff --git a/training/checkpoints/run01/ppo_2699892_steps.zip b/training/checkpoints/run01/ppo_2699892_steps.zip new file mode 100644 index 00000000..9771fbbc Binary files /dev/null and b/training/checkpoints/run01/ppo_2699892_steps.zip differ diff --git a/training/checkpoints/run01/ppo_2799888_steps.zip b/training/checkpoints/run01/ppo_2799888_steps.zip new file mode 100644 index 00000000..ef9e502c Binary files /dev/null and b/training/checkpoints/run01/ppo_2799888_steps.zip differ diff --git a/training/checkpoints/run01/ppo_2899884_steps.zip b/training/checkpoints/run01/ppo_2899884_steps.zip new file mode 100644 index 00000000..ec97eb08 Binary files /dev/null and b/training/checkpoints/run01/ppo_2899884_steps.zip differ diff --git a/training/checkpoints/run01/ppo_2999880_steps.zip b/training/checkpoints/run01/ppo_2999880_steps.zip new file mode 100644 index 00000000..c147a57f Binary files /dev/null and b/training/checkpoints/run01/ppo_2999880_steps.zip differ diff --git a/training/checkpoints/run01/ppo_299988_steps.zip b/training/checkpoints/run01/ppo_299988_steps.zip new file mode 100644 index 00000000..a12f7271 Binary files /dev/null and b/training/checkpoints/run01/ppo_299988_steps.zip differ diff --git a/training/checkpoints/run01/ppo_3099876_steps.zip b/training/checkpoints/run01/ppo_3099876_steps.zip new file mode 100644 index 00000000..d953132b Binary files /dev/null and b/training/checkpoints/run01/ppo_3099876_steps.zip differ diff --git a/training/checkpoints/run01/ppo_3199872_steps.zip b/training/checkpoints/run01/ppo_3199872_steps.zip new file mode 100644 index 00000000..ec9513b8 Binary files /dev/null and b/training/checkpoints/run01/ppo_3199872_steps.zip differ diff --git a/training/checkpoints/run01/ppo_3299868_steps.zip b/training/checkpoints/run01/ppo_3299868_steps.zip new file mode 100644 index 00000000..c6594dc7 Binary files /dev/null and b/training/checkpoints/run01/ppo_3299868_steps.zip differ diff --git a/training/checkpoints/run01/ppo_3399864_steps.zip b/training/checkpoints/run01/ppo_3399864_steps.zip new file mode 100644 index 00000000..23e415f8 Binary files /dev/null and b/training/checkpoints/run01/ppo_3399864_steps.zip differ diff --git a/training/checkpoints/run01/ppo_3499860_steps.zip b/training/checkpoints/run01/ppo_3499860_steps.zip new file mode 100644 index 00000000..0c2b3aa3 Binary files /dev/null and b/training/checkpoints/run01/ppo_3499860_steps.zip differ diff --git a/training/checkpoints/run01/ppo_3599856_steps.zip b/training/checkpoints/run01/ppo_3599856_steps.zip new file mode 100644 index 00000000..c4b1485e Binary files /dev/null and b/training/checkpoints/run01/ppo_3599856_steps.zip differ diff --git a/training/checkpoints/run01/ppo_3699852_steps.zip b/training/checkpoints/run01/ppo_3699852_steps.zip new file mode 100644 index 00000000..219052be Binary files /dev/null and b/training/checkpoints/run01/ppo_3699852_steps.zip differ diff --git a/training/checkpoints/run01/ppo_3799848_steps.zip b/training/checkpoints/run01/ppo_3799848_steps.zip new file mode 100644 index 00000000..46cf398f Binary files /dev/null and b/training/checkpoints/run01/ppo_3799848_steps.zip differ diff --git a/training/checkpoints/run01/ppo_3899844_steps.zip b/training/checkpoints/run01/ppo_3899844_steps.zip new file mode 100644 index 00000000..92f8d5f5 Binary files /dev/null and b/training/checkpoints/run01/ppo_3899844_steps.zip differ diff --git a/training/checkpoints/run01/ppo_3999840_steps.zip b/training/checkpoints/run01/ppo_3999840_steps.zip new file mode 100644 index 00000000..05fcc825 Binary files /dev/null and b/training/checkpoints/run01/ppo_3999840_steps.zip differ diff --git a/training/checkpoints/run01/ppo_399984_steps.zip b/training/checkpoints/run01/ppo_399984_steps.zip new file mode 100644 index 00000000..de416056 Binary files /dev/null and b/training/checkpoints/run01/ppo_399984_steps.zip differ diff --git a/training/checkpoints/run01/ppo_4099836_steps.zip b/training/checkpoints/run01/ppo_4099836_steps.zip new file mode 100644 index 00000000..20e6becf Binary files /dev/null and b/training/checkpoints/run01/ppo_4099836_steps.zip differ diff --git a/training/checkpoints/run01/ppo_4199832_steps.zip b/training/checkpoints/run01/ppo_4199832_steps.zip new file mode 100644 index 00000000..9e4ab1af Binary files /dev/null and b/training/checkpoints/run01/ppo_4199832_steps.zip differ diff --git a/training/checkpoints/run01/ppo_4299828_steps.zip b/training/checkpoints/run01/ppo_4299828_steps.zip new file mode 100644 index 00000000..a32bbf9b Binary files /dev/null and b/training/checkpoints/run01/ppo_4299828_steps.zip differ diff --git a/training/checkpoints/run01/ppo_4399824_steps.zip b/training/checkpoints/run01/ppo_4399824_steps.zip new file mode 100644 index 00000000..c3ac2479 Binary files /dev/null and b/training/checkpoints/run01/ppo_4399824_steps.zip differ diff --git a/training/checkpoints/run01/ppo_4499820_steps.zip b/training/checkpoints/run01/ppo_4499820_steps.zip new file mode 100644 index 00000000..b5a25b2a Binary files /dev/null and b/training/checkpoints/run01/ppo_4499820_steps.zip differ diff --git a/training/checkpoints/run01/ppo_4599816_steps.zip b/training/checkpoints/run01/ppo_4599816_steps.zip new file mode 100644 index 00000000..0ba14e8d Binary files /dev/null and b/training/checkpoints/run01/ppo_4599816_steps.zip differ diff --git a/training/checkpoints/run01/ppo_4699812_steps.zip b/training/checkpoints/run01/ppo_4699812_steps.zip new file mode 100644 index 00000000..b0cf89f4 Binary files /dev/null and b/training/checkpoints/run01/ppo_4699812_steps.zip differ diff --git a/training/checkpoints/run01/ppo_4799808_steps.zip b/training/checkpoints/run01/ppo_4799808_steps.zip new file mode 100644 index 00000000..400fdd5f Binary files /dev/null and b/training/checkpoints/run01/ppo_4799808_steps.zip differ diff --git a/training/checkpoints/run01/ppo_4899804_steps.zip b/training/checkpoints/run01/ppo_4899804_steps.zip new file mode 100644 index 00000000..522c7c25 Binary files /dev/null and b/training/checkpoints/run01/ppo_4899804_steps.zip differ diff --git a/training/checkpoints/run01/ppo_4999800_steps.zip b/training/checkpoints/run01/ppo_4999800_steps.zip new file mode 100644 index 00000000..b5f484aa Binary files /dev/null and b/training/checkpoints/run01/ppo_4999800_steps.zip differ diff --git a/training/checkpoints/run01/ppo_499980_steps.zip b/training/checkpoints/run01/ppo_499980_steps.zip new file mode 100644 index 00000000..2ffff7af Binary files /dev/null and b/training/checkpoints/run01/ppo_499980_steps.zip differ diff --git a/training/checkpoints/run01/ppo_5099796_steps.zip b/training/checkpoints/run01/ppo_5099796_steps.zip new file mode 100644 index 00000000..ae17a14a Binary files /dev/null and b/training/checkpoints/run01/ppo_5099796_steps.zip differ diff --git a/training/checkpoints/run01/ppo_5199792_steps.zip b/training/checkpoints/run01/ppo_5199792_steps.zip new file mode 100644 index 00000000..6967a6cd Binary files /dev/null and b/training/checkpoints/run01/ppo_5199792_steps.zip differ diff --git a/training/checkpoints/run01/ppo_5299788_steps.zip b/training/checkpoints/run01/ppo_5299788_steps.zip new file mode 100644 index 00000000..e96155d5 Binary files /dev/null and b/training/checkpoints/run01/ppo_5299788_steps.zip differ diff --git a/training/checkpoints/run01/ppo_5399784_steps.zip b/training/checkpoints/run01/ppo_5399784_steps.zip new file mode 100644 index 00000000..c83ad89d Binary files /dev/null and b/training/checkpoints/run01/ppo_5399784_steps.zip differ diff --git a/training/checkpoints/run01/ppo_5499780_steps.zip b/training/checkpoints/run01/ppo_5499780_steps.zip new file mode 100644 index 00000000..e150cfc9 Binary files /dev/null and b/training/checkpoints/run01/ppo_5499780_steps.zip differ diff --git a/training/checkpoints/run01/ppo_5599776_steps.zip b/training/checkpoints/run01/ppo_5599776_steps.zip new file mode 100644 index 00000000..6b433676 Binary files /dev/null and b/training/checkpoints/run01/ppo_5599776_steps.zip differ diff --git a/training/checkpoints/run01/ppo_5699772_steps.zip b/training/checkpoints/run01/ppo_5699772_steps.zip new file mode 100644 index 00000000..56231799 Binary files /dev/null and b/training/checkpoints/run01/ppo_5699772_steps.zip differ diff --git a/training/checkpoints/run01/ppo_5799768_steps.zip b/training/checkpoints/run01/ppo_5799768_steps.zip new file mode 100644 index 00000000..8d3ce2a5 Binary files /dev/null and b/training/checkpoints/run01/ppo_5799768_steps.zip differ diff --git a/training/checkpoints/run01/ppo_5899764_steps.zip b/training/checkpoints/run01/ppo_5899764_steps.zip new file mode 100644 index 00000000..22527049 Binary files /dev/null and b/training/checkpoints/run01/ppo_5899764_steps.zip differ diff --git a/training/checkpoints/run01/ppo_5999760_steps.zip b/training/checkpoints/run01/ppo_5999760_steps.zip new file mode 100644 index 00000000..ec0d7b56 Binary files /dev/null and b/training/checkpoints/run01/ppo_5999760_steps.zip differ diff --git a/training/checkpoints/run01/ppo_599976_steps.zip b/training/checkpoints/run01/ppo_599976_steps.zip new file mode 100644 index 00000000..a6021601 Binary files /dev/null and b/training/checkpoints/run01/ppo_599976_steps.zip differ diff --git a/training/checkpoints/run01/ppo_6099756_steps.zip b/training/checkpoints/run01/ppo_6099756_steps.zip new file mode 100644 index 00000000..035ee7ef Binary files /dev/null and b/training/checkpoints/run01/ppo_6099756_steps.zip differ diff --git a/training/checkpoints/run01/ppo_6199752_steps.zip b/training/checkpoints/run01/ppo_6199752_steps.zip new file mode 100644 index 00000000..c0fb49bb Binary files /dev/null and b/training/checkpoints/run01/ppo_6199752_steps.zip differ diff --git a/training/checkpoints/run01/ppo_6299748_steps.zip b/training/checkpoints/run01/ppo_6299748_steps.zip new file mode 100644 index 00000000..bc9d1cb2 Binary files /dev/null and b/training/checkpoints/run01/ppo_6299748_steps.zip differ diff --git a/training/checkpoints/run01/ppo_6399744_steps.zip b/training/checkpoints/run01/ppo_6399744_steps.zip new file mode 100644 index 00000000..1cecb075 Binary files /dev/null and b/training/checkpoints/run01/ppo_6399744_steps.zip differ diff --git a/training/checkpoints/run01/ppo_6499740_steps.zip b/training/checkpoints/run01/ppo_6499740_steps.zip new file mode 100644 index 00000000..2f3cf0b4 Binary files /dev/null and b/training/checkpoints/run01/ppo_6499740_steps.zip differ diff --git a/training/checkpoints/run01/ppo_6599736_steps.zip b/training/checkpoints/run01/ppo_6599736_steps.zip new file mode 100644 index 00000000..799d4caf Binary files /dev/null and b/training/checkpoints/run01/ppo_6599736_steps.zip differ diff --git a/training/checkpoints/run01/ppo_6699732_steps.zip b/training/checkpoints/run01/ppo_6699732_steps.zip new file mode 100644 index 00000000..922eec58 Binary files /dev/null and b/training/checkpoints/run01/ppo_6699732_steps.zip differ diff --git a/training/checkpoints/run01/ppo_6799728_steps.zip b/training/checkpoints/run01/ppo_6799728_steps.zip new file mode 100644 index 00000000..89165e1b Binary files /dev/null and b/training/checkpoints/run01/ppo_6799728_steps.zip differ diff --git a/training/checkpoints/run01/ppo_6899724_steps.zip b/training/checkpoints/run01/ppo_6899724_steps.zip new file mode 100644 index 00000000..0e1be937 Binary files /dev/null and b/training/checkpoints/run01/ppo_6899724_steps.zip differ diff --git a/training/checkpoints/run01/ppo_6999720_steps.zip b/training/checkpoints/run01/ppo_6999720_steps.zip new file mode 100644 index 00000000..63fc1148 Binary files /dev/null and b/training/checkpoints/run01/ppo_6999720_steps.zip differ diff --git a/training/checkpoints/run01/ppo_699972_steps.zip b/training/checkpoints/run01/ppo_699972_steps.zip new file mode 100644 index 00000000..576e01e3 Binary files /dev/null and b/training/checkpoints/run01/ppo_699972_steps.zip differ diff --git a/training/checkpoints/run01/ppo_7099716_steps.zip b/training/checkpoints/run01/ppo_7099716_steps.zip new file mode 100644 index 00000000..9c314f3f Binary files /dev/null and b/training/checkpoints/run01/ppo_7099716_steps.zip differ diff --git a/training/checkpoints/run01/ppo_7199712_steps.zip b/training/checkpoints/run01/ppo_7199712_steps.zip new file mode 100644 index 00000000..a2255313 Binary files /dev/null and b/training/checkpoints/run01/ppo_7199712_steps.zip differ diff --git a/training/checkpoints/run01/ppo_7299708_steps.zip b/training/checkpoints/run01/ppo_7299708_steps.zip new file mode 100644 index 00000000..7f1f9e5d Binary files /dev/null and b/training/checkpoints/run01/ppo_7299708_steps.zip differ diff --git a/training/checkpoints/run01/ppo_7399704_steps.zip b/training/checkpoints/run01/ppo_7399704_steps.zip new file mode 100644 index 00000000..52b61220 Binary files /dev/null and b/training/checkpoints/run01/ppo_7399704_steps.zip differ diff --git a/training/checkpoints/run01/ppo_7499700_steps.zip b/training/checkpoints/run01/ppo_7499700_steps.zip new file mode 100644 index 00000000..f41cd1bf Binary files /dev/null and b/training/checkpoints/run01/ppo_7499700_steps.zip differ diff --git a/training/checkpoints/run01/ppo_7599696_steps.zip b/training/checkpoints/run01/ppo_7599696_steps.zip new file mode 100644 index 00000000..231efccf Binary files /dev/null and b/training/checkpoints/run01/ppo_7599696_steps.zip differ diff --git a/training/checkpoints/run01/ppo_7699692_steps.zip b/training/checkpoints/run01/ppo_7699692_steps.zip new file mode 100644 index 00000000..610dd37e Binary files /dev/null and b/training/checkpoints/run01/ppo_7699692_steps.zip differ diff --git a/training/checkpoints/run01/ppo_7799688_steps.zip b/training/checkpoints/run01/ppo_7799688_steps.zip new file mode 100644 index 00000000..f7e9d571 Binary files /dev/null and b/training/checkpoints/run01/ppo_7799688_steps.zip differ diff --git a/training/checkpoints/run01/ppo_7899684_steps.zip b/training/checkpoints/run01/ppo_7899684_steps.zip new file mode 100644 index 00000000..b78feb38 Binary files /dev/null and b/training/checkpoints/run01/ppo_7899684_steps.zip differ diff --git a/training/checkpoints/run01/ppo_7999680_steps.zip b/training/checkpoints/run01/ppo_7999680_steps.zip new file mode 100644 index 00000000..a5dea260 Binary files /dev/null and b/training/checkpoints/run01/ppo_7999680_steps.zip differ diff --git a/training/checkpoints/run01/ppo_799968_steps.zip b/training/checkpoints/run01/ppo_799968_steps.zip new file mode 100644 index 00000000..6851cfcb Binary files /dev/null and b/training/checkpoints/run01/ppo_799968_steps.zip differ diff --git a/training/checkpoints/run01/ppo_8099676_steps.zip b/training/checkpoints/run01/ppo_8099676_steps.zip new file mode 100644 index 00000000..42428859 Binary files /dev/null and b/training/checkpoints/run01/ppo_8099676_steps.zip differ diff --git a/training/checkpoints/run01/ppo_8199672_steps.zip b/training/checkpoints/run01/ppo_8199672_steps.zip new file mode 100644 index 00000000..9f8dd846 Binary files /dev/null and b/training/checkpoints/run01/ppo_8199672_steps.zip differ diff --git a/training/checkpoints/run01/ppo_8299668_steps.zip b/training/checkpoints/run01/ppo_8299668_steps.zip new file mode 100644 index 00000000..8003007f Binary files /dev/null and b/training/checkpoints/run01/ppo_8299668_steps.zip differ diff --git a/training/checkpoints/run01/ppo_8399664_steps.zip b/training/checkpoints/run01/ppo_8399664_steps.zip new file mode 100644 index 00000000..14339c97 Binary files /dev/null and b/training/checkpoints/run01/ppo_8399664_steps.zip differ diff --git a/training/checkpoints/run01/ppo_8499660_steps.zip b/training/checkpoints/run01/ppo_8499660_steps.zip new file mode 100644 index 00000000..5002909a Binary files /dev/null and b/training/checkpoints/run01/ppo_8499660_steps.zip differ diff --git a/training/checkpoints/run01/ppo_8599656_steps.zip b/training/checkpoints/run01/ppo_8599656_steps.zip new file mode 100644 index 00000000..3fb29aea Binary files /dev/null and b/training/checkpoints/run01/ppo_8599656_steps.zip differ diff --git a/training/checkpoints/run01/ppo_8699652_steps.zip b/training/checkpoints/run01/ppo_8699652_steps.zip new file mode 100644 index 00000000..61ae73b4 Binary files /dev/null and b/training/checkpoints/run01/ppo_8699652_steps.zip differ diff --git a/training/checkpoints/run01/ppo_8799648_steps.zip b/training/checkpoints/run01/ppo_8799648_steps.zip new file mode 100644 index 00000000..9be4d1a4 Binary files /dev/null and b/training/checkpoints/run01/ppo_8799648_steps.zip differ diff --git a/training/checkpoints/run01/ppo_8899644_steps.zip b/training/checkpoints/run01/ppo_8899644_steps.zip new file mode 100644 index 00000000..66bbab30 Binary files /dev/null and b/training/checkpoints/run01/ppo_8899644_steps.zip differ diff --git a/training/checkpoints/run01/ppo_8999640_steps.zip b/training/checkpoints/run01/ppo_8999640_steps.zip new file mode 100644 index 00000000..26ba3a49 Binary files /dev/null and b/training/checkpoints/run01/ppo_8999640_steps.zip differ diff --git a/training/checkpoints/run01/ppo_899964_steps.zip b/training/checkpoints/run01/ppo_899964_steps.zip new file mode 100644 index 00000000..b8fc9d6f Binary files /dev/null and b/training/checkpoints/run01/ppo_899964_steps.zip differ diff --git a/training/checkpoints/run01/ppo_9099636_steps.zip b/training/checkpoints/run01/ppo_9099636_steps.zip new file mode 100644 index 00000000..bc1e0bff Binary files /dev/null and b/training/checkpoints/run01/ppo_9099636_steps.zip differ diff --git a/training/checkpoints/run01/ppo_9199632_steps.zip b/training/checkpoints/run01/ppo_9199632_steps.zip new file mode 100644 index 00000000..1ed0e5fe Binary files /dev/null and b/training/checkpoints/run01/ppo_9199632_steps.zip differ diff --git a/training/checkpoints/run01/ppo_9299628_steps.zip b/training/checkpoints/run01/ppo_9299628_steps.zip new file mode 100644 index 00000000..57209858 Binary files /dev/null and b/training/checkpoints/run01/ppo_9299628_steps.zip differ diff --git a/training/checkpoints/run01/ppo_9399624_steps.zip b/training/checkpoints/run01/ppo_9399624_steps.zip new file mode 100644 index 00000000..b707f1e9 Binary files /dev/null and b/training/checkpoints/run01/ppo_9399624_steps.zip differ diff --git a/training/checkpoints/run01/ppo_9499620_steps.zip b/training/checkpoints/run01/ppo_9499620_steps.zip new file mode 100644 index 00000000..91cc21e2 Binary files /dev/null and b/training/checkpoints/run01/ppo_9499620_steps.zip differ diff --git a/training/checkpoints/run01/ppo_9599616_steps.zip b/training/checkpoints/run01/ppo_9599616_steps.zip new file mode 100644 index 00000000..f54b7b29 Binary files /dev/null and b/training/checkpoints/run01/ppo_9599616_steps.zip differ diff --git a/training/checkpoints/run01/ppo_9699612_steps.zip b/training/checkpoints/run01/ppo_9699612_steps.zip new file mode 100644 index 00000000..9b152171 Binary files /dev/null and b/training/checkpoints/run01/ppo_9699612_steps.zip differ diff --git a/training/checkpoints/run01/ppo_9799608_steps.zip b/training/checkpoints/run01/ppo_9799608_steps.zip new file mode 100644 index 00000000..62dba11a Binary files /dev/null and b/training/checkpoints/run01/ppo_9799608_steps.zip differ diff --git a/training/checkpoints/run01/ppo_9899604_steps.zip b/training/checkpoints/run01/ppo_9899604_steps.zip new file mode 100644 index 00000000..9cd9190b Binary files /dev/null and b/training/checkpoints/run01/ppo_9899604_steps.zip differ diff --git a/training/checkpoints/run01/ppo_9999600_steps.zip b/training/checkpoints/run01/ppo_9999600_steps.zip new file mode 100644 index 00000000..fb1a0b81 Binary files /dev/null and b/training/checkpoints/run01/ppo_9999600_steps.zip differ diff --git a/training/checkpoints/run01/ppo_999960_steps.zip b/training/checkpoints/run01/ppo_999960_steps.zip new file mode 100644 index 00000000..7a29bb63 Binary files /dev/null and b/training/checkpoints/run01/ppo_999960_steps.zip differ diff --git a/training/checkpoints/run01/ppo_99996_steps.zip b/training/checkpoints/run01/ppo_99996_steps.zip new file mode 100644 index 00000000..844849b1 Binary files /dev/null and b/training/checkpoints/run01/ppo_99996_steps.zip differ diff --git a/training/checkpoints/smoke/final.zip b/training/checkpoints/smoke/final.zip new file mode 100644 index 00000000..97bdec80 Binary files /dev/null and b/training/checkpoints/smoke/final.zip differ diff --git a/training/checkpoints/smoke/ppo_100000_steps.zip b/training/checkpoints/smoke/ppo_100000_steps.zip new file mode 100644 index 00000000..8771699d Binary files /dev/null and b/training/checkpoints/smoke/ppo_100000_steps.zip differ diff --git a/training/logs/protosmoke2_1/events.out.tfevents.1784399198.Joshs-Mac-mini.local.92431.0 b/training/logs/protosmoke2_1/events.out.tfevents.1784399198.Joshs-Mac-mini.local.92431.0 new file mode 100644 index 00000000..b0d06158 Binary files /dev/null and b/training/logs/protosmoke2_1/events.out.tfevents.1784399198.Joshs-Mac-mini.local.92431.0 differ diff --git a/training/logs/protosmoke3_1/events.out.tfevents.1784399254.Joshs-Mac-mini.local.92548.0 b/training/logs/protosmoke3_1/events.out.tfevents.1784399254.Joshs-Mac-mini.local.92548.0 new file mode 100644 index 00000000..4d017028 Binary files /dev/null and b/training/logs/protosmoke3_1/events.out.tfevents.1784399254.Joshs-Mac-mini.local.92548.0 differ diff --git a/training/logs/protosmoke_1/events.out.tfevents.1784398540.Joshs-Mac-mini.local.91198.0 b/training/logs/protosmoke_1/events.out.tfevents.1784398540.Joshs-Mac-mini.local.91198.0 new file mode 100644 index 00000000..57e1fcf6 Binary files /dev/null and b/training/logs/protosmoke_1/events.out.tfevents.1784398540.Joshs-Mac-mini.local.91198.0 differ diff --git a/training/logs/run01_1/events.out.tfevents.1784402470.Joshs-Mac-mini.local.99530.0 b/training/logs/run01_1/events.out.tfevents.1784402470.Joshs-Mac-mini.local.99530.0 new file mode 100644 index 00000000..7ed846da Binary files /dev/null and b/training/logs/run01_1/events.out.tfevents.1784402470.Joshs-Mac-mini.local.99530.0 differ diff --git a/training/logs/smoke_1/events.out.tfevents.1784398606.Joshs-Mac-mini.local.91287.0 b/training/logs/smoke_1/events.out.tfevents.1784398606.Joshs-Mac-mini.local.91287.0 new file mode 100644 index 00000000..faf96abd Binary files /dev/null and b/training/logs/smoke_1/events.out.tfevents.1784398606.Joshs-Mac-mini.local.91287.0 differ diff --git a/training/run_training.sh b/training/run_training.sh new file mode 100755 index 00000000..e87a17b2 --- /dev/null +++ b/training/run_training.sh @@ -0,0 +1,39 @@ +#!/usr/bin/env bash +# Train, export the policy, and commit every artifact to git so no training +# is ever stranded on one machine. Idempotent and re-runnable: pulls latest +# before training, commits only when there is something new, and a Ctrl-C'd +# run still exports and commits (final.zip is written on the way out). +# +# Usage: ./run_training.sh [train.py args...] +# e.g.: ./run_training.sh run02 --timesteps 20000000 --n-parallel 14 --speedup 16 \ +# --resume checkpoints/run01/final.zip --ent-coef 0.001 --reset-std 0.3 +set -uo pipefail + +cd "$(dirname "$0")" +EXP="${1:?usage: run_training.sh [train.py args...]}" +shift + +# Train on the latest code and checkpoints from any machine +git pull --rebase + +# Let Ctrl-C stop train.py without killing this script, so the export and +# commit below still run +trap ':' INT +.venv/bin/python train.py --experiment "$EXP" "$@" +trap - INT + +if [ ! -f "checkpoints/$EXP/final.zip" ]; then + echo "No checkpoints/$EXP/final.zip — nothing to export or commit" >&2 + exit 1 +fi + +# Export for in-game use (parity-checked); models live in Game/bots/ +.venv/bin/python export_policy.py "checkpoints/$EXP/final.zip" "../Game/bots/$EXP.json" + +git add -A checkpoints logs eval_history.json "../Game/bots" +if git diff --cached --quiet; then + echo "Nothing new to commit" +else + git commit -m "chore(training): Add $EXP checkpoints, logs, and exported policy" + git push +fi diff --git a/training/setup_linux.sh b/training/setup_linux.sh new file mode 100755 index 00000000..1bc38ad0 --- /dev/null +++ b/training/setup_linux.sh @@ -0,0 +1,42 @@ +#!/usr/bin/env bash +# Idempotent setup for a Linux training box — safe to re-run at any time; +# every step checks before it acts. See TRAINING_LINUX.md. +set -euo pipefail + +cd "$(dirname "$0")" + +GODOT_VERSION="4.7.1" +GODOT_BIN="${GODOT_BIN:-$HOME/ai-training/godot/Godot_v${GODOT_VERSION}-stable_linux.x86_64}" + +# Godot binary +if [ ! -x "$GODOT_BIN" ]; then + echo "Downloading Godot $GODOT_VERSION to $(dirname "$GODOT_BIN")" + mkdir -p "$(dirname "$GODOT_BIN")" + wget -q "https://github.com/godotengine/godot/releases/download/${GODOT_VERSION}-stable/Godot_v${GODOT_VERSION}-stable_linux.x86_64.zip" \ + -O /tmp/godot_dl.zip + unzip -o -q /tmp/godot_dl.zip -d "$(dirname "$GODOT_BIN")" + rm /tmp/godot_dl.zip + chmod +x "$GODOT_BIN" +fi +echo "Godot: $GODOT_BIN" + +# Python env +[ -d .venv ] || python3 -m venv .venv +.venv/bin/pip install -q -r requirements.txt + +# CUDA check; pip's default torch is CUDA-enabled on Linux, but fall back to +# the explicit CUDA index if this box ended up with a CPU-only build +if [ "$(.venv/bin/python -c 'import torch; print(torch.cuda.is_available())')" != "True" ]; then + echo "CUDA not available — reinstalling torch from the CUDA wheel index" + .venv/bin/pip install -q torch --index-url https://download.pytorch.org/whl/cu121 +fi + +# Import pass — a fresh clone has no .godot/ cache, so class_name scripts +# (GameMode, Goal, …) aren't registered and scenes fail to load until the +# project is imported once. Cheap when the cache already exists. +"$GODOT_BIN" --headless --path ../Game --import + +# Headless smoke test — the game must boot without rendering +"$GODOT_BIN" --headless --path ../Game res://scenes/free_play.tscn --quit-after 300 + +echo "Setup OK — train with: ./run_training.sh [train.py args...]"