Compare commits

...

8 Commits

Author SHA1 Message Date
Josh Creek d229bccd19 fix(arena): contain goal pockets 2026-08-08 15:03:46 +01:00
Josh Creek 341a67f6da feat(training): add generation 5 curriculum 2026-08-08 14:56:17 +01:00
Josh Creek 33952b3cd0 feat(training): pair evaluations across physical sides 2026-08-08 14:55:12 +01:00
Josh Creek 57a298dc06 fix(ai): map team-relative rotation actions 2026-08-08 14:54:27 +01:00
Josh Creek 7e217df898 feat(ai): promote stage 3 policy as easy bot 2026-08-08 14:52:36 +01:00
Josh Creek 0285116ddc fix(feedback): remove ball contact screen flash 2026-08-08 08:52:39 +01:00
Josh Creek 0223b55de8 fix(vfx): remove ball contact sparks 2026-08-08 08:47:48 +01:00
Josh Creek e0a1cbaf6d fix(presentation): simplify camera and engine effects 2026-08-08 08:44:26 +01:00
32 changed files with 1110 additions and 370 deletions
-93
View File
@@ -1,93 +0,0 @@
Copyright 2018 The Orbitron Project Authors (https://github.com/theleagueof/orbitron), with Reserved Font Name: "Orbitron"
This Font Software is licensed under the SIL Open Font License, Version 1.1.
This license is copied below, and is also available with a FAQ at:
http://scripts.sil.org/OFL
-----------------------------------------------------------
SIL OPEN FONT LICENSE Version 1.1 - 26 February 2007
-----------------------------------------------------------
PREAMBLE
The goals of the Open Font License (OFL) are to stimulate worldwide
development of collaborative font projects, to support the font creation
efforts of academic and linguistic communities, and to provide a free and
open framework in which fonts may be shared and improved in partnership
with others.
The OFL allows the licensed fonts to be used, studied, modified and
redistributed freely as long as they are not sold by themselves. The
fonts, including any derivative works, can be bundled, embedded,
redistributed and/or sold with any software provided that any reserved
names are not used by derivative works. The fonts and derivatives,
however, cannot be released under any other type of license. The
requirement for fonts to remain under this license does not apply
to any document created using the fonts or their derivatives.
DEFINITIONS
"Font Software" refers to the set of files released by the Copyright
Holder(s) under this license and clearly marked as such. This may
include source files, build scripts and documentation.
"Reserved Font Name" refers to any names specified as such after the
copyright statement(s).
"Original Version" refers to the collection of Font Software components as
distributed by the Copyright Holder(s).
"Modified Version" refers to any derivative made by adding to, deleting,
or substituting -- in part or in whole -- any of the components of the
Original Version, by changing formats or by porting the Font Software to a
new environment.
"Author" refers to any designer, engineer, programmer, technical
writer or other person who contributed to the Font Software.
PERMISSION & CONDITIONS
Permission is hereby granted, free of charge, to any person obtaining
a copy of the Font Software, to use, study, copy, merge, embed, modify,
redistribute, and sell modified and unmodified copies of the Font
Software, subject to the following conditions:
1) Neither the Font Software nor any of its individual components,
in Original or Modified Versions, may be sold by itself.
2) Original or Modified Versions of the Font Software may be bundled,
redistributed and/or sold with any software, provided that each copy
contains the above copyright notice and this license. These can be
included either as stand-alone text files, human-readable headers or
in the appropriate machine-readable metadata fields within text or
binary files as long as those fields can be easily viewed by the user.
3) No Modified Version of the Font Software may use the Reserved Font
Name(s) unless explicit written permission is granted by the corresponding
Copyright Holder. This restriction only applies to the primary font name as
presented to the users.
4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font
Software shall not be used to promote, endorse or advertise any
Modified Version, except to acknowledge the contribution(s) of the
Copyright Holder(s) and the Author(s) or with their explicit written
permission.
5) The Font Software, modified or unmodified, in part or in whole,
must be distributed entirely under this license, and must not be
distributed under any other license. The requirement for fonts to
remain under this license does not apply to any document created
using the Font Software.
TERMINATION
This license becomes null and void if any of the above conditions are
not met.
DISCLAIMER
THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT
OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE
COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL
DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM
OTHER DEALINGS IN THE FONT SOFTWARE.
Binary file not shown.
@@ -1,36 +0,0 @@
[remap]
importer="font_data_dynamic"
type="FontFile"
uid="uid://bhbpb712ouagv"
path="res://.godot/imported/Orbitron-Medium.ttf-0d3cd88166f12a3096066c2eef570744.fontdata"
[deps]
source_file="res://assets/fonts/Orbitron-Medium.ttf"
dest_files=["res://.godot/imported/Orbitron-Medium.ttf-0d3cd88166f12a3096066c2eef570744.fontdata"]
[params]
Rendering=null
antialiasing=1
generate_mipmaps=false
disable_embedded_bitmaps=true
multichannel_signed_distance_field=false
msdf_pixel_range=8
msdf_size=48
allow_system_fallback=true
force_autohinter=false
modulate_color_glyphs=false
hinting=3
subpixel_positioning=4
keep_rounding_remainders=true
oversampling=0.0
Fallbacks=null
fallbacks=[]
Compress=null
compress=true
preload=[]
language_support={}
script_support={}
opentype_features={}
File diff suppressed because one or more lines are too long
+1 -14
View File
@@ -1,10 +1,9 @@
[gd_scene load_steps=10 format=3 uid="uid://c8kak2l3m4n5"]
[gd_scene load_steps=9 format=3 uid="uid://c8kak2l3m4n5"]
[ext_resource type="Script" uid="uid://du7y176h5aaq4" path="res://scripts/HUDController.gd" id="1_hud_controller"]
[ext_resource type="Script" path="res://scripts/hud_attitude_indicator.gd" id="2_adi"]
[ext_resource type="Script" path="res://scripts/hud_heading_tape.gd" id="3_tape"]
[ext_resource type="Script" path="res://scripts/hud_gauge.gd" id="4_gauge"]
[ext_resource type="Theme" path="res://themes/hud_theme.tres" id="5_theme"]
[sub_resource type="LabelSettings" id="LabelSettings_hud"]
font_size = 32
@@ -51,7 +50,6 @@ anchor_bottom = 1.0
grow_horizontal = 2
grow_vertical = 2
mouse_filter = 2
theme = ExtResource("5_theme")
[node name="ScoreboardPanel" type="PanelContainer" parent="Control"]
layout_mode = 1
@@ -120,17 +118,6 @@ label_settings = SubResource("LabelSettings_kickoff")
horizontal_alignment = 1
vertical_alignment = 1
[node name="ImpactFlash" type="ColorRect" parent="Control"]
visible = false
layout_mode = 1
anchors_preset = 15
anchor_right = 1.0
anchor_bottom = 1.0
grow_horizontal = 2
grow_vertical = 2
mouse_filter = 2
color = Color(0.72, 0.9, 1, 0)
[node name="GoalCelebration" type="Control" parent="Control"]
visible = false
layout_mode = 1
+1 -11
View File
@@ -1,17 +1,8 @@
[gd_scene load_steps=5 format=3]
[gd_scene load_steps=4 format=3]
[ext_resource type="Script" path="res://scripts/ship_camera.gd" id="1_rig"]
[ext_resource type="Shader" path="res://shaders/post_process.gdshader" id="2_post"]
[sub_resource type="CameraAttributesPractical" id="CameraAttributes_gameplay"]
dof_blur_far_enabled = true
dof_blur_far_distance = 24.0
dof_blur_far_transition = 14.0
dof_blur_near_enabled = true
dof_blur_near_distance = 1.25
dof_blur_near_transition = 0.75
dof_blur_amount = 0.08
[sub_resource type="ShaderMaterial" id="ShaderMaterial_post"]
shader = ExtResource("2_post")
@@ -20,7 +11,6 @@ script = ExtResource("1_rig")
[node name="Camera3D" type="Camera3D" parent="."]
current = true
attributes = SubResource("CameraAttributes_gameplay")
[node name="PostProcess" type="CanvasLayer" parent="."]
layer = 0
-14
View File
@@ -18,7 +18,6 @@ class_name HUDController
@onready var result_panel = get_node_or_null("Control/ResultOverlay/Center/ResultPanel")
@onready var result_label = get_node_or_null("Control/ResultOverlay/Center/ResultPanel/VBox/ResultLabel")
@onready var final_score_label = get_node_or_null("Control/ResultOverlay/Center/ResultPanel/VBox/FinalScoreLabel")
@onready var impact_flash = get_node_or_null("Control/ImpactFlash")
@onready var goal_celebration = get_node_or_null("Control/GoalCelebration")
@onready var goal_tint = get_node_or_null("Control/GoalCelebration/Tint")
@onready var goal_title = get_node_or_null("Control/GoalCelebration/Center/GoalTitle")
@@ -32,7 +31,6 @@ class_name HUDController
var ship: Node
var _last_score := {0: 0, 1: 0}
var _impact_flash_tween: Tween
var _goal_tween: Tween
func _ready():
@@ -177,18 +175,6 @@ func _flash_score_label(label: Label) -> void:
tween.tween_property(label, "scale", Vector2(1.0, 1.0), 0.2)
func flash_impact(intensity: float) -> void:
if not impact_flash or not is_instance_valid(impact_flash):
return
if _impact_flash_tween and _impact_flash_tween.is_valid():
_impact_flash_tween.kill()
impact_flash.color = Color(0.72, 0.9, 1.0, lerpf(0.06, 0.24, intensity))
impact_flash.visible = true
_impact_flash_tween = create_tween()
_impact_flash_tween.tween_property(impact_flash, "color:a", 0.0, lerpf(0.08, 0.18, intensity))
_impact_flash_tween.tween_callback(func(): impact_flash.visible = false)
func show_goal_celebration(scoring_team: int) -> void:
if not goal_celebration or not goal_title or not goal_tint:
return
+26 -2
View File
@@ -34,6 +34,11 @@ var _policy: PolicyNetwork
var _action := ShipAction.new()
var _ticks_until_decision := 0
# League mode revisits a small model pool every episode. Policies are
# immutable after load, so share parsed networks by path instead of reading
# and flattening the same 100KB+ JSON on every reset for every frozen ship.
static var _policy_cache: Dictionary = {}
var _ship: Ship
var _teammates: Array[Ship] = []
var _opponents: Array[Ship] = []
@@ -44,7 +49,24 @@ var _scene_refs_ready := false
func _ready():
if not model_path.is_empty():
load_policy(model_path)
# League training swaps a frozen opponent's policy between episodes without
# despawning the ship. Reset the held action/decision cadence with the model
# so no command from the previous opponent leaks into the next episode.
func load_policy(path: String) -> void:
model_path = path
if model_path.is_empty():
_policy = null
elif _policy_cache.has(model_path):
_policy = _policy_cache[model_path]
else:
_policy = PolicyNetwork.load_from_file(model_path)
if _policy != null:
_policy_cache[model_path] = _policy
_action = ShipAction.new()
_ticks_until_decision = 0
func get_action() -> ShipAction:
@@ -75,11 +97,13 @@ func _decide() -> void:
_action.thrust.y = 0.0
else:
_action = ShipActionCodec.from_logits(out, action_noise)
_action = ShipActionCodec.apply_team_frame(_action, _ship.team)
# Find ship/ball/teammates/opponents/goal once everything is spawned.
# ShipAction axes are body-frame so only observations need team context
# (ShipObservations). Rosters never change mid-match (no despawn path exists
# ShipAction thrust axes are body-frame, while its rotation axes are mapped
# from the canonical team frame by ShipActionCodec. Rosters never change
# mid-match (no despawn path exists
# anywhere in this codebase), so this only needs to run once — sorted by
# spawn_index so a given ship keeps the same observation slot for the whole
# match, matching TrainingMode's identically-sorted lists.
-2
View File
@@ -164,8 +164,6 @@ func spawn_camera_rig(target: Ship) -> ShipCameraRig:
func _on_player_impact(intensity: float) -> void:
if hud:
hud.flash_impact(intensity)
if not _goal_slowmo_active:
_run_hit_stop(intensity)
+29
View File
@@ -14,6 +14,14 @@ signal goal_scored(team: int)
# wall is thick or it pokes out the back of the arena. ArenaBoundary cuts the
# matching aperture in the hull (see its GOAL_APERTURE_* constants).
const POCKET_DEPTH := ArenaBoundary.SURFACE_THICKNESS
# Physical back wall for the goal pocket. The arena end wall has a real hole
# at the scoring aperture; without this, the visual-only net lets ships and
# the ball leave the enclosure after crossing the sensor. A small overlap
# into the surrounding end-wall panels closes numerical seams at the rim.
const BACKSTOP_THICKNESS := 0.4
const BACKSTOP_EDGE_OVERLAP := 0.25
const ARENA_COLLISION_LAYER := 1 << 2
const ARENA_COLLISION_MASK := (1 << 0) | (1 << 1)
# Clearance between the mouth and the pocket shell, so the pocket's side walls
# stay hidden behind the hull rather than showing at the aperture edge.
const POCKET_CLEARANCE := 0.2
@@ -32,6 +40,7 @@ var _goal_burst: GPUParticles3D
func _ready():
# Group lets AI controllers and game modes discover goals
add_to_group("goal")
_build_backstop()
# Everything below _on_body_entered is decoration, and training spawns
# headless arenas that never render it. The sensor is unaffected.
if DisplayServer.get_name() != "headless":
@@ -46,6 +55,26 @@ func _on_body_entered(body):
goal_scored.emit(team)
func _build_backstop() -> void:
var mouth := ($CollisionShape3D.shape as BoxShape3D).size
var body := StaticBody3D.new()
body.name = "PocketBackstop"
body.collision_layer = ARENA_COLLISION_LAYER
body.collision_mask = ARENA_COLLISION_MASK
var collision := CollisionShape3D.new()
collision.name = "CollisionShape3D"
var box := BoxShape3D.new()
box.size = Vector3(
mouth.x + BACKSTOP_EDGE_OVERLAP * 2.0,
mouth.y + BACKSTOP_EDGE_OVERLAP * 2.0,
BACKSTOP_THICKNESS
)
collision.shape = box
collision.position = Vector3(0.0, 0.0, POCKET_DEPTH)
body.add_child(collision)
add_child(body)
# --- visuals -----------------------------------------------------------------
# Cosmetic only. The mouth is measured off the sensor's own collision shape, so
# the frame can never drift from the volume that actually scores.
+1 -1
View File
@@ -73,7 +73,7 @@ func _draw() -> void:
draw_line(Vector2(-half_chord, horizon_y), Vector2(half_chord, horizon_y), LINE_COLOR, 2.0)
# Pitch ladder: rungs every 10 degrees, labelled, scrolling with pitch.
var font := hud_font()
var font := ThemeDB.fallback_font
for ladder_deg: int in [-30, -20, -10, 10, 20, 30]:
var y := horizon_y - ladder_deg * px_per_deg
if absf(y) > radius - 12.0:
+1 -1
View File
@@ -37,7 +37,7 @@ func _process(delta: float) -> void:
func _draw() -> void:
var font := hud_font()
var font := ThemeDB.fallback_font
var fraction := clampf(_value / max_value, 0.0, 1.0)
var reading := value_format % _value
+1 -1
View File
@@ -39,7 +39,7 @@ func _process(delta: float) -> void:
func _draw() -> void:
var font := hud_font()
var font := ThemeDB.fallback_font
var center_x := size.x / 2.0
var px_per_deg := size.x / VISIBLE_RANGE
-8
View File
@@ -23,11 +23,3 @@ static func angle_delta_deg(from: float, to: float) -> float:
func _smoothing_weight(delta: float) -> float:
return 1.0 - exp(-SMOOTHING * delta) # frame-rate independent
# Procedural draw calls do not automatically consume a Control's theme font;
# resolve it explicitly so every instrument uses the same packaged typeface
# as ordinary Labels instead of ThemeDB's debug-looking fallback.
func hud_font() -> Font:
var font := get_theme_default_font()
return font if font else ThemeDB.fallback_font
+6 -6
View File
@@ -11,13 +11,13 @@ extends Control
const BOTS_DIR := "res://bots"
# All three tiers deliberately point at the one trained checkpoint
# (easy.json) and differ only by reaction_ticks/action_noise handicap, not by
# model skill. Superseded once real medium/hard checkpoints are promoted into
# res://bots/promoted/ — see the "AI opponent" section of TODO.md.
# All three tiers deliberately point at the same promoted checkpoint at its
# full trained capability. The labels are placeholders until genuinely
# stronger models are promoted as medium.json/hard.json; difficulty is not
# simulated with reaction delay or action noise in the meantime.
const DIFFICULTIES := [
{"name": "Easy", "model": "res://bots/promoted/easy.json", "reaction_ticks": 24, "action_noise": 0.35},
{"name": "Medium", "model": "res://bots/promoted/easy.json", "reaction_ticks": 14, "action_noise": 0.15},
{"name": "Easy", "model": "res://bots/promoted/easy.json", "reaction_ticks": 8, "action_noise": 0.0},
{"name": "Medium", "model": "res://bots/promoted/easy.json", "reaction_ticks": 8, "action_noise": 0.0},
{"name": "Hard", "model": "res://bots/promoted/easy.json", "reaction_ticks": 8, "action_noise": 0.0},
]
+42 -72
View File
@@ -117,10 +117,8 @@ const ATTITUDE_THRESHOLD = 1.0 # degrees
const THRUST_THRESHOLD = 1.0 # percent
var _engine_cores: Array[MeshInstance3D] = []
var _engine_plumes: Array[GPUParticles3D] = []
var _turbo_flames: Array[GPUParticles3D] = []
var _engine_flames: Array[MeshInstance3D] = []
var _engine_lights: Array[OmniLight3D] = []
var _impact_sparks: GPUParticles3D
func _ready():
@@ -215,7 +213,7 @@ func _build_movement_vfx() -> void:
for x in [-0.42, 0.42]:
var engine_pos := Vector3(x, -0.05, 1.42)
var core_mat := _vfx_material(Color(0.35, 0.85, 1.0, 1.0), 1.0)
var core_mat := _vfx_material(Color(1.0, 0.48, 0.1, 1.0), 1.0)
var core_mesh := SphereMesh.new()
core_mesh.radius = 0.11
core_mesh.height = 0.22
@@ -228,45 +226,43 @@ func _build_movement_vfx() -> void:
add_child(core)
_engine_cores.append(core)
var plume := _make_particles(
"EnginePlumeL" if x < 0.0 else "EnginePlumeR",
Color(0.3, 0.82, 1.0, 0.72), Vector2(0.16, 0.42), 36, 0.24,
Vector3(0, 0, 1), 12.0, 2.0, 5.0
)
plume.position = engine_pos + Vector3(0, 0, 0.08)
add_child(plume)
_engine_plumes.append(plume)
var turbo := _make_particles(
"TurboFlameL" if x < 0.0 else "TurboFlameR",
Color(0.78, 0.34, 1.0, 0.9), Vector2(0.22, 0.72), 48, 0.18,
Vector3(0, 0, 1), 7.0, 6.0, 10.0
)
turbo.position = engine_pos + Vector3(0, 0, 0.12)
add_child(turbo)
_turbo_flames.append(turbo)
# A single conventional orange flame replaces the layered particle plume
# and separate purple turbo effect. Turbo only lengthens and brightens the
# same flame, keeping the engine silhouette simple and readable.
var flame_mat := StandardMaterial3D.new()
flame_mat.transparency = BaseMaterial3D.TRANSPARENCY_ALPHA
flame_mat.shading_mode = BaseMaterial3D.SHADING_MODE_UNSHADED
flame_mat.cull_mode = BaseMaterial3D.CULL_DISABLED
flame_mat.albedo_color = Color(1.0, 0.34, 0.04, 0.92)
flame_mat.emission_enabled = true
flame_mat.emission = Color(1.0, 0.16, 0.015)
flame_mat.emission_energy_multiplier = 2.0
var flame_mesh := CylinderMesh.new()
flame_mesh.top_radius = 0.015
flame_mesh.bottom_radius = 0.14
flame_mesh.height = 1.0
flame_mesh.radial_segments = 12
flame_mesh.material = flame_mat
var flame := MeshInstance3D.new()
flame.name = "EngineFlameL" if x < 0.0 else "EngineFlameR"
flame.position = engine_pos + Vector3(0, 0, 0.3)
flame.rotation_degrees.x = 90.0
flame.mesh = flame_mesh
flame.visible = false
flame.cast_shadow = GeometryInstance3D.SHADOW_CASTING_SETTING_OFF
add_child(flame)
_engine_flames.append(flame)
var light := OmniLight3D.new()
light.name = "EngineLightL" if x < 0.0 else "EngineLightR"
light.position = engine_pos
light.light_color = Color(0.3, 0.75, 1.0)
light.light_color = Color(1.0, 0.32, 0.08)
light.omni_range = 3.5
light.omni_attenuation = 2.0
light.shadow_enabled = false
add_child(light)
_engine_lights.append(light)
_impact_sparks = _make_particles(
"ImpactSparks", Color(1.0, 0.72, 0.22, 1.0), Vector2(0.09, 0.3),
28, 0.32, Vector3.UP, 180.0, 5.0, 12.0
)
_impact_sparks.one_shot = true
_impact_sparks.explosiveness = 1.0
_impact_sparks.local_coords = false
_impact_sparks.emitting = false
add_child(_impact_sparks)
func _vfx_material(color: Color, energy: float) -> StandardMaterial3D:
var mat := StandardMaterial3D.new()
mat.transparency = BaseMaterial3D.TRANSPARENCY_ALPHA
@@ -279,47 +275,25 @@ func _vfx_material(color: Color, energy: float) -> StandardMaterial3D:
return mat
func _make_particles(
particle_name: String, color: Color, size: Vector2, amount: int,
lifetime: float, direction: Vector3, spread: float,
velocity_min: float, velocity_max: float
) -> GPUParticles3D:
var quad := QuadMesh.new()
quad.size = size
quad.material = _vfx_material(color, 2.2)
var process := ParticleProcessMaterial.new()
process.direction = direction
process.spread = spread
process.initial_velocity_min = velocity_min
process.initial_velocity_max = velocity_max
process.gravity = Vector3.ZERO
process.scale_min = 0.35
process.scale_max = 1.0
var particles := GPUParticles3D.new()
particles.name = particle_name
particles.amount = amount
particles.lifetime = lifetime
particles.local_coords = true
particles.process_material = process
particles.draw_pass_1 = quad
particles.visibility_aabb = AABB(Vector3(-3, -3, -1), Vector3(6, 6, 12))
particles.emitting = false
return particles
func _update_movement_vfx() -> void:
if _engine_plumes.is_empty():
if _engine_flames.is_empty():
return
# These are the two rear main engines, so lateral/vertical maneuvering jets
# must not make them flare. Reverse thrust also comes from separate attitude
# jets conceptually; only positive Z drives this rear-facing plume.
# jets conceptually; only positive Z drives this rear-facing flame.
var thrust := clampf(maxf(_current_action.thrust.z, 0.0), 0.0, 1.0)
var turbo := _current_action.turbo and thrust > 0.05
for i in _engine_plumes.size():
_engine_plumes[i].emitting = thrust > 0.02
_engine_plumes[i].amount_ratio = 0.25 + thrust * 0.75
_engine_plumes[i].speed_scale = 0.65 + thrust * 0.75
_turbo_flames[i].emitting = turbo
for i in _engine_flames.size():
var flame := _engine_flames[i]
var flame_length := 0.28 + thrust * 0.82 + (0.62 if turbo else 0.0)
var flame_width := 0.72 + thrust * 0.32 + (0.12 if turbo else 0.0)
flame.visible = thrust > 0.02
flame.scale = Vector3(flame_width, flame_length, flame_width)
# CylinderMesh is centred on local Y (rotated to ship +Z), so moving its
# centre by half the length keeps the flame root fixed at the engine bell.
flame.position.z = 1.48 + flame_length * 0.5
var flame_mat := flame.mesh.material as StandardMaterial3D
flame_mat.emission_energy_multiplier = 1.8 + thrust * 2.2 + (1.8 if turbo else 0.0)
_engine_lights[i].light_energy = 0.35 + thrust * 2.1 + (2.3 if turbo else 0.0)
var core_mat := _engine_cores[i].mesh.material as StandardMaterial3D
core_mat.emission_energy_multiplier = 0.65 + thrust * 2.0 + (2.0 if turbo else 0.0)
@@ -340,10 +314,6 @@ func _on_body_entered(body: Node) -> void:
var relative_speed := (linear_velocity - (body as Ball).linear_velocity).length()
var intensity := clampf(inverse_lerp(3.0, 24.0, relative_speed), 0.12, 1.0)
var contact_pos := (global_position + (body as Ball).global_position) * 0.5
if _impact_sparks:
_impact_sparks.position = to_local(contact_pos)
_impact_sparks.amount_ratio = lerpf(0.25, 1.0, intensity)
_impact_sparks.restart()
ball_contact.emit(intensity, contact_pos)
+18
View File
@@ -8,6 +8,13 @@ extends RefCounted
# "do not fork this logic" role for observations; the train/inference seam
# broke once before over exactly this kind of divergence (commit 8c15c46).
#
# Ship thrust is body-local, so its axes must not be mirrored for team 1.
# Ship rotation, however, is applied directly as world-space torque in
# Ship.apply_rotation_forces(). Team 1 observes a canonical frame rotated
# 180 degrees about world Y, so its canonical pitch/roll outputs must be
# rotated back to world space before they reach the ship. apply_team_frame()
# is the shared training/inference seam for that conversion.
#
# Curriculum generation 4 replaces the old continuous Gaussian action space
# (Box(7), see the "continuous" path below) with a per-axis MultiDiscrete
# space: PPO's Gaussian std reliably collapsed to ~0.13-0.15 within the first
@@ -100,6 +107,17 @@ static func from_logits(logits: Array, noise: float) -> ShipAction:
return result
# Map a policy's canonical-frame rotation intent back into the physical
# team's world frame. A 180-degree Y rotation negates X and Z and leaves Y
# unchanged. Translation remains untouched because Ship applies it through
# the ship's local basis rather than as a world-space vector.
static func apply_team_frame(action: ShipAction, team: int) -> ShipAction:
if team == 1:
action.rotation.x = -action.rotation.x
action.rotation.z = -action.rotation.z
return action
# Legacy continuous decode — moved verbatim from ai_ship_controller.gd so
# every model exported before generation 4 (no "action_space" block in its
# JSON, e.g. Game/bots/promoted/easy.json) keeps behaving byte-identically.
+70 -1
View File
@@ -39,6 +39,13 @@ extends AIController3D
# the objective" framing.
@export_range(0.0, 1.0) var ball_touch_direction_floor := 0.3
@export var velocity_to_ball_weight := 0.02
# Dense reward for approaching the ball *nose first* near the floor. Unlike
# velocity_to_ball_weight, sideways/reverse closing velocity earns nothing:
# the planar ship-forward vector must face the ball and planar velocity must
# have a positive component along it. Default off so existing curricula and
# frozen checkpoints keep their original objective; generation 5 handling
# turns it on while reducing the orientation-agnostic term.
@export var forward_velocity_to_ball_weight := 0.0
@export var ball_velocity_to_goal_weight := 0.004
# Per-tick penalty scaled by distance to the ball (full value at the arena's
# far diagonal, 0 on top of the ball). Run04 lesson: with idling worth a flat
@@ -62,6 +69,11 @@ extends AIController3D
# to teach. Not removed outright — an always-inverted bot still looks bad in
# a shipped game.
@export var tilt_penalty := 0.0005
# Additional tilt cost that fades to zero over the first few metres above the
# floor. This can teach readable, upright ground handling without opposing
# pitch/roll during a real aerial. Generation 5 uses this instead of raising
# the global tilt_penalty back to its pre-flight value.
@export var ground_tilt_penalty := 0.0
# Per-tick bonus for own speed: 0 stationary, full value (+0.24/s) at
# max_speed. Run07 lesson: after the kickoff flurry both ships parked next to
# a cornered ball — with every other dense term near zero there, standing
@@ -90,6 +102,15 @@ extends AIController3D
# air-touch reward.
const AIR_TOUCH_HEIGHT := 5.0
# Generation-5 ground-handling telemetry/reward thresholds. Fixed constants
# keep the logged metrics comparable across stages; changing one starts a new
# metric definition and therefore requires a fresh baseline.
const GROUND_HANDLING_HEIGHT := 3.0
const UPRIGHT_DOT_THRESHOLD := 0.7
const FORWARD_MOTION_DOT_THRESHOLD := 0.7
const MIN_HANDLING_SPEED := 1.0
const PRODUCTIVE_AIR_TOUCH_ALIGNMENT := 0.5
# Contact normals with y above this are floor contact (exempt from the wall
# penalty); below it they read as wall (sideways) or ceiling (downward).
# Mirrors ShipObservations.FLOOR_NORMAL_MIN_Y (see that file's comment).
@@ -144,6 +165,11 @@ var _altitude_sum := 0.0
var _thrust_y_sum := 0.0
var _touches := 0
var _air_touches := 0
var _productive_air_touches := 0
var _ground_ticks := 0
var _upright_ground_ticks := 0
var _moving_ground_ticks := 0
var _forward_moving_ground_ticks := 0
# Wire up references after the ship is spawned. `attack_goal` is the goal
@@ -201,6 +227,9 @@ func get_info() -> Dictionary:
info["mean_altitude"] = _altitude_sum / _telemetry_ticks if _telemetry_ticks > 0 else 0.0
info["vertical_thrust_mean"] = _thrust_y_sum / _telemetry_ticks if _telemetry_ticks > 0 else 0.0
info["air_touch_fraction"] = float(_air_touches) / _touches if _touches > 0 else 0.0
info["productive_air_touch_fraction"] = float(_productive_air_touches) / _touches if _touches > 0 else 0.0
info["upright_fraction"] = float(_upright_ground_ticks) / _ground_ticks if _ground_ticks > 0 else 0.0
info["forward_motion_fraction"] = float(_forward_moving_ground_ticks) / _moving_ground_ticks if _moving_ground_ticks > 0 else 0.0
return info
@@ -209,7 +238,9 @@ func get_action_space() -> Dictionary:
func set_action(action) -> void:
rl_controller.action = ShipActionCodec.from_indices(action)
rl_controller.action = ShipActionCodec.apply_team_frame(
ShipActionCodec.from_indices(action), ship.team
)
func reset():
@@ -221,6 +252,11 @@ func reset():
_thrust_y_sum = 0.0
_touches = 0
_air_touches = 0
_productive_air_touches = 0
_ground_ticks = 0
_upright_ground_ticks = 0
_moving_ground_ticks = 0
_forward_moving_ground_ticks = 0
func _physics_process(delta):
@@ -238,6 +274,20 @@ func _physics_process(delta):
var closing_speed := ship.linear_velocity.dot(to_ball.normalized())
reward += velocity_to_ball_weight * closing_speed / ship.max_speed
# Ground-handling shaping: forward planar motion while the nose faces the
# ball. It fades out with altitude so an aerial remains free to approach a
# ball using whatever body attitude is effective.
if forward_velocity_to_ball_weight > 0.0 and ship.global_position.y < GROUND_HANDLING_HEIGHT:
var planar_forward := Vector3(-ship.global_transform.basis.z.x, 0.0, -ship.global_transform.basis.z.z)
var planar_velocity := Vector3(ship.linear_velocity.x, 0.0, ship.linear_velocity.z)
var planar_to_ball := Vector3(to_ball.x, 0.0, to_ball.z)
if planar_forward.length_squared() > 0.0001 and planar_to_ball.length_squared() > 0.0001:
planar_forward = planar_forward.normalized()
var facing_ball: float = maxf(planar_forward.dot(planar_to_ball.normalized()), 0.0)
var forward_speed: float = maxf(planar_velocity.dot(planar_forward), 0.0) / ship.max_speed
var handling_ground_factor: float = 1.0 - clampf(ship.global_position.y / GROUND_HANDLING_HEIGHT, 0.0, 1.0)
reward += forward_velocity_to_ball_weight * forward_speed * facing_ball * handling_ground_factor
# Dense penalty: distance to the ball, so idling far away bleeds reward
# instead of scoring a safe zero (see ball_distance_penalty).
if ball_distance_penalty > 0.0:
@@ -268,6 +318,12 @@ func _physics_process(delta):
var uprightness: float = ship.global_transform.basis.y.dot(Vector3.UP)
reward -= tilt_penalty * (1.0 - uprightness) * 0.5
# Low-altitude-only posture pressure (see ground_tilt_penalty).
if ground_tilt_penalty > 0.0 and ship.global_position.y < GROUND_HANDLING_HEIGHT:
var ground_uprightness: float = ship.global_transform.basis.y.dot(Vector3.UP)
var tilt_ground_factor: float = 1.0 - clampf(ship.global_position.y / GROUND_HANDLING_HEIGHT, 0.0, 1.0)
reward -= ground_tilt_penalty * (1.0 - ground_uprightness) * 0.5 * tilt_ground_factor
# Dense penalty: height above the floor (see airborne_penalty). The
# floor sits at world y = 0 (see training_mode.gd's FIELD_MIN_Y/
# _escaped bounds); normalized so the worst case is pinned at the
@@ -283,6 +339,17 @@ func _physics_process(delta):
if ship.global_position.y > AIRBORNE_ALTITUDE_THRESHOLD:
_airborne_ticks += 1
_thrust_y_sum += rl_controller.action.thrust.y
if ship.global_position.y < GROUND_HANDLING_HEIGHT:
_ground_ticks += 1
if ship.global_transform.basis.y.dot(Vector3.UP) >= UPRIGHT_DOT_THRESHOLD:
_upright_ground_ticks += 1
var planar_velocity := Vector3(ship.linear_velocity.x, 0.0, ship.linear_velocity.z)
if planar_velocity.length() >= MIN_HANDLING_SPEED:
_moving_ground_ticks += 1
var planar_forward := Vector3(-ship.global_transform.basis.z.x, 0.0, -ship.global_transform.basis.z.z)
if planar_forward.length_squared() > 0.0001 \
and planar_velocity.normalized().dot(planar_forward.normalized()) >= FORWARD_MOTION_DOT_THRESHOLD:
_forward_moving_ground_ticks += 1
func _wall_or_ceiling_contact() -> bool:
@@ -309,3 +376,5 @@ func _on_ship_body_entered(body: Node) -> void:
_touches += 1
if ball.global_position.y > AIR_TOUCH_HEIGHT:
_air_touches += 1
if alignment >= PRODUCTIVE_AIR_TOUCH_ALIGNMENT:
_productive_air_touches += 1
+1 -6
View File
@@ -183,9 +183,6 @@ func _update_speed_feel(delta: float) -> void:
_effective_distance = lerpf(
_effective_distance, camera_distance + _turbo_blend * turbo_distance_kick, feel_t
)
post_material.set_shader_parameter(
"motion_blur_amount", _speed_blend * _speed_blend * 0.006 + _turbo_blend * 0.006
)
post_material.set_shader_parameter("chromatic_aberration", _turbo_blend * 0.007)
post_material.set_shader_parameter("vignette_strength", 0.22 + _turbo_blend * 0.16)
@@ -233,8 +230,7 @@ func begin_goal_cut(goal_position: Vector3) -> void:
camera.fov = 58.0
camera.look_at(_goal_cut_look_at, Vector3.UP)
# A hard, stationary cinematic camera must not inherit the scoring frame's
# ship-speed smear or turbo chroma. Keep only a deliberate cinematic edge.
post_material.set_shader_parameter("motion_blur_amount", 0.0)
# turbo chroma; keep only a deliberate cinematic edge.
post_material.set_shader_parameter("chromatic_aberration", 0.0)
post_material.set_shader_parameter("vignette_strength", 0.3)
@@ -243,6 +239,5 @@ func end_goal_cut() -> void:
_goal_cut_active = false
_speed_blend = 0.0
_turbo_blend = 0.0
post_material.set_shader_parameter("motion_blur_amount", 0.0)
post_material.set_shader_parameter("chromatic_aberration", 0.0)
post_material.set_shader_parameter("vignette_strength", 0.22)
+77 -2
View File
@@ -64,6 +64,11 @@ extends GameMode
# stayed floor-pinned even though the initial one wasn't. See
# _place_air_drill.
@export_range(0.0, 1.0) var air_drill_chance := 0.0
# Moving-ball aerial interception branch used by generation 5. Unlike the
# stationary/random air drill, the ball follows a reachable trajectory toward
# a real goal and ships start low behind/lateral to it, so a useful touch is
# naturally reinforced by the existing goal-directed ball rewards.
@export_range(0.0, 1.0) var air_intercept_chance := 0.0
# Ships per team. Default 1 preserves every existing curriculum script's 1v1
# behaviour unchanged; up to 5 matches ShipObservations.MAX_TEAMMATES/
@@ -129,6 +134,8 @@ var _episode_ticks := 0
# already uses, just for one side of a live training episode.
var _opponent_mode := "self_play"
var _opponent_model_path := ""
var _opponent_model_pool: Array[String] = []
var _frozen_opponent_bots: Array[AIShipController] = []
# ShipAIController @export overrides collected from --ai_<name>=<value> args,
# applied to every ShipAIController this run creates (see _attach_agent).
var _ai_overrides := {}
@@ -164,7 +171,7 @@ func _start() -> void:
team0_ships.append(spawn_ship(0, i, RLShipController.new()))
# The opponent_mode branch applies uniformly to every ship on team 1: an
# "inert"/"frozen" run means the whole opposing team gets that treatment,
# "inert"/"frozen"/"league" run means the whole opposing team gets that treatment,
# not just one ship.
var team1_ships: Array[Ship] = []
for i in team_size:
@@ -177,6 +184,12 @@ func _start() -> void:
var bot := AIShipController.new()
bot.model_path = _opponent_model_path
ship1 = spawn_ship(1, i, bot)
_frozen_opponent_bots.append(bot)
"league":
var bot := AIShipController.new()
bot.model_path = _opponent_model_pool[0] if not _opponent_model_pool.is_empty() else ""
ship1 = spawn_ship(1, i, bot)
_frozen_opponent_bots.append(bot)
_:
ship1 = spawn_ship(1, i, RLShipController.new())
team1_ships.append(ship1)
@@ -230,13 +243,15 @@ func _parse_eval_args() -> void:
const TRAINING_MODE_OVERRIDES := [
"goal_reward", "draw_penalty", "kickoff_state_chance",
"ball_near_goal_chance", "attack_goal_bias", "air_drill_chance",
"air_intercept_chance", "team_size",
]
# ShipAIController @export names a curriculum run may override, read as
# --ai_<name>=<value> to avoid colliding with the names above.
const SHIP_AI_OVERRIDES := [
"ball_touch_reward", "ball_touch_cooldown_ticks", "ball_touch_direction_floor",
"velocity_to_ball_weight", "ball_velocity_to_goal_weight", "ball_distance_penalty",
"wall_contact_penalty", "tilt_penalty", "speed_reward_weight", "time_penalty",
"forward_velocity_to_ball_weight", "wall_contact_penalty", "tilt_penalty",
"ground_tilt_penalty", "speed_reward_weight", "time_penalty",
"airborne_penalty",
]
@@ -246,10 +261,20 @@ func _parse_curriculum_args() -> void:
if args.has("opponent_mode"):
_opponent_mode = args["opponent_mode"]
_opponent_model_path = args.get("opponent_model", _opponent_model_path)
if args.has("opponent_model_pool"):
for path in String(args["opponent_model_pool"]).split(",", false):
if not path.is_empty():
_opponent_model_pool.append(path)
if _opponent_mode == "league" and _opponent_model_pool.is_empty():
push_error("TrainingMode: opponent_mode=league requires --opponent_model_pool=path,path")
for name in TRAINING_MODE_OVERRIDES:
if args.has(name):
set(name, _typed_like(args[name], get(name)))
var start_probability := kickoff_state_chance + ball_near_goal_chance \
+ air_drill_chance + air_intercept_chance
if start_probability > 1.0:
push_error("TrainingMode: episode-start probabilities sum to %.3f (> 1.0)" % start_probability)
for name in SHIP_AI_OVERRIDES:
var key := "ai_%s" % name
@@ -281,10 +306,12 @@ func _ai_default(name: String) -> Variant:
"ball_touch_cooldown_ticks": return 60
"ball_touch_direction_floor": return 0.3
"velocity_to_ball_weight": return 0.02
"forward_velocity_to_ball_weight": return 0.0
"ball_velocity_to_goal_weight": return 0.004
"ball_distance_penalty": return 0.002
"wall_contact_penalty": return 0.0025
"tilt_penalty": return 0.0005
"ground_tilt_penalty": return 0.0
"speed_reward_weight": return 0.004
"time_penalty": return 0.001
"airborne_penalty": return 0.0
@@ -386,6 +413,7 @@ func _end_eval_episode() -> void:
func _reset_episode() -> void:
for agent in _agents:
agent.reset()
_select_league_opponent()
var roll := randf()
if roll < kickoff_state_chance:
@@ -396,6 +424,8 @@ func _reset_episode() -> void:
_place_ball_near_goal()
elif roll < kickoff_state_chance + ball_near_goal_chance + air_drill_chance:
_place_air_drill()
elif roll < kickoff_state_chance + ball_near_goal_chance + air_drill_chance + air_intercept_chance:
_place_air_intercept()
else:
_place_ships_random()
_place_ball_random()
@@ -416,6 +446,15 @@ func _place_ball_random() -> void:
# out via reward shaping.
const AIR_DRILL_BALL_WALL_CLEARANCE := 5.0
func _select_league_opponent() -> void:
if _opponent_mode != "league" or _opponent_model_pool.is_empty():
return
var path := _opponent_model_pool[randi() % _opponent_model_pool.size()]
for bot in _frozen_opponent_bots:
bot.load_policy(path)
# Air drill state (see air_drill_chance): ball spawned high, both ships
# spawned low and lateral, so the state is unsolvable without climbing.
func _place_air_drill() -> void:
@@ -456,6 +495,42 @@ func _place_air_drill() -> void:
_place_body(ship, Transform3D(orientation, ship_position), Vector3.ZERO, Vector3.ZERO)
# Goal-relevant aerial intercept: a high ball is already travelling toward a
# randomly selected goal, while ships begin low and behind/lateral to its
# path. The generous wall clearance prevents rebound farming and an upright
# yaw-only spawn avoids wasting the short drill window on random recovery.
func _place_air_intercept() -> void:
var goal := _goal_for_team(randi() % 2)
var ball_position := Vector3(
randf_range(-8.0, 8.0),
randf_range(6.0, minf(12.0, FIELD_MAX_Y)),
randf_range(-10.0, 10.0)
)
var to_goal := (goal.global_position - ball_position).normalized()
var ball_velocity := (to_goal + Vector3(randf_range(-0.15, 0.15), randf_range(0.0, 0.15), 0.0)).normalized() \
* randf_range(6.0, 11.0)
_place_body(ball, Transform3D(Basis.IDENTITY, ball_position), ball_velocity, Vector3.ZERO)
var placed: Array[Vector3] = []
var behind := -Vector3(ball_velocity.x, 0.0, ball_velocity.z).normalized()
for ship in ships:
if ship in _inert_ships:
continue
var ship_position := Vector3.ZERO
for _attempt in 20:
var lateral := Vector3(-behind.z, 0.0, behind.x) * randf_range(-7.0, 7.0)
ship_position = ball_position + behind * randf_range(7.0, 13.0) + lateral
ship_position.x = clampf(ship_position.x, -FIELD_HALF_X, FIELD_HALF_X)
ship_position.y = randf_range(FIELD_MIN_Y, 3.0)
ship_position.z = clampf(ship_position.z, -FIELD_HALF_Z, FIELD_HALF_Z)
if _spawn_position_clear(ship_position) and _far_enough_from(ship_position, placed):
break
placed.append(ship_position)
var face_ball := ball_position - ship_position
var yaw := atan2(-face_ball.x, -face_ball.z)
_place_body(ship, Transform3D(Basis.from_euler(Vector3(0.0, yaw, 0.0)), ship_position), Vector3.ZERO, Vector3.ZERO)
# Attacking/defending drill states: ball close to a goal, moving toward it.
# Which goal is picked is biased by attack_goal_bias (0.5 = uniform between
# both, matching historical behaviour; 1.0 = always the goal team 0 attacks).
+5 -21
View File
@@ -1,37 +1,21 @@
shader_type canvas_item;
render_mode unshaded;
uniform sampler2D screen_texture : hint_screen_texture, filter_linear_mipmap;
uniform float motion_blur_amount : hint_range(0.0, 0.03) = 0.0;
uniform sampler2D screen_texture : hint_screen_texture, filter_linear;
uniform float chromatic_aberration : hint_range(0.0, 0.02) = 0.0;
uniform float vignette_strength : hint_range(0.0, 1.0) = 0.22;
vec4 motion_sample(vec2 uv) {
// A restrained centre-directed velocity smear. The camera script scales it
// with ship speed/turbo; mipmapped screen reads soften the outer samples.
if (motion_blur_amount < 0.0001) {
return textureLod(screen_texture, uv, 0.0);
}
vec2 step_uv = (uv - vec2(0.5)) * motion_blur_amount;
vec4 color = textureLod(screen_texture, uv, 0.0) * 0.36;
color += textureLod(screen_texture, uv - step_uv * 0.35, 0.5) * 0.25;
color += textureLod(screen_texture, uv - step_uv * 0.7, 1.0) * 0.21;
color += textureLod(screen_texture, uv - step_uv, 1.5) * 0.18;
return color;
}
void fragment() {
vec2 radial = UV - vec2(0.5);
vec2 chroma = radial * chromatic_aberration;
vec4 center = motion_sample(UV);
vec4 center = texture(screen_texture, UV);
vec3 color = center.rgb;
// The branch is driven by one uniform for the whole draw, so non-turbo
// frames avoid eight redundant screen taps without causing warp divergence.
// frames avoid two redundant screen taps without causing warp divergence.
if (chromatic_aberration > 0.0001) {
color.r = motion_sample(UV + chroma).r;
color.b = motion_sample(UV - chroma).b;
color.r = texture(screen_texture, UV + chroma).r;
color.b = texture(screen_texture, UV - chroma).b;
}
// Aspect-correct vignette, always subtle and tightened slightly on turbo.
-29
View File
@@ -1,29 +0,0 @@
[gd_resource type="Theme" load_steps=3 format=3]
[ext_resource type="FontFile" path="res://assets/fonts/Orbitron-Medium.ttf" id="1_font"]
[sub_resource type="StyleBoxFlat" id="StyleBoxFlat_panel"]
bg_color = Color(0.035, 0.05, 0.085, 0.76)
border_width_left = 1
border_width_top = 1
border_width_right = 1
border_width_bottom = 1
border_color = Color(0.3, 0.7, 1, 0.38)
corner_radius_top_left = 8
corner_radius_top_right = 8
corner_radius_bottom_right = 8
corner_radius_bottom_left = 8
content_margin_left = 12.0
content_margin_top = 8.0
content_margin_right = 12.0
content_margin_bottom = 8.0
[resource]
default_font = ExtResource("1_font")
default_font_size = 14
Label/colors/font_color = Color(0.86, 0.94, 1, 1)
Label/colors/font_shadow_color = Color(0, 0, 0, 0.72)
Label/constants/outline_size = 1
Label/constants/shadow_offset_x = 2
Label/constants/shadow_offset_y = 2
PanelContainer/styles/panel = SubResource("StyleBoxFlat_panel")
+9 -7
View File
@@ -6,9 +6,11 @@ Deferred work, in rough priority order. The current architecture (ShipAction/Shi
The training pipeline is built — see `TRAINING.md` (self-play PPO via the vendored godot_rl_agents bridge, JSON policy export, in-game GDScript inference, eval ladder). Remaining:
- [ ] Long training runs on the Linux/3090 box to produce actually-good bots; promote further checkpoints into `Game/bots/promoted/` as `medium`/`hard` tiers once they clear `easy.json` in `evaluate.py`.
- [ ] Frozen-opponent league: train the live policy against a *pool* of past exported checkpoints, sampled per-episode (today's `--opponent-mode=frozen` only supports one fixed model per run) to prevent self-play strategy collapse on long runs.
- [ ] Richer state setter / curriculum: aerial states, wall plays, rebound scenarios as skill grows (beyond the score/defend/draw staging already in place).
- [x] Promote generation 4's stage-3 gauntlet policy as the new `Game/bots/promoted/easy.json` baseline.
- [ ] Run the generation-5 handling/intercepts/league/teamplay curriculum described in `TRAINING.md`; promote later checkpoints as `medium`/`hard` only after they clear the match and behaviour gates.
- [x] Frozen-opponent league plumbing: `--opponent-mode=league` samples a past exported checkpoint per episode; generation 5 Stage 6 supplies the curated pool.
- [ ] Extend generation 5's moving aerial-intercept states with wall plays and rebound scenarios after Stage 5 establishes a productive-air-touch baseline.
- [ ] Design team-credit rewards and paired 2v2 evaluation before enabling the deferred teamplay stage.
## Presentation / AAA polish
@@ -16,14 +18,14 @@ The largest gap between this and a AAA-feeling product is presentation, not code
- [ ] **Audio — there is none.** Zero sound files, zero `AudioStreamPlayer` nodes, no bus layout. Needs: engine hum pitched to throttle, turbo whoosh, ball impacts scaled by collision impulse, wall scrapes, goal explosion, crowd bed, UI clicks, countdown beeps, music. Can be driven off `Ship`'s existing telemetry signals.
- [x] SSAO/SSIL in the arena Environments — cheapest single perceived-quality win available; grounds the ships against the deck and gives the fillets and goal recesses real depth.
- [x] VFX on anything that moves: throttle-reactive engine cores and plumes, distinct turbo flames, a speed-scaled ball trail, contact sparks and team-tinted goal bursts.
- [x] Impact feedback — intensity-scaled screen shake, unscaled hit-stop, HUD flash and controller rumble on ball contact.
- [x] VFX on anything that moves: throttle-reactive engine cores and simple rear flames, a speed-scaled ball trail and team-tinted goal bursts.
- [x] Impact feedback — intensity-scaled screen shake, unscaled hit-stop and controller rumble on ball contact.
- [x] Camera feel in `ship_camera.gd` — speed-based FOV widening, turbo kick-back and decaying impact shake.
- [x] Goal celebration sequence: goal burst, team-tinted flash/title card, dedicated camera cut and slow-motion beat after authoritative score registration and before reset/kickoff.
- [x] Local lighting / dynamic GI — four local pitch lights plus SDFGI provide bounce and contact lighting for the runtime-generated arena shell; the shader's `hull_fill` stand-in is retired. (The shell is generated in `_ready`, so it cannot participate in an editor LightmapGI bake.)
- [x] Dress `arena_03` — its asteroid field now has mining stations, debris clusters and a distant planet.
- [x] Custom font + a real `Theme` resource for the HUD. Packaged Orbitron typography, shared label styling and themed procedural-instrument text replace `ThemeDB.fallback_font`.
- [x] Post-processing beyond glow: camera DoF plus speed-scaled motion blur, a cinematic vignette and turbo-driven chromatic aberration.
- [ ] Custom font + a real `Theme` resource for the HUD. `ThemeDB.fallback_font` at 10-13 px reads as a debug overlay.
- [x] Post-processing beyond glow: a cinematic vignette and turbo-driven chromatic aberration. DoF and motion blur were intentionally omitted after playtesting.
## Multiplayer (long term)
+84 -9
View File
@@ -162,11 +162,9 @@ flat files there, never touching subdirectories.
`Game/bots/promoted/<tier>.json` is the small, curated, hand-maintained set
actually referenced by the shipped game — currently `easy.json` (promoted
2026-07-24 from `curric-s6-unmask`, the strongest checkpoint at the
time — note `curric-s6-unmask` was itself generation 1's *failed* unmask
stage, so `easy.json` is weaker than `reference-grounded.json` below; a
strong generation 4 result should promote a real replacement, plus
`medium.json`/`hard.json`) and `reference-grounded.json` (added for
2026-08-08 from generation 4's `20260806-1939-curric-s3-gauntlet`; this is
the 320M-step MultiDiscrete policy and the foundation for the planned
generation-5 curriculum below) and `reference-grounded.json` (added for
generation 4 — a copy of generation 3's `curric-s5-aggression`, made before
the flat `Game/bots/` dump was scrapped for the redesign, kept as the
strongest grounded-era artifact and the fixed yardstick generations 1-3 were
@@ -176,6 +174,11 @@ promoted file is never touched by training scripts, never overwritten by a
same-named future export, and never disturbed by pruning old experiment
files from the flat dump.
Until distinct `medium.json` and `hard.json` policies earn promotion, the
three menu tiers all run this same `easy.json` policy at its full trained
cadence (`reaction_ticks=8`, `action_noise=0`). The tiers are labels only;
the game does not manufacture difficulty gaps by handicapping this model.
To promote a new bot into a tier: copy the chosen `Game/bots/<experiment>.json`
to `Game/bots/promoted/<tier>.json` (overwriting the old one), and note the
source experiment + date in this section. Do this for `medium.json`/
@@ -493,10 +496,13 @@ alone (`train.py` only forwards a flag when you pass it), so ordinary runs
are unaffected. Full flag list: `--opponent-mode {self_play,inert,frozen}`,
`--opponent-model <path>` (for `frozen`), `--draw-penalty`,
`--attack-goal-bias`, `--kickoff-chance`, `--near-goal-chance`,
`--air-drill-chance` (generation 4's state-setter aerial curriculum),
`--velocity-to-ball-weight`, `--ball-distance-penalty`, `--ball-touch-reward`,
`--airborne-penalty`, `--tilt-penalty`, `--ball-velocity-to-goal-weight`,
`--goal-reward`. (`--vertical-ramp`/`--pitch-roll-ramp` are gone — generation
`--air-drill-chance`, `--air-intercept-chance`, `--team-size`,
`--velocity-to-ball-weight`, `--forward-velocity-to-ball-weight`,
`--ball-distance-penalty`, `--ball-touch-reward`, `--airborne-penalty`,
`--tilt-penalty`, `--ground-tilt-penalty`, `--speed-reward-weight`,
`--ball-velocity-to-goal-weight`, `--goal-reward`, and
`--opponent-pool` with `--opponent-mode=league`.
(`--vertical-ramp`/`--pitch-roll-ramp` are gone — generation
4 has no locomotion mask/ramp to control.)
### Running it automatically
@@ -546,6 +552,75 @@ Running a stage by hand (e.g. to experiment with flags before trusting the
orchestrator) still works exactly as the table above describes — just call
`next_run.sh`/`run_training.sh` directly with that stage's flags.
### Generation 5 follow-on
Generation 4's stage-3 export is the foundation rather than a throwaway
baseline: all generation-5 stages resume from
`checkpoints/20260806-1939-curric-s3-gauntlet/final.zip`. Its match results
are strong, but playtesting and its final telemetry expose the next learning
targets: it spends about 39% of play above the airborne threshold while only
about 0.04% of episode-level touches are aerial, and it often travels on its
side and strikes the ball with its roof. This is a successful scoring policy
that now needs control quality and a more productive use of flight.
Turbo remains forward-only for players and policies: it activates only with
positive forward thrust and multiplies the resulting combined thrust vector.
Generation 5 preserves the same control contract Stage 3 was trained under.
Generation 5 adds three episode telemetry signals to TensorBoard:
`upright_fraction` (low-altitude ticks with the
ship's up vector substantially upright), `forward_motion_fraction`
(low-altitude moving ticks whose planar velocity points broadly along the
nose), and `productive_air_touch_fraction` (touches above the aerial height
that send the ball toward the attack goal). The automatic gates are
deliberately conservative catastrophe floors; every stage records its final
500-rollout tail means in `generation5_state.json` so later threshold changes
can be based on evidence instead of a single watched match.
| Stage | Regime | Budget | Learning target | Advancement gate |
|---|---|---:|---|---|
| 4 — `handling` | Self-play, current balanced start mix | 40M (~4h) | Prefer upright, nose-led travel near the floor. Replace the orientation-agnostic speed bonus with low-altitude forward-motion shaping, and apply the stronger tilt cost only near the floor so pitch/roll remain free in genuine aerial play. | Before Stage 5: at least 80% training goal rate, at least 80% non-draw rate in the paired evaluation versus promoted Stage 3, no clear head-to-head regression, no more than 20% physical-side win imbalance, and the upright/forward-motion telemetry floors. |
| 5 — `intercepts` | Self-play with 4050% improved air-intercept starts | 60M (~6h) | Convert existing vertical movement into useful aerial touches. Spawn a moving high ball on reachable attacking and defensive trajectories, away from walls, so contact is instrumental to scoring or saving rather than independently rewarded. | No clear regression versus Stage 4; productive aerial-touch telemetry must improve materially without reducing upright/forward-motion telemetry back to the Stage-3 baseline. |
| 6 — `league` | Live policy against a frozen opponent sampled per episode from Stage 3, Stage 4, and Stage 5 | 100M (~10h) | Prevent a narrow self-play equilibrium and consolidate ground handling, aerial interception, attack, and defence against distinct styles. | No clear head-to-head regression against any pool member plus conservative handling/aerial telemetry floors. Promote the passing result to `medium.json` after these recorded evaluations support it. |
Stage 7 teamplay remains deliberately unconfigured. The fixed roster
observation and `team_size` plumbing can run 2v2, but there is no paired 2v2
evaluation or team-credit reward yet; spending 120M steps without those gates
would make a pass meaningless.
`training/generation5.py` implements Stages 46 separately from the completed
generation-4 orchestrator and state. It always begins Stage 4 from
`checkpoints/20260806-1939-curric-s3-gauntlet/final.zip`, then resumes each
later stage from its passing predecessor. `generation5.sh` runs it detached,
and retries/blocks use the same restart-safe pattern as the earlier
curriculum:
```bash
cd training
.venv/bin/python generation5.py --dry-run # print and validate the next command only
./generation5.sh # run/resume in tmux
tmux attach -t cosmic-generation5
cat generation5_state.json
```
Stage 4 removes the generic speed bonus, halves the old orientation-agnostic
closing reward, and adds a nose-led planar approach reward plus a tilt cost
that fades to zero by 3m altitude. Its scoring gates deliberately run before
Stage 5: becoming upright is not progress if the resulting policy stops
finishing goals. Stage 5 adds moving high-ball intercept
starts aimed toward real goals rather than a standalone air-touch reward.
Stage 6's `league` opponent mode samples a historical exported policy at each
episode reset. Each later stage preserves the preceding shaping and adds one
new difficulty.
The physical-side gate is separate from the model-vs-model score. A paired
side swap can make an identical policy appear perfectly balanced overall even
when the Player 2 ship never functions. This caught the canonical action-frame
bug exposed by Stage 3's pitch/roll use: team 1 observations are rotated 180°
about Y, but rotation commands feed world-space torque, so team 1 pitch and
roll must be rotated back (X/Z signs inverted). Thrust remains unchanged
because it is applied through the ship's local basis.
## Self-play notes
By default both ships share the live policy (mirrored, team-relative
+85 -28
View File
@@ -2,8 +2,9 @@
Uses the same in-Godot inference path that ships in the game
(AIShipController + PolicyNetwork), so eval strength = in-game strength.
Episodes are golden-goal: first goal wins, timeout is a draw. Half the
episodes are played with sides swapped for fairness. Results are appended to
Episodes are golden-goal: first goal wins, timeout is a draw. Every randomized
starting state is played twice with sides swapped, so physical-team or arena
asymmetry cannot be mistaken for model strength. Results are appended to
eval_history.json — the bot-progress-over-time record.
Example:
@@ -47,9 +48,9 @@ def run_half(
f"--env_seed={seed}",
]
# Must match how each model was actually trained (see AIShipController's
# allow_vertical/allow_pitch_roll) — a curriculum stage 1/2 model never
# got a reward gradient on these axes, so leaving them unmasked here adds
# untrained aerial noise the model's own training never had to contend with.
# allow_vertical/allow_pitch_roll) — a grounded pre-generation-4 model
# never got a reward gradient on these axes, so leaving them unmasked here
# adds untrained aerial noise its training never had to contend with.
if grounded_a:
cmd += ["--eval_allow_vertical_a=false", "--eval_allow_pitch_roll_a=false"]
if grounded_b:
@@ -62,48 +63,95 @@ def run_half(
raise RuntimeError(f"No EVAL_RESULT in godot output:\n{result.stdout[-2000:]}\n{result.stderr[-2000:]}")
def evaluate_pair(
godot_bin: str,
model_a: str,
model_b: str,
episodes: int,
speedup: int,
seed: int,
grounded_a: bool = False,
grounded_b: bool = False,
) -> dict:
"""Replay one seeded state sequence with the models on opposite sides."""
if episodes < 2 or episodes % 2 != 0:
raise ValueError("--episodes must be an even number of at least 2 for paired side swaps")
episodes_per_side = episodes // 2
first = run_half(
godot_bin, model_a, model_b, episodes_per_side, speedup, seed,
grounded_a=grounded_a, grounded_b=grounded_b,
)
second = run_half(
godot_bin, model_b, model_a, episodes_per_side, speedup, seed,
grounded_a=grounded_b, grounded_b=grounded_a,
)
a_team_0 = {
"wins_a": first["goals_a"],
"wins_b": first["goals_b"],
"draws": first["draws"],
}
a_team_1 = {
"wins_a": second["goals_b"],
"wins_b": second["goals_a"],
"draws": second["draws"],
}
record = {
"timestamp": datetime.datetime.now(datetime.timezone.utc).isoformat(timespec="seconds"),
"model_a": model_a,
"model_b": model_b,
"seed": seed,
"episodes": first["episodes"] + second["episodes"],
"wins_a": a_team_0["wins_a"] + a_team_1["wins_a"],
"wins_b": a_team_0["wins_b"] + a_team_1["wins_b"],
"draws": a_team_0["draws"] + a_team_1["draws"],
"side_results": {
"a_team_0": a_team_0,
"a_team_1": a_team_1,
},
"physical_team_wins": {
"team_0": first["goals_a"] + second["goals_a"],
"team_1": first["goals_b"] + second["goals_b"],
},
}
record["win_rate_a"] = round(record["wins_a"] / record["episodes"], 3)
return record
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("model_a", help="Path to first exported policy .json")
parser.add_argument("model_b", help="Path to second exported policy .json")
parser.add_argument("--episodes", type=int, default=20, help="Total episodes (split across side swap)")
parser.add_argument(
"--episodes", type=int, default=20,
help="Total episodes; must be even so every seeded state is replayed with sides swapped",
)
parser.add_argument(
"--godot_bin",
default=os.environ.get("GODOT_BIN", DEFAULT_GODOT_MACOS),
help="Path to the Godot binary (or set GODOT_BIN)",
)
parser.add_argument("--speedup", type=int, default=16)
parser.add_argument("--seed", type=int, default=1, help="Seed for the paired starting-state sequence")
parser.add_argument("--history", default=str(TRAINING_DIR / "eval_history.json"))
parser.add_argument(
"--grounded-a", action="store_true", help="model_a was trained with locomotion masked (curriculum stages 1-2)"
"--grounded-a", action="store_true", help="model_a was trained with locomotion masked (pre-generation-4 models)"
)
parser.add_argument(
"--grounded-b", action="store_true", help="model_b was trained with locomotion masked (curriculum stages 1-2)"
"--grounded-b", action="store_true", help="model_b was trained with locomotion masked (pre-generation-4 models)"
)
args = parser.parse_args()
model_a = str(pathlib.Path(args.model_a).resolve())
model_b = str(pathlib.Path(args.model_b).resolve())
half = max(args.episodes // 2, 1)
# Half the episodes on each side to cancel any residual side asymmetry;
# different seeds so the halves see different randomized episode states.
# Groundedness is per physical model, so it swaps sides along with it.
first = run_half(args.godot_bin, model_a, model_b, half, args.speedup, seed=1,
grounded_a=args.grounded_a, grounded_b=args.grounded_b)
second = run_half(args.godot_bin, model_b, model_a, half, args.speedup, seed=2,
grounded_a=args.grounded_b, grounded_b=args.grounded_a)
record = {
"timestamp": datetime.datetime.now(datetime.timezone.utc).isoformat(timespec="seconds"),
"model_a": model_a,
"model_b": model_b,
"episodes": first["episodes"] + second["episodes"],
"wins_a": first["goals_a"] + second["goals_b"],
"wins_b": first["goals_b"] + second["goals_a"],
"draws": first["draws"] + second["draws"],
}
record["win_rate_a"] = round(record["wins_a"] / record["episodes"], 3)
try:
record = evaluate_pair(
args.godot_bin, model_a, model_b, args.episodes, args.speedup, args.seed,
grounded_a=args.grounded_a, grounded_b=args.grounded_b,
)
except ValueError as error:
parser.error(str(error))
history_path = pathlib.Path(args.history)
history = json.loads(history_path.read_text()) if history_path.exists() else []
@@ -115,6 +163,15 @@ def main():
f"{record['wins_a']}-{record['wins_b']} ({record['draws']} draws), "
f"win rate A = {record['win_rate_a']:.0%}"
)
a_team_0 = record["side_results"]["a_team_0"]
a_team_1 = record["side_results"]["a_team_1"]
physical = record["physical_team_wins"]
print(
f"Paired seed {record['seed']} side split: "
f"A as team 0 {a_team_0['wins_a']}-{a_team_0['wins_b']} ({a_team_0['draws']} draws); "
f"A as team 1 {a_team_1['wins_a']}-{a_team_1['wins_b']} ({a_team_1['draws']} draws); "
f"physical teams 0-1 = {physical['team_0']}-{physical['team_1']}"
)
print(f"Appended to {history_path}")
+407
View File
@@ -0,0 +1,407 @@
"""Run the post-generation-4 curriculum from the promoted Stage-3 policy.
This is intentionally separate from curriculum.py/curriculum_state.json:
generation 4 is a completed lineage and its final checkpoint is generation
5's fixed foundation. Stages 4-6 add one difficulty at a time:
4 handling -- upright, nose-led low-altitude movement
5 intercepts -- useful moving-ball aerial interceptions
6 league -- robustness against a pool of frozen historical styles
Each stage resumes from its passing predecessor, exports through the normal
run_training.sh parity check, records tail telemetry, and runs a paired
100-episode regression evaluation. State is restart-safe in
generation5_state.json.
"""
from __future__ import annotations
import argparse
import json
import pathlib
import subprocess
import sys
from datetime import datetime
from tensorboard.backend.event_processing.event_accumulator import EventAccumulator
TRAINING_DIR = pathlib.Path(__file__).resolve().parent
REPO_ROOT = TRAINING_DIR.parent
STATE_PATH = TRAINING_DIR / "generation5_state.json"
EVAL_HISTORY_PATH = TRAINING_DIR / "eval_history.json"
FOUNDATION_EXPERIMENT = "20260806-1939-curric-s3-gauntlet"
FOUNDATION_CHECKPOINT = TRAINING_DIR / "checkpoints" / FOUNDATION_EXPERIMENT / "final.zip"
FOUNDATION_EXPORT = REPO_ROOT / "Game" / "bots" / f"{FOUNDATION_EXPERIMENT}.json"
PROMOTED_EASY = REPO_ROOT / "Game" / "bots" / "promoted" / "easy.json"
MAX_RETRIES = 2
EVAL_EPISODES = 100
REGRESSION_MARGIN = 0.15
STANDING_ARGS = ["--ent-coef", "0.01", "--entropy-floor"]
# Scoring/ball-direction shaping inherited from generation 4. Handling
# replaces half the orientation-agnostic closing reward and all generic speed
# reward with nose-led ground approach, while keeping global tilt pressure
# small enough for flight and adding a stronger floor-local term.
HANDLING_REWARD_FLAGS = [
"--velocity-to-ball-weight", "0.04",
"--forward-velocity-to-ball-weight", "0.06",
"--ball-distance-penalty", "0.01",
"--ball-touch-reward", "0.7",
"--ball-velocity-to-goal-weight", "0.06",
"--goal-reward", "80",
"--speed-reward-weight", "0.0",
"--tilt-penalty", "0.0002",
"--ground-tilt-penalty", "0.003",
]
STAGES = [
{
"number": 4,
"name": "handling",
"timesteps": 40_000_000,
"flags": [
"--opponent-mode", "self_play",
"--kickoff-chance", "0.15",
"--near-goal-chance", "0.25",
"--air-drill-chance", "0.20",
"--air-intercept-chance", "0.0",
*HANDLING_REWARD_FLAGS,
],
# Conservative catastrophe floors, not claims of mastery. Tail values
# are recorded in state so later thresholds can be based on evidence.
"telemetry_floors": {
"rollout/goal_rate": 0.80,
"rollout/upright_fraction": 0.45,
"rollout/forward_motion_fraction": 0.25,
},
# At least 80% of the paired candidate-vs-Stage-3 episodes must end
# in a goal. This is separate from win-rate regression: a draw-heavy
# handling policy must not advance merely because neither bot won.
"evaluation_goal_rate_floor": 0.80,
# The paired side swap also measures physical spawn/team bias. This
# catches a broken team-frame action mapping even when model A's
# aggregate result looks balanced because it plays both sides.
"physical_side_imbalance_ceiling": 0.20,
},
{
"number": 5,
"name": "intercepts",
"timesteps": 60_000_000,
"flags": [
"--opponent-mode", "self_play",
"--kickoff-chance", "0.10",
"--near-goal-chance", "0.20",
"--air-drill-chance", "0.10",
"--air-intercept-chance", "0.45",
*HANDLING_REWARD_FLAGS,
],
"telemetry_floors": {
"rollout/goal_rate": 0.75,
"rollout/upright_fraction": 0.40,
"rollout/forward_motion_fraction": 0.20,
"rollout/productive_air_touch_fraction": 0.005,
},
"evaluation_goal_rate_floor": 0.75,
"physical_side_imbalance_ceiling": 0.20,
},
{
"number": 6,
"name": "league",
"timesteps": 100_000_000,
"flags": [
"--opponent-mode", "league",
"--kickoff-chance", "0.15",
"--near-goal-chance", "0.25",
"--air-drill-chance", "0.15",
"--air-intercept-chance", "0.25",
*HANDLING_REWARD_FLAGS,
],
"telemetry_floors": {
"rollout/goal_rate": 0.70,
"rollout/upright_fraction": 0.35,
"rollout/forward_motion_fraction": 0.18,
"rollout/productive_air_touch_fraction": 0.003,
},
"evaluation_goal_rate_floor": 0.70,
"physical_side_imbalance_ceiling": 0.20,
"league_pool": True,
},
]
def fresh_state() -> dict:
return {"stage_index": 0, "attempt": 0, "status": "in_progress", "log": []}
def load_state() -> dict:
return json.loads(STATE_PATH.read_text()) if STATE_PATH.exists() else fresh_state()
def save_state(state: dict) -> None:
STATE_PATH.write_text(json.dumps(state, indent=2) + "\n")
def passing_entry(state: dict, stage_index: int) -> dict:
for entry in state["log"]:
if entry["stage_index"] == stage_index and entry["decision"] == "pass":
return entry
raise RuntimeError(f"No passing generation-5 stage index {stage_index}")
def previous_attempt_entry(state: dict, stage_index: int, attempt: int) -> dict:
for entry in reversed(state["log"]):
if entry["stage_index"] == stage_index and entry["attempt"] == attempt - 1:
return entry
raise RuntimeError(f"No previous attempt for stage index {stage_index}, attempt {attempt}")
def resume_checkpoint(state: dict, stage_index: int, attempt: int, foundation: pathlib.Path) -> pathlib.Path:
if attempt > 0:
exp = previous_attempt_entry(state, stage_index, attempt)["experiment"]
return TRAINING_DIR / "checkpoints" / exp / "final.zip"
if stage_index == 0:
return foundation
exp = passing_entry(state, stage_index - 1)["experiment"]
return TRAINING_DIR / "checkpoints" / exp / "final.zip"
def reference_export(state: dict, stage_index: int) -> pathlib.Path:
if stage_index == 0:
return PROMOTED_EASY
exp = passing_entry(state, stage_index - 1)["experiment"]
return REPO_ROOT / "Game" / "bots" / f"{exp}.json"
def league_pool(state: dict) -> list[pathlib.Path]:
stage4 = passing_entry(state, 0)["experiment"]
stage5 = passing_entry(state, 1)["experiment"]
return [
FOUNDATION_EXPORT,
REPO_ROOT / "Game" / "bots" / f"{stage4}.json",
REPO_ROOT / "Game" / "bots" / f"{stage5}.json",
]
def telemetry_tail(experiment: str, count: int = 500) -> dict[str, float]:
log_dirs = sorted((TRAINING_DIR / "logs").glob(f"{experiment}_*"))
if not log_dirs:
return {}
event_files = sorted(log_dirs[-1].glob("events.out.tfevents.*"))
if not event_files:
return {}
accumulator = EventAccumulator(str(event_files[-1]), size_guidance={"scalars": 0})
accumulator.Reload()
result = {}
for tag in accumulator.Tags().get("scalars", []):
if not tag.startswith("rollout/"):
continue
values = [point.value for point in accumulator.Scalars(tag)[-count:]]
if values:
result[tag] = sum(values) / len(values)
return result
def telemetry_passes(stage: dict, telemetry: dict[str, float]) -> tuple[bool, list[str]]:
failures = []
for metric, floor in stage.get("telemetry_floors", {}).items():
value = telemetry.get(metric)
if value is None:
failures.append(f"{metric} missing")
elif value < floor:
failures.append(f"{metric}={value:.4f} < {floor:.4f}")
return not failures, failures
def run_training(state: dict, stage_index: int, attempt: int, args) -> str:
stage = STAGES[stage_index]
suffix = "" if attempt == 0 else f"-retry{attempt}"
experiment = f"{datetime.now().strftime('%Y%m%d-%H%M')}-gen5-s{stage['number']}-{stage['name']}{suffix}"
resume = resume_checkpoint(state, stage_index, attempt, pathlib.Path(args.foundation_checkpoint))
if not resume.exists():
raise FileNotFoundError(f"Resume checkpoint not found: {resume}")
cmd = [
"./run_training.sh", experiment,
"--timesteps", str(stage["timesteps"]),
"--n-parallel", str(args.n_parallel),
"--speedup", str(args.speedup),
"--resume", str(resume),
*STANDING_ARGS,
*stage["flags"],
]
if stage.get("league_pool"):
pool = league_pool(state)
missing = [str(path) for path in pool if not path.exists()]
if missing:
raise FileNotFoundError(f"League pool models missing: {missing}")
cmd += ["--opponent-pool", ",".join(str(path) for path in pool)]
print(f"\n=== Generation 5 Stage {stage['number']} {stage['name']} attempt {attempt + 1} ===")
print(" ".join(cmd))
if args.dry_run:
return experiment
subprocess.run(cmd, cwd=TRAINING_DIR, check=True)
return experiment
def evaluate(experiment: str, reference: pathlib.Path, args) -> dict:
candidate = REPO_ROOT / "Game" / "bots" / f"{experiment}.json"
cmd = [
".venv/bin/python", "evaluate.py", str(candidate), str(reference),
"--episodes", str(EVAL_EPISODES), "--speedup", str(args.speedup),
]
if args.godot_bin:
cmd += ["--godot_bin", args.godot_bin]
subprocess.run(cmd, cwd=TRAINING_DIR, check=True)
return json.loads(EVAL_HISTORY_PATH.read_text())[-1]
def match_passes(record: dict) -> bool:
candidate = record["wins_a"] / record["episodes"]
reference = record["wins_b"] / record["episodes"]
return reference - candidate < REGRESSION_MARGIN
def evaluation_goal_rate(record: dict) -> float:
"""Fraction of paired evaluation episodes that ended in either bot scoring."""
return (record["wins_a"] + record["wins_b"]) / record["episodes"]
def physical_side_imbalance(record: dict) -> float:
"""Absolute physical-team win margin as a fraction of all episodes."""
physical = record["physical_team_wins"]
return abs(physical["team_0"] - physical["team_1"]) / record["episodes"]
def commit_progress(experiment: str) -> None:
subprocess.run(["git", "add", STATE_PATH.name, EVAL_HISTORY_PATH.name], cwd=TRAINING_DIR, check=True)
if subprocess.run(["git", "diff", "--cached", "--quiet"], cwd=TRAINING_DIR).returncode == 0:
return
subprocess.run(
["git", "commit", "-m", f"chore(training): generation 5 progress after {experiment}"],
cwd=TRAINING_DIR,
check=True,
)
subprocess.run(["git", "push"], cwd=TRAINING_DIR, check=True)
def main() -> None:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--n-parallel", type=int, default=14)
parser.add_argument("--speedup", type=int, default=16)
parser.add_argument("--godot-bin", default=None, help="Godot binary for post-stage evaluation")
parser.add_argument("--foundation-checkpoint", default=str(FOUNDATION_CHECKPOINT))
parser.add_argument("--force-retry", action="store_true")
parser.add_argument("--skip-to-next-stage", action="store_true")
parser.add_argument("--dry-run", action="store_true", help="Print the next run command without executing it")
args = parser.parse_args()
state = load_state()
if state["status"] == "done":
print("Generation 5 is already complete.")
return
if state["status"] == "blocked":
if args.force_retry:
state["attempt"] += 1
state["status"] = "in_progress"
save_state(state)
elif args.skip_to_next_stage:
state["stage_index"] += 1
state["attempt"] = 0
state["status"] = "in_progress"
save_state(state)
else:
stage = STAGES[state["stage_index"]]
print(f"BLOCKED at Stage {stage['number']} {stage['name']}; inspect {STATE_PATH.name}.")
print("Use --force-retry after adjustment or --skip-to-next-stage after human review.")
sys.exit(1)
while state["stage_index"] < len(STAGES):
stage_index = state["stage_index"]
attempt = state["attempt"]
stage = STAGES[stage_index]
experiment = run_training(state, stage_index, attempt, args)
if args.dry_run:
return
telemetry = telemetry_tail(experiment)
telemetry_ok, telemetry_failures = telemetry_passes(stage, telemetry)
references = [reference_export(state, stage_index)]
if stage.get("league_pool"):
references.extend(league_pool(state))
# Preserve order while avoiding a duplicate Stage-5 evaluation in
# the league stage (its predecessor is also in the pool).
references = list(dict.fromkeys(references))
records = [evaluate(experiment, reference, args) for reference in references]
match_ok = all(match_passes(record) for record in records)
evaluation_goal_floor = stage.get("evaluation_goal_rate_floor", 0.0)
evaluation_goal_failures = [
f"{pathlib.Path(record['model_b']).name}: goal_rate={evaluation_goal_rate(record):.3f} "
f"< {evaluation_goal_floor:.3f}"
for record in records
if evaluation_goal_rate(record) < evaluation_goal_floor
]
scoring_ok = not evaluation_goal_failures
side_imbalance_ceiling = stage.get("physical_side_imbalance_ceiling", 1.0)
side_balance_failures = [
f"{pathlib.Path(record['model_b']).name}: physical_side_imbalance="
f"{physical_side_imbalance(record):.3f} > {side_imbalance_ceiling:.3f}"
for record in records
if physical_side_imbalance(record) > side_imbalance_ceiling
]
side_balance_ok = not side_balance_failures
decision = "pass" if match_ok and telemetry_ok else "fail"
if not scoring_ok or not side_balance_ok:
decision = "fail"
entry = {
"stage_index": stage_index,
"stage_number": stage["number"],
"stage_name": stage["name"],
"experiment": experiment,
"attempt": attempt,
"telemetry_tail": telemetry,
"telemetry_failures": telemetry_failures,
"evaluation_goal_failures": evaluation_goal_failures,
"side_balance_failures": side_balance_failures,
"eval": records[0],
"evals": records,
"decision": decision,
}
state["log"].append(entry)
print(
f"{experiment}: match={'pass' if match_ok else 'fail'}, "
f"scoring={'pass' if scoring_ok else 'fail'}, "
f"side_balance={'pass' if side_balance_ok else 'fail'}, "
f"telemetry={'pass' if telemetry_ok else 'fail'} -> {decision}"
)
for failure in telemetry_failures:
print(f" {failure}")
for failure in evaluation_goal_failures:
print(f" {failure}")
for failure in side_balance_failures:
print(f" {failure}")
if decision == "pass":
state["stage_index"] += 1
state["attempt"] = 0
save_state(state)
commit_progress(experiment)
continue
if attempt >= MAX_RETRIES:
state["status"] = "blocked"
save_state(state)
commit_progress(experiment)
print(f"BLOCKED after {MAX_RETRIES + 1} attempts at Stage {stage['number']}.")
sys.exit(1)
state["attempt"] += 1
save_state(state)
commit_progress(experiment)
state["status"] = "done"
save_state(state)
commit_progress(state["log"][-1]["experiment"])
print("Generation 5 complete: handling, intercepts, and league stages passed.")
if __name__ == "__main__":
main()
+29
View File
@@ -0,0 +1,29 @@
#!/usr/bin/env bash
# Run/resume the post-Stage-3 generation-5 curriculum in a detached tmux
# session. Safe to disconnect; rerunning attaches to the live session.
set -euo pipefail
cd "$(dirname "$0")"
SESSION="cosmic-generation5"
TB_PORT=6006
command -v tmux >/dev/null 2>&1 || { echo "tmux is required: sudo apt install tmux" >&2; exit 1; }
if tmux has-session -t "$SESSION" 2>/dev/null; then
echo "Session '$SESSION' already running — attaching (detach with Ctrl-B then D)."
exec tmux attach -t "$SESSION"
fi
tmux new-session -d -s "$SESSION" -n curriculum \
".venv/bin/python generation5.py $*; echo; echo '=== generation5.py exited — press Enter to close ==='; read"
if ! (exec 3<>"/dev/tcp/127.0.0.1/$TB_PORT") 2>/dev/null; then
tmux new-window -d -t "$SESSION" -n dashboard \
".venv/bin/tensorboard --logdir logs --host 0.0.0.0 --port $TB_PORT"
fi
IP=$(hostname -I 2>/dev/null | awk '{print $1}')
echo "Generation 5 started in tmux session '$SESSION'."
echo " watch it: tmux attach -t $SESSION"
echo " dashboard: http://${IP:-<this-box>}:$TB_PORT"
echo " progress: cat generation5_state.json"
+6
View File
@@ -0,0 +1,6 @@
{
"stage_index": 0,
"attempt": 0,
"status": "in_progress",
"log": []
}
+13
View File
@@ -17,6 +17,7 @@ Usage:
"""
import sys
from pathlib import Path
import gymnasium as gym
import numpy as np
@@ -30,6 +31,7 @@ from export_policy import ACTION_HEADS
# out of order too, not just catch nothing because both sides changed
# together.
EXPECTED_ORDER = ["rot_x", "rot_y", "rot_z", "thrust_x", "thrust_y", "thrust_z", "turbo"]
GAME_SCRIPTS = Path(__file__).resolve().parents[1] / "Game" / "scripts"
def check_action_heads_match_expected_order() -> None:
@@ -87,12 +89,23 @@ def check_round_trip_preserves_per_head_values() -> None:
np.testing.assert_array_equal(np.asarray(original[head_index]), expected_column)
def check_team_frame_mapping_is_shared() -> None:
codec = (GAME_SCRIPTS / "ship_action_codec.gd").read_text()
training = (GAME_SCRIPTS / "ship_ai_controller.gd").read_text()
inference = (GAME_SCRIPTS / "ai_ship_controller.gd").read_text()
assert "action.rotation.x = -action.rotation.x" in codec
assert "action.rotation.z = -action.rotation.z" in codec
assert "ShipActionCodec.apply_team_frame(" in training
assert "ShipActionCodec.apply_team_frame(" in inference
def main() -> int:
checks = [
check_action_heads_match_expected_order,
check_gymnasium_sorts_to_expected_order,
check_action_space_processor_produces_expected_multi_discrete,
check_round_trip_preserves_per_head_values,
check_team_frame_mapping_is_shared,
]
for check in checks:
check()
+62
View File
@@ -0,0 +1,62 @@
"""Offline regression checks for evaluate.py's paired side-swap logic."""
import unittest
from unittest.mock import patch
import evaluate
class EvaluatePairTests(unittest.TestCase):
@patch("evaluate.run_half")
def test_replays_same_seed_with_models_and_masks_swapped(self, run_half) -> None:
run_half.side_effect = [
{"episodes": 4, "goals_a": 3, "goals_b": 1, "draws": 0},
{"episodes": 4, "goals_a": 2, "goals_b": 1, "draws": 1},
]
record = evaluate.evaluate_pair(
"godot", "candidate.json", "reference.json", 8, 16, 42,
grounded_a=False, grounded_b=True,
)
self.assertEqual(run_half.call_args_list[0].args, ("godot", "candidate.json", "reference.json", 4, 16, 42))
self.assertEqual(run_half.call_args_list[1].args, ("godot", "reference.json", "candidate.json", 4, 16, 42))
self.assertEqual(
run_half.call_args_list[0].kwargs,
{"grounded_a": False, "grounded_b": True},
)
self.assertEqual(
run_half.call_args_list[1].kwargs,
{"grounded_a": True, "grounded_b": False},
)
self.assertEqual(record["wins_a"], 4)
self.assertEqual(record["wins_b"], 3)
self.assertEqual(record["draws"], 1)
self.assertEqual(record["physical_team_wins"], {"team_0": 5, "team_1": 2})
self.assertEqual(record["side_results"]["a_team_0"]["wins_a"], 3)
self.assertEqual(record["side_results"]["a_team_1"]["wins_a"], 1)
def test_rejects_unpaired_episode_counts(self) -> None:
for episodes in (0, 1, 3, 99):
with self.subTest(episodes=episodes):
with self.assertRaisesRegex(ValueError, "even number"):
evaluate.evaluate_pair("godot", "a", "b", episodes, 16, 1)
@patch("evaluate.run_half")
def test_identical_policy_results_cancel_physical_side_bias(self, run_half) -> None:
# Replaying the same deterministic matchup must produce the same
# physical-team result. Model A receives opposite sides in the two
# halves, so even a large team-0 advantage cancels exactly.
physical_result = {"episodes": 10, "goals_a": 8, "goals_b": 1, "draws": 1}
run_half.side_effect = [physical_result, physical_result]
record = evaluate.evaluate_pair("godot", "same.json", "same.json", 20, 16, 7)
self.assertEqual(record["wins_a"], 9)
self.assertEqual(record["wins_b"], 9)
self.assertEqual(record["draws"], 2)
self.assertEqual(record["win_rate_a"], 0.45)
if __name__ == "__main__":
unittest.main()
+89
View File
@@ -0,0 +1,89 @@
"""Offline checks for the generation-5 stage configuration and gates."""
import unittest
import generation5
def flag_value(flags: list[str], name: str) -> str:
index = flags.index(name)
return flags[index + 1]
class Generation5ConfigTests(unittest.TestCase):
def test_stage_sequence_and_lineage(self) -> None:
self.assertEqual([stage["number"] for stage in generation5.STAGES], [4, 5, 6])
state = generation5.fresh_state()
checkpoint = generation5.resume_checkpoint(
state, stage_index=0, attempt=0, foundation=generation5.FOUNDATION_CHECKPOINT
)
self.assertEqual(checkpoint, generation5.FOUNDATION_CHECKPOINT)
def test_start_state_probabilities_leave_random_remainder(self) -> None:
for stage in generation5.STAGES:
flags = stage["flags"]
total = sum(
float(flag_value(flags, name))
for name in (
"--kickoff-chance",
"--near-goal-chance",
"--air-drill-chance",
"--air-intercept-chance",
)
)
with self.subTest(stage=stage["name"]):
self.assertLessEqual(total, 1.0)
def test_only_league_stage_requests_pool(self) -> None:
self.assertEqual(
[stage["name"] for stage in generation5.STAGES if stage.get("league_pool")],
["league"],
)
self.assertEqual(flag_value(generation5.STAGES[2]["flags"], "--opponent-mode"), "league")
def test_telemetry_floors_fail_closed_on_missing_metric(self) -> None:
ok, failures = generation5.telemetry_passes(
generation5.STAGES[0], {"rollout/upright_fraction": 1.0}
)
self.assertFalse(ok)
self.assertIn("rollout/forward_motion_fraction missing", failures)
def test_match_gate_only_blocks_clear_regression(self) -> None:
self.assertTrue(
generation5.match_passes({"wins_a": 40, "wins_b": 54, "episodes": 100})
)
self.assertFalse(
generation5.match_passes({"wins_a": 40, "wins_b": 55, "episodes": 100})
)
def test_evaluation_goal_rate_counts_either_scorer(self) -> None:
self.assertEqual(
generation5.evaluation_goal_rate(
{"wins_a": 45, "wins_b": 35, "draws": 20, "episodes": 100}
),
0.8,
)
def test_physical_side_imbalance_exposes_broken_player_slot(self) -> None:
self.assertEqual(
generation5.physical_side_imbalance(
{
"physical_team_wins": {"team_0": 90, "team_1": 5},
"episodes": 100,
}
),
0.85,
)
self.assertEqual(
generation5.physical_side_imbalance(
{
"physical_team_wins": {"team_0": 42, "team_1": 38},
"episodes": 100,
}
),
0.04,
)
if __name__ == "__main__":
unittest.main()
+46 -5
View File
@@ -62,8 +62,7 @@ class GoalRateCallback(BaseCallback):
class FlightTelemetryCallback(BaseCallback):
"""Logs rollout/{airborne_fraction,mean_altitude,air_touch_fraction,
vertical_thrust_mean} — leading indicators for curriculum generation 4's
"""Logs flight and handling telemetry — leading indicators for curriculum generation 4's
core hypothesis (a discrete action space lets the policy actually hold a
sustained vertical set-point, e.g. hovering), visible from the very
first rollout instead of only in a win-rate number measured a full
@@ -72,7 +71,15 @@ class FlightTelemetryCallback(BaseCallback):
"airborne_fraction", "mean_altitude", "air_touch_fraction",
"vertical_thrust_mean")) — see ShipAIController.get_info."""
_KEYS = ("airborne_fraction", "mean_altitude", "air_touch_fraction", "vertical_thrust_mean")
_KEYS = (
"airborne_fraction",
"mean_altitude",
"air_touch_fraction",
"vertical_thrust_mean",
"productive_air_touch_fraction",
"upright_fraction",
"forward_motion_fraction",
)
def _on_step(self) -> bool:
return True
@@ -287,13 +294,18 @@ def parse_args():
)
curriculum.add_argument(
"--opponent-mode",
choices=["self_play", "inert", "frozen"],
choices=["self_play", "inert", "frozen", "league"],
default=None,
help="self_play (default): both ships are live trainees. inert: team 1 is a "
"do-nothing placeholder (isolated scoring practice). frozen: team 1 runs a "
"fixed exported policy (--opponent-model)",
"fixed exported policy (--opponent-model); league: sample a fixed policy per episode "
"from --opponent-pool",
)
curriculum.add_argument("--opponent-model", default=None, help="Exported policy .json for --opponent-mode=frozen")
curriculum.add_argument(
"--opponent-pool", default=None,
help="Comma-separated exported policy paths for --opponent-mode=league; one is sampled per episode",
)
curriculum.add_argument(
"--draw-penalty", type=float, default=None, help="One-time penalty when an episode times out with no goal"
)
@@ -310,6 +322,14 @@ def parse_args():
help="Overrides air_drill_chance: ball spawned high, both ships spawned low and lateral — "
"unsolvable without climbing (curriculum generation 4's state-setter aerial curriculum)",
)
curriculum.add_argument(
"--air-intercept-chance", type=float, default=None,
help="Moving high-ball interception starts aimed at a real goal (generation-5 aerial stage)",
)
curriculum.add_argument(
"--team-size", type=int, choices=range(1, 6), default=None,
help="Ships per team (1-5); generation-5 automated stages remain 1v1 until 2v2 evaluation exists",
)
curriculum.add_argument(
"--tilt-penalty", type=float, default=None,
help="Overrides ShipAIController.tilt_penalty (dense per-tick cost scaled by non-upright tilt)",
@@ -318,6 +338,10 @@ def parse_args():
"--velocity-to-ball-weight", type=float, default=None,
help="Overrides ShipAIController.velocity_to_ball_weight (dense reward for closing speed toward the ball)",
)
curriculum.add_argument(
"--forward-velocity-to-ball-weight", type=float, default=None,
help="Low-altitude dense reward for nose-led planar approach toward the ball",
)
curriculum.add_argument(
"--ball-distance-penalty", type=float, default=None,
help="Overrides ShipAIController.ball_distance_penalty (dense per-tick cost scaled by distance to the ball)",
@@ -330,6 +354,14 @@ def parse_args():
"--airborne-penalty", type=float, default=None,
help="Overrides ShipAIController.airborne_penalty (dense per-tick cost scaled by height above the floor)",
)
curriculum.add_argument(
"--ground-tilt-penalty", type=float, default=None,
help="Low-altitude-only tilt cost that fades to zero by the handling-height threshold",
)
curriculum.add_argument(
"--speed-reward-weight", type=float, default=None,
help="Overrides the orientation-agnostic own-speed reward (generation 5 handling sets it to zero)",
)
curriculum.add_argument(
"--ball-velocity-to-goal-weight", type=float, default=None,
help="Overrides ShipAIController.ball_velocity_to_goal_weight (dense reward for the ball's velocity toward the attack goal)",
@@ -349,16 +381,22 @@ def _curriculum_kwargs(args) -> dict:
mapping = {
"opponent_mode": args.opponent_mode,
"opponent_model": args.opponent_model,
"opponent_model_pool": args.opponent_pool,
"draw_penalty": args.draw_penalty,
"attack_goal_bias": args.attack_goal_bias,
"kickoff_state_chance": args.kickoff_chance,
"ball_near_goal_chance": args.near_goal_chance,
"air_drill_chance": args.air_drill_chance,
"air_intercept_chance": args.air_intercept_chance,
"team_size": args.team_size,
"ai_tilt_penalty": args.tilt_penalty,
"ai_ground_tilt_penalty": args.ground_tilt_penalty,
"ai_velocity_to_ball_weight": args.velocity_to_ball_weight,
"ai_forward_velocity_to_ball_weight": args.forward_velocity_to_ball_weight,
"ai_ball_distance_penalty": args.ball_distance_penalty,
"ai_ball_touch_reward": args.ball_touch_reward,
"ai_airborne_penalty": args.airborne_penalty,
"ai_speed_reward_weight": args.speed_reward_weight,
"ai_ball_velocity_to_goal_weight": args.ball_velocity_to_goal_weight,
"goal_reward": args.goal_reward,
}
@@ -395,6 +433,9 @@ def main():
"mean_altitude",
"air_touch_fraction",
"vertical_thrust_mean",
"productive_air_touch_fraction",
"upright_fraction",
"forward_motion_fraction",
),
)