Files
CosmicClash/Game/scripts/ship.gd
T
Josh Creek 1811e9333e feat(training): curriculum generation 4 — MultiDiscrete action space redesign
Three curriculum generations (2026-07-21 through 2026-08-04) all tried
gating *when* the policy could use vertical thrust/pitch-roll on top of a
continuous Gaussian action space, and all three failed the same way: PPO's
action-distribution std collapsed within ~10% of steps and never recovered,
landing at a 15-32% win rate vs the grounded reference regardless of
mechanism (hard mask, then a gradual ramp). Generation 3's final attempt
just landed at 24% — the worst of the three.

Root cause, verified against this project's own physics: hovering this ship
requires *holding* thrust.y ~= 0.408 continuously (mass 5.0, vertical_thrust
120, gravity 9.8). A collapsed near-zero-mean Gaussian can brush that value
but never sustain it long enough to earn the reward gradient that would
move the mean — no amount of gating *when* the axis acts fixes a problem in
*how* the policy represents a decision on it. This also independently found
and fixes a real bug: godot_rl never marks an episode timeout as a
truncation, so PPO was bootstrapping V(s)=0 on every 30s draw in every
generation to date.

- Game/scripts/ship_action_codec.gd (new): single source of truth for a
  per-axis MultiDiscrete action space (7 heads, nvec [5,5,5,5,5,5,2]) shared
  by training and in-game inference, replacing the continuous Gaussian.
  thrust_y's bins are deliberately asymmetric so a random policy drifts
  through the volume instead of floor-pinning. Legacy continuous decode
  (ai_ship_controller.gd's old logic) preserved verbatim so every
  pre-generation-4 export (e.g. Game/bots/promoted/easy.json) keeps working
  unchanged via an optional "action_space" JSON field.
- ship_observations.gd: append own contact state (SIZE 31 -> 35, append-only)
  so the value function can see what wall_contact_penalty fires on.
- ship_ai_controller.gd: action space/decode via the codec; drop the
  vertical_ramp/pitch_roll_ramp mechanism entirely; tilt_penalty default
  lowered 4x (aerial approaches require pitching); flight telemetry
  (airborne_fraction, mean_altitude, air_touch_fraction, vertical_thrust_mean)
  and truncation-snapshot fields on get_info().
- training_mode.gd: new air_drill_chance state-setter branch (ball spawned
  high, ships low, kept clear of walls) so aerial practice is forced by the
  environment instead of relying on reward-driven exploration alone; snapshot
  terminal observations before a timeout reset for the truncation fix.
- cosmic_env.py: remap ShipAIController's truncated/terminal_obs info into
  SB3's TimeLimit.truncated/terminal_observation keys.
- train.py: --reset-logits (+ --reset-logits-heads) replaces the
  now-meaningless --reset-std; new EntropyFloorCallback (a persistent
  per-rollout ent_coef controller replacing the one-shot std-reset shock)
  and per-head entropy logging; FlightTelemetryCallback; --air-drill-chance/
  --tilt-penalty flags; optional AbortIfCallback kill-criterion.
- export_policy.py: writes the action_space block for MultiDiscrete models;
  index-level parity check (argmax per head) instead of comparing floats.
- curriculum.py: full rewrite — 3 stages (bootstrap/selfplay/gauntlet), no
  grounded stage, full action space live from step 1; deletes generation
  1-3's checkpoint-lineage machinery (nothing to resume from); final report
  evaluates against both promoted/easy.json and the new
  promoted/reference-grounded.json (a copy of curric-s5-aggression, the
  strongest grounded-era artifact, kept as a fixed yardstick).
- run_training.sh/.gitignore: commit only final.zip, not the ~2400
  intermediate checkpoint files a single stage was writing (~500MB ->
  ~0.2MB per run); requirements.txt pinned (behaviour here now depends on
  specific library internals, not just public APIs).
- test_action_space.py (new): offline rung-0 check catching a head-order
  mismatch before it silently corrupts 24h of training.

Validated: GDScript compiles clean (Godot --headless --import + script
validation), free_play.tscn and training.tscn both boot headless without
errors, offline action-space assertions pass. Not yet run: the actual
smoke-training/A-B validation ladder steps in TRAINING.md's "Generation 4"
section, before committing to the full ~32h curriculum.

See TRAINING.md's "Generation 4" section for the full design writeup.
2026-08-04 23:27:57 +01:00

373 lines
15 KiB
GDScript
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
class_name Ship
extends RigidBody3D
# Physics-driven spaceship. All movement is force/torque-based, applied in
# _integrate_forces from a ShipAction supplied by a pluggable ShipController
# child node (player input, AI policy, or network replication — see
# set_controller). A ship without a controller is inert but still simulated,
# which is what a placeholder opponent or a headless RL ship needs.
# Physics properties (mass, inertia, friction material) live in ship.tscn.
@export_group("Movement")
@export var thrust_power = 150.0 # Main thruster power
@export var maneuvering_thrust = 75.0 # Side thruster power
@export var vertical_thrust = 120.0 # Up/down thruster power
@export var turbo_multiplier = 2.5 # Turbo boost multiplier
@export var max_speed = 35.0 # Maximum velocity
@export var rotation_power = 20.0 # Angular thrust power
@export var max_angular_speed = 3.0 # Maximum rotation speed
@export var drag_coefficient = 0.98 # Linear drag (air resistance)
@export var angular_drag = 0.95 # Rotational drag
@export_group("Surface Pull")
@export var wall_pull_strength = 6.0 # Wall grav-plating strength (m/s^2-equivalent)
@export var wall_pull_range = 3.0 # Metres from a wall where pull begins
@export var ceiling_pull_strength = 11.5 # Ceiling grav-plating strength; nets above gravity so a ship can hold a ceiling
@export var ceiling_pull_range = 3.0 # Metres from the ceiling where pull begins
# Non-tinted hull meshes, runtime-merged into one ArrayMesh by
# _build_merged_hull() (Nose/TailFin stay separate MeshInstance3Ds since
# _apply_team_color() retints them per-team and must keep addressing them by
# name). Verified via get_surface_count()/surface_get_material() before
# writing this: hull and canopy are each a single surface with their own
# distinct opaque StandardMaterial3D (canopy is NOT alpha/transparent despite
# the name), and engine_l/engine_r are each 2 surfaces, also all distinct
# materials — none of the 6 source surfaces share a material with any other,
# including the L/R engine pair. So this merge does not collapse draw calls
# the way TODO.md's "6 draw calls down to 3" assumed (Godot still issues one
# draw call per surface regardless of how many MeshInstance3Ds they're spread
# across); the real win is scene-tree node count, 4 MeshInstance3D children
# down to 1, cutting per-frame transform/visibility overhead.
const MERGED_MESH_PATHS := [
"res://assets/models/ship_hull.res",
"res://assets/models/ship_canopy.res",
"res://assets/models/ship_engine_l.res",
"res://assets/models/ship_engine_r.res",
]
const MERGED_MESH_TRANSFORMS := [
Transform3D(Basis(Vector3(-1, 0, 0), Vector3(0, 1, 0), Vector3(0, 0, -1)), Vector3(0, 0, 0)), # Hull
Transform3D(Basis(Vector3(-1, 0, 0), Vector3(0, 1, 0), Vector3(0, 0, -1)), Vector3(0, 0.31, -0.55)), # Canopy
Transform3D(Basis(Vector3(-1, 0, 0), Vector3(0, 1, 0), Vector3(0, 0, -1)), Vector3(-0.42, -0.05, 0.95)), # EngineGlowL
Transform3D(Basis(Vector3(-1, 0, 0), Vector3(0, 1, 0), Vector3(0, 0, -1)), Vector3(0.42, -0.05, 0.95)), # EngineGlowR
]
# Which team this ship plays for (0 or 1). Set by the game mode on spawn.
var team: int = 0:
set(value):
team = value
_apply_team_color()
# Shared per-team accent material, built once per team and reused by every
# ship — avoids allocating a fresh StandardMaterial3D from both _ready and
# the team setter (previously ran at least twice per ship).
static var _team_materials: Dictionary = {} # team:int -> StandardMaterial3D
static func _get_team_material(team: int) -> StandardMaterial3D:
if _team_materials.has(team):
return _team_materials[team]
var color: Color = TeamColors.TEAM_COLORS.get(team, TeamColors.TEAM_COLORS[0])
var accent := StandardMaterial3D.new()
accent.albedo_color = color
accent.metallic = 0.3
accent.roughness = 0.5
accent.emission_enabled = true
accent.emission = color
accent.emission_energy_multiplier = 0.35
_team_materials[team] = accent
return accent
var controller: ShipController
var _current_action: ShipAction = ShipAction.new()
var _inert_action: ShipAction = ShipAction.new()
var _boundary: ArenaBoundary
# Instrument signals for efficient data distribution
signal speed_changed(speed: float)
signal attitude_changed(pitch: float, roll: float, yaw: float)
signal altitude_changed(altitude: float)
signal thrust_changed(thrust_percent: float)
signal angular_velocity_changed(angular_speed: float)
signal heading_changed(heading_degrees: float)
# Performance optimization - track last emitted values to avoid unnecessary signals
var _last_speed: float = -1.0
var _last_altitude: float = -999999.0
var _last_angular_speed: float = -1.0
var _last_pitch: float = -999.0
var _last_roll: float = -999.0
var _last_yaw: float = -999.0
var _last_heading: float = -999.0
var _last_thrust: float = -1.0
# Thresholds for signal emission (only emit if change is significant)
const SPEED_THRESHOLD = 0.1 # m/s
const ALTITUDE_THRESHOLD = 0.5 # meters
const ANGULAR_THRESHOLD = 0.01 # rad/s
const ATTITUDE_THRESHOLD = 1.0 # degrees
const THRUST_THRESHOLD = 1.0 # percent
func _ready():
# Add ship to group for instrument discovery
add_to_group("ship")
# Pick up a controller placed in the scene, if any; game modes usually
# attach one at spawn time via set_controller instead.
for child in get_children():
if child is ShipController:
controller = child
break
# Always on (moved here from ShipAIController.setup, which only enabled
# it for training-side ships): ShipObservations now reads own-contact
# state (see its "contact" section) for every ship, training or shipped,
# so the RigidBody3D contact list must exist unconditionally rather than
# only for whichever ship happened to be a training agent. Cheap — a
# short per-tick contact list from the physics engine, not a rendering
# cost like the headless skips just below.
contact_monitor = true
max_contacts_reported = 8
_apply_team_color()
_boundary = get_tree().get_first_node_in_group("arena_boundary")
# Telemetry emission and the merged-hull visual mesh are both render-only
# work; headless (RL/CI) instances never render or have a HUD watching
# them, so skip both rather than relying on later no-ops.
if DisplayServer.get_name() == "headless":
set_physics_process(false)
else:
_build_merged_hull()
func _apply_team_color() -> void:
if not is_inside_tree():
return
var accent := _get_team_material(team)
for mesh_name in ["Nose", "TailFin"]:
var mesh := get_node_or_null(mesh_name) as MeshInstance3D
if mesh:
mesh.material_override = accent
# Runtime-bakes Hull/Canopy/EngineGlowL/EngineGlowR (see MERGED_MESH_PATHS
# comment above) into one ArrayMesh, one destination surface per source
# surface via SurfaceTool.append_from, each keeping its own original
# material — preserves current visuals exactly regardless of surface count.
# Mirrors the runtime-bake pattern already used by goal.gd/arena_boundary.gd.
# Skipped in headless mode (see _ready): purely visual, costs nothing
# physics/RL cares about.
func _build_merged_hull() -> void:
var mesh := ArrayMesh.new()
var dest_idx := 0
for i in MERGED_MESH_PATHS.size():
var src: ArrayMesh = load(MERGED_MESH_PATHS[i])
var xform: Transform3D = MERGED_MESH_TRANSFORMS[i]
for surf in src.get_surface_count():
var st := SurfaceTool.new()
st.begin(Mesh.PRIMITIVE_TRIANGLES)
st.append_from(src, surf, xform)
st.commit(mesh)
mesh.surface_set_material(dest_idx, src.surface_get_material(surf))
dest_idx += 1
var instance := MeshInstance3D.new()
instance.name = "MergedHull"
instance.mesh = mesh
add_child(instance)
# Attach the node that drives this ship (player, AI, or network). Replaces
# any existing controller; parents the new one under the ship if needed.
func set_controller(new_controller: ShipController) -> void:
if is_instance_valid(controller) and controller.get_parent() == self:
controller.queue_free()
controller = new_controller
if new_controller and new_controller.get_parent() == null:
add_child(new_controller)
func _physics_process(_delta):
if _has_telemetry_listeners():
_emit_telemetry_data()
# Covers a second/AI ship with no HUD watching it - headless mode is already
# handled by disabling _physics_process entirely in _ready.
func _has_telemetry_listeners() -> bool:
return speed_changed.get_connections().size() > 0 \
or altitude_changed.get_connections().size() > 0 \
or attitude_changed.get_connections().size() > 0 \
or heading_changed.get_connections().size() > 0 \
or thrust_changed.get_connections().size() > 0
func _integrate_forces(state):
# One action per physics tick, pulled from the controller (deterministic)
_current_action = controller.get_action() if controller else _inert_action
# === TRANSLATION (Movement) ===
apply_thruster_forces(state, _current_action)
apply_surface_pull(state)
# === ROTATION (Turning) ===
apply_rotation_forces(state, _current_action.rotation)
# === DRAG AND LIMITS ===
apply_drag_and_limits(state, _current_action.rotation)
func apply_thruster_forces(state: PhysicsDirectBodyState3D, action: ShipAction):
var thrust_input := action.thrust
if thrust_input.length() < 0.01:
return
# Convert thrust input to world space forces based on ship orientation
# Physics: F = m * a (Newton's Second Law: Force = mass × acceleration)
# World force = Local force × Rotation matrix (basis transformation)
var ship_basis = global_transform.basis
var world_thrust = Vector3.ZERO
# All thrusters should work relative to ship orientation
# Physics: Vector transformation from local to world coordinates
# F_world = R * F_local (where R is rotation matrix)
# Forward/backward thrust (main engines)
world_thrust += -ship_basis.z * thrust_input.z * thrust_power
# Strafe thrust (left/right maneuvering thrusters)
world_thrust += ship_basis.x * thrust_input.x * maneuvering_thrust
# Vertical thrust (up/down thrusters relative to ship orientation)
world_thrust += ship_basis.y * thrust_input.y * vertical_thrust
# Turbo only boosts forward thrust
if action.turbo and thrust_input.z > 0:
world_thrust *= turbo_multiplier
# Apply the force
# Physics: Δv = F * Δt / m (change in velocity = force × time / mass)
state.apply_central_force(world_thrust)
func apply_surface_pull(state: PhysicsDirectBodyState3D) -> void:
if _boundary == null:
return
var pull := _boundary.get_surface_pull(
global_position, wall_pull_strength, wall_pull_range,
ceiling_pull_strength, ceiling_pull_range
)
state.apply_central_force(pull * mass)
func apply_rotation_forces(state: PhysicsDirectBodyState3D, rotation_input: Vector3):
if rotation_input.length() < 0.01:
return
# Apply torque for rotation - simple and effective
# Physics: τ = I * α (torque = moment of inertia × angular acceleration)
# Also: α = τ / I (angular acceleration = torque / moment of inertia)
# Lower inertia = higher angular acceleration for same torque
var torque = Vector3(
rotation_input.x * rotation_power, # Pitch (rotation around X-axis)
rotation_input.y * rotation_power, # Yaw (rotation around Y-axis)
rotation_input.z * rotation_power # Roll (rotation around Z-axis)
)
# Physics: Δω = τ * Δt / I (change in angular velocity = torque × time / inertia)
state.apply_torque(torque)
func apply_drag_and_limits(state: PhysicsDirectBodyState3D, rotation_input: Vector3):
# Linear drag (air resistance)
# Physics: F_drag = -½ * ρ * v² * C_d * A (drag force equation)
# Simplified: v_new = v_old * drag_coefficient (exponential decay)
# This simulates air resistance reducing velocity over time
state.linear_velocity *= drag_coefficient
# Angular drag (rotational resistance)
# Physics: Similar to linear drag but for rotational motion
# τ_drag = -C_angular * ω² (angular drag torque)
# Simplified: ω_new = ω_old * angular_drag (exponential decay)
if rotation_input.length() < 0.01:
# More drag when not actively rotating to stop quicker
state.angular_velocity *= 0.9
else:
# Normal drag when actively rotating
state.angular_velocity *= angular_drag
# Limit maximum speeds
# Physics: Terminal velocity concept - maximum achievable speed
# When thrust force = drag force, acceleration = 0, velocity = constant
if state.linear_velocity.length() > max_speed:
# Normalize to unit vector, then scale to max speed
# Physics: v̂ = v / |v| (unit vector), v_limited = v̂ * v_max
state.linear_velocity = state.linear_velocity.normalized() * max_speed
if state.angular_velocity.length() > max_angular_speed:
# Same concept for angular velocity
# Physics: ω̂ = ω / |ω|, ω_limited = ω̂ * ω_max
state.angular_velocity = state.angular_velocity.normalized() * max_angular_speed
func _emit_telemetry_data():
# Ship only calculates and emits data - HUD handles display
# Performance optimization: only emit signals when values change significantly
# Speed telemetry
# Physics: |v| = √(vₓ² + vᵧ² + vᵤ²) (magnitude of velocity vector)
var current_speed = linear_velocity.length()
if abs(current_speed - _last_speed) > SPEED_THRESHOLD:
speed_changed.emit(current_speed)
_last_speed = current_speed
# Altitude telemetry
# Physics: Height measurement from reference point (y = 0)
var current_altitude = global_transform.origin.y
if abs(current_altitude - _last_altitude) > ALTITUDE_THRESHOLD:
altitude_changed.emit(current_altitude)
_last_altitude = current_altitude
# Angular velocity telemetry
# Physics: |ω| = √(ωₓ² + ωᵧ² + ωᵤ²) (magnitude of angular velocity vector)
var angular_speed = angular_velocity.length()
if abs(angular_speed - _last_angular_speed) > ANGULAR_THRESHOLD:
angular_velocity_changed.emit(angular_speed)
_last_angular_speed = angular_speed
# Attitude telemetry (pitch, roll, yaw from ship orientation)
# Physics: Euler angles from rotation matrix
# Aviation convention (yaw → pitch → roll, EULER_ORDER_YXZ): pitch stays in
# ±90° and yaw covers the full circle. XYZ order would instead constrain
# yaw to ±90°, so an upright ship facing "south" would be reported as
# pitch 180 + roll 180 — mathematically equivalent, but it reads as
# upside-down on the attitude indicator and breaks the heading readout.
var ship_rotation = global_transform.basis.get_euler(EULER_ORDER_YXZ)
var pitch_deg = rad_to_deg(ship_rotation.x)
var roll_deg = rad_to_deg(ship_rotation.z)
var yaw_deg = rad_to_deg(ship_rotation.y)
if abs(pitch_deg - _last_pitch) > ATTITUDE_THRESHOLD or \
abs(roll_deg - _last_roll) > ATTITUDE_THRESHOLD or \
abs(yaw_deg - _last_yaw) > ATTITUDE_THRESHOLD:
attitude_changed.emit(pitch_deg, roll_deg, yaw_deg)
_last_pitch = pitch_deg
_last_roll = roll_deg
_last_yaw = yaw_deg
# Heading telemetry (yaw - direction ship is facing)
# Physics: Yaw = rotation around Y-axis (compass heading)
# Convert to 0-360° range for traditional compass display
var heading = fmod(yaw_deg + 360.0, 360.0) # Normalize to 0-360°
if abs(heading - _last_heading) > ATTITUDE_THRESHOLD:
heading_changed.emit(heading)
_last_heading = heading
# Thrust telemetry
# Physics: Thrust output as percentage of maximum available thrust
var thrust_percent = _current_action.thrust.length() * 100.0
if abs(thrust_percent - _last_thrust) > THRUST_THRESHOLD:
thrust_changed.emit(thrust_percent)
_last_thrust = thrust_percent