mirror of
https://github.com/jcreek/CosmicClash.git
synced 2026-09-10 16:04:04 +00:00
Josh Creek
ca70568fad
Several independent causes, all of which had to be right before the Fleet could reach Ready. The supervisor pointed --sdk-base-url at 127.0.0.1:9357, which is the Agones sidecar's gRPC port; its HTTP surface is 9358, and that is what AGONES_SDK_HTTP_PORT carries and what agones_sdk.gd reads. An HTTP client against the gRPC port could never have worked, in kind or in production. The supervisor also treated the sidecar's first incomplete /gameserver response as fatal. The sidecar accepts requests before the controller populates status.address and status.ports, so this produced a restart loop precisely during normal Agones startup. It now polls until the endpoint is assigned or ReadyTimeout elapses. server_boot.gd started ServerControl and the Agones SDK only under --allocated-mode, but the kind smoke deliberately strips that flag, so nothing served the readiness probe and the GameServer could never become Ready. Lifecycle now keys on AGONES_SDK_HTTP_PORT, which Agones injects into every managed container, while allocation and roster semantics stay tied to --allocated-mode. The SDK node is added to the tree non-deferred, since start_health() creates a Timer immediately. Fleet: Agones assigns its own SDK service account and masks that token from the game container while keeping it for the injected sidecar, so the manifest must not pin serviceAccountName or automountServiceAccountToken. Godot stores user:// under HOME, so HOME points at the writable runtime volume to keep the root filesystem read-only, and fsGroup makes that volume writable for the non-root user. Namespace: Agones' Dynamic port policy injects a hostPort, which both the baseline and restricted Pod Security Standards forbid, so the workload namespace enforces privileged while continuing to audit and warn against restricted. NetworkPolicy: the injected sidecar reaches the Kubernetes API over HTTPS, and NetworkPolicy applies to the whole Pod rather than to the container whose token was masked. The kind runner creates the namespace before Helm so Agones can install its per-namespace SDK RBAC, scopes gameservers.namespaces to it, forces the allocator and ping Services to ClusterIP because LoadBalancer ingress never becomes ready in plain kind, and labels the node so the production Fleet's on-demand/zone constraints are exercised rather than edited out of the rendered manifest.
268 lines
7.1 KiB
YAML
268 lines
7.1 KiB
YAML
apiVersion: networking.k8s.io/v1
|
|
kind: NetworkPolicy
|
|
metadata:
|
|
name: default-deny-ingress-egress
|
|
namespace: cosmic-clash
|
|
spec:
|
|
podSelector: {}
|
|
policyTypes: [Ingress, Egress]
|
|
---
|
|
apiVersion: networking.k8s.io/v1
|
|
kind: NetworkPolicy
|
|
metadata:
|
|
name: control-plane-allowed-flows
|
|
namespace: cosmic-clash
|
|
spec:
|
|
podSelector:
|
|
matchLabels:
|
|
app.kubernetes.io/name: control-plane
|
|
policyTypes: [Ingress, Egress]
|
|
ingress:
|
|
- from:
|
|
- namespaceSelector: {}
|
|
podSelector:
|
|
matchLabels:
|
|
app.kubernetes.io/name: edge-gateway
|
|
ports:
|
|
- protocol: TCP
|
|
port: 8080
|
|
# Allocated game servers are control-plane clients too: roster fetch,
|
|
# registration, connection receipts, shutdown acknowledgement and result
|
|
# submission all target this port. Their egress was already permitted, but
|
|
# without a matching ingress rule every one of those calls was dropped, so
|
|
# no allocated match could complete even inside the cluster.
|
|
- from:
|
|
- podSelector:
|
|
matchLabels:
|
|
app.kubernetes.io/name: game-server
|
|
ports:
|
|
- protocol: TCP
|
|
port: 8080
|
|
egress:
|
|
- to:
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
kubernetes.io/metadata.name: data
|
|
podSelector:
|
|
matchLabels:
|
|
app.kubernetes.io/name: postgres
|
|
ports:
|
|
- protocol: TCP
|
|
port: 5432
|
|
- to:
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
kubernetes.io/metadata.name: data
|
|
podSelector:
|
|
matchLabels:
|
|
app.kubernetes.io/name: redis
|
|
ports:
|
|
- protocol: TCP
|
|
port: 6379
|
|
- ports:
|
|
- protocol: UDP
|
|
port: 53
|
|
- protocol: TCP
|
|
port: 53
|
|
to:
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
kubernetes.io/metadata.name: kube-system
|
|
podSelector:
|
|
matchLabels:
|
|
k8s-app: kube-dns
|
|
---
|
|
apiVersion: networking.k8s.io/v1
|
|
kind: NetworkPolicy
|
|
metadata:
|
|
name: game-server-allowed-egress
|
|
namespace: cosmic-clash
|
|
spec:
|
|
podSelector:
|
|
matchLabels:
|
|
app.kubernetes.io/name: game-server
|
|
policyTypes: [Egress]
|
|
egress:
|
|
- to:
|
|
- podSelector:
|
|
matchLabels:
|
|
app.kubernetes.io/name: control-plane
|
|
ports:
|
|
- protocol: TCP
|
|
port: 8080
|
|
# The injected Agones SDK sidecar updates its GameServer through the
|
|
# kubernetes.default HTTPS Service. Its token is masked from the public
|
|
# game-server container by Agones, but NetworkPolicy applies to the Pod.
|
|
- ports:
|
|
- protocol: TCP
|
|
port: 443
|
|
- ports:
|
|
- protocol: UDP
|
|
port: 53
|
|
- protocol: TCP
|
|
port: 53
|
|
to:
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
kubernetes.io/metadata.name: kube-system
|
|
podSelector:
|
|
matchLabels:
|
|
k8s-app: kube-dns
|
|
---
|
|
apiVersion: networking.k8s.io/v1
|
|
kind: NetworkPolicy
|
|
metadata:
|
|
name: allocator-allowed-flows
|
|
namespace: cosmic-clash
|
|
spec:
|
|
podSelector:
|
|
matchLabels:
|
|
app.kubernetes.io/name: allocator
|
|
policyTypes: [Ingress, Egress]
|
|
ingress:
|
|
- from:
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
kubernetes.io/metadata.name: monitoring
|
|
podSelector:
|
|
matchLabels:
|
|
app.kubernetes.io/name: prometheus
|
|
ports:
|
|
- protocol: TCP
|
|
port: 9091
|
|
egress:
|
|
- to:
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
kubernetes.io/metadata.name: data
|
|
podSelector:
|
|
matchLabels:
|
|
app.kubernetes.io/name: postgres
|
|
ports:
|
|
- protocol: TCP
|
|
port: 5432
|
|
# The kubernetes.default Service endpoint is implementation-specific and
|
|
# may be a control-plane/node IP that cannot be selected by pod labels.
|
|
# Keep API egress portable while limiting it to TLS only.
|
|
- ports:
|
|
- protocol: TCP
|
|
port: 443
|
|
- ports:
|
|
- protocol: UDP
|
|
port: 53
|
|
- protocol: TCP
|
|
port: 53
|
|
to:
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
kubernetes.io/metadata.name: kube-system
|
|
podSelector:
|
|
matchLabels:
|
|
k8s-app: kube-dns
|
|
---
|
|
apiVersion: networking.k8s.io/v1
|
|
kind: NetworkPolicy
|
|
metadata:
|
|
name: maintenance-allowed-egress
|
|
namespace: cosmic-clash
|
|
spec:
|
|
podSelector:
|
|
matchLabels:
|
|
app.kubernetes.io/name: maintenance
|
|
policyTypes: [Egress]
|
|
egress:
|
|
- to:
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
kubernetes.io/metadata.name: data
|
|
podSelector:
|
|
matchLabels:
|
|
app.kubernetes.io/name: postgres
|
|
ports:
|
|
- protocol: TCP
|
|
port: 5432
|
|
- ports:
|
|
- protocol: UDP
|
|
port: 53
|
|
- protocol: TCP
|
|
port: 53
|
|
to:
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
kubernetes.io/metadata.name: kube-system
|
|
podSelector:
|
|
matchLabels:
|
|
k8s-app: kube-dns
|
|
---
|
|
# Public players connect straight to the allocated GameServer's UDP port; the
|
|
# control plane only ever hands out its address. The namespace-wide default
|
|
# deny blocked that ingress entirely, so an allocated server was unreachable
|
|
# from the internet and no matchmade game could be joined.
|
|
#
|
|
# The source cannot be narrowed by selector: these peers are player machines
|
|
# outside the cluster. It is narrowed instead to exactly one protocol and port
|
|
# on exactly the game-server pods, and the game server admits a peer only with
|
|
# a valid signed join authorisation for its own match.
|
|
apiVersion: networking.k8s.io/v1
|
|
kind: NetworkPolicy
|
|
metadata:
|
|
name: game-server-allowed-ingress
|
|
namespace: cosmic-clash
|
|
spec:
|
|
podSelector:
|
|
matchLabels:
|
|
app.kubernetes.io/name: game-server
|
|
policyTypes:
|
|
- Ingress
|
|
ingress:
|
|
- ports:
|
|
- protocol: UDP
|
|
port: 7777
|
|
---
|
|
# The matcher reads queued candidates and writes proposals. It exposes nothing
|
|
# and talks to nobody but its two datastores.
|
|
apiVersion: networking.k8s.io/v1
|
|
kind: NetworkPolicy
|
|
metadata:
|
|
name: matcher-allowed-egress
|
|
namespace: cosmic-clash
|
|
spec:
|
|
podSelector:
|
|
matchLabels:
|
|
app.kubernetes.io/name: matcher
|
|
policyTypes:
|
|
- Egress
|
|
egress:
|
|
- to:
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
kubernetes.io/metadata.name: data
|
|
podSelector:
|
|
matchLabels:
|
|
app.kubernetes.io/name: postgres
|
|
ports:
|
|
- protocol: TCP
|
|
port: 5432
|
|
- to:
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
kubernetes.io/metadata.name: data
|
|
podSelector:
|
|
matchLabels:
|
|
app.kubernetes.io/name: redis
|
|
ports:
|
|
- protocol: TCP
|
|
port: 6379
|
|
- to:
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
kubernetes.io/metadata.name: kube-system
|
|
podSelector:
|
|
matchLabels:
|
|
k8s-app: kube-dns
|
|
ports:
|
|
- protocol: UDP
|
|
port: 53
|
|
- protocol: TCP
|
|
port: 53
|