feat(#14): Add zero-setup Docker data initialization

This commit is contained in:
Josh Creek
2026-07-24 22:24:45 +01:00
parent 5976b46cad
commit 0b1d647aec
14 changed files with 509 additions and 72 deletions
+85 -12
View File
@@ -8,7 +8,7 @@
**A declarative, self-hosted containerised tool for visualising and managing home lab & network architecture diagrams.** **A declarative, self-hosted containerised tool for visualising and managing home lab & network architecture diagrams.**
Open Network Diagram helps you document your infrastructure in a visual UI while keeping a real JSON source of truth you can version, back up, and reuse. Open Network Diagram helps you document your infrastructure in a visual UI while keeping a real JSON source of truth you can back up, reuse, and optionally version after reviewing it for sensitive information.
- Homelab-friendly: run it in minutes with Docker. - Homelab-friendly: run it in minutes with Docker.
- Practical: edit in the UI and autosave to `network.json`. - Practical: edit in the UI and autosave to `network.json`.
@@ -47,11 +47,10 @@ Open Network Diagram helps you document your infrastructure in a visual UI while
This is the fastest way to run Open Network Diagram for a home lab. This is the fastest way to run Open Network Diagram for a home lab.
1. Create a local data folder and seed your first `network.json`: 1. Create a local data folder:
```bash ```bash
mkdir -p ond-data mkdir -p data
curl -fsSL https://raw.githubusercontent.com/jcreek/OpenNetworkDiagram/main/data/network.json.example -o ond-data/network.json
``` ```
2. Run the published Docker image: 2. Run the published Docker image:
@@ -61,15 +60,18 @@ docker run -d \
--name open-network-diagram \ --name open-network-diagram \
--restart unless-stopped \ --restart unless-stopped \
-p 8080:3000 \ -p 8080:3000 \
--user "$(id -u):$(id -g)" \
-e NETWORK_DATA_FILE=/app/data/network.json \ -e NETWORK_DATA_FILE=/app/data/network.json \
-e NETWORK_BACKUP_DIR=/app/data/.backups \ -e NETWORK_BACKUP_DIR=/app/data/.backups \
-v "$(pwd)/ond-data:/app/data" \ -v "$(pwd)/data:/app/data" \
jcreek23/open-network-diagram:latest jcreek23/open-network-diagram:latest
``` ```
On first access, Open Network Diagram creates a valid blank `data/network.json` and shows the getting-started card.
3. Open the app at `http://localhost:8080`. 3. Open the app at `http://localhost:8080`.
4. Edit your topology in the UI. Changes persist to `ond-data/network.json`. 4. Add your first machine. Changes and rolling backups remain directly accessible in the local `data` directory across container updates and restarts.
Useful follow-up commands: Useful follow-up commands:
@@ -79,6 +81,29 @@ docker stop open-network-diagram
docker rm open-network-diagram docker rm open-network-diagram
``` ```
The `--user` option runs the container with your host UID and GID so files in the bind mount remain writable and owned by your account. Docker Desktop commonly handles bind-mount permissions without this mapping, so macOS users can omit `--user` if their Docker Desktop configuration requires the image's built-in user. If storage is not writable, the app leaves it untouched, loads the bundled read-only demo, and reports the exact permission problem in the UI.
### Optional Sample Data
The application starts with a blank network by default. To explore the sample topology instead, download it before starting the container:
```bash
curl -fsSL https://raw.githubusercontent.com/jcreek/OpenNetworkDiagram/main/data/network.json.example \
-o data/network.json
```
The sample is optional and is never required for startup.
## Deployment Security
Open Network Diagram does not include authentication or TLS. The quick-start port mapping `-p 8080:3000` listens on all host interfaces, so any device that can reach the host can view the topology and, in writable mode, change it.
Do not expose a writable deployment directly to the internet. Restrict access with a firewall, LAN or VPN, or place the application behind an authenticated reverse proxy that terminates TLS.
To make the service reachable only from the Docker host, replace the quick-start mapping with `-p 127.0.0.1:8080:3000`.
For a public demonstration, add `-e NETWORK_READ_ONLY=true` to `docker run`, or add `NETWORK_READ_ONLY: 'true'` under Compose's `environment`. Read-only mode prevents changes and uses the bundled demo if the configured data file is missing, but it does not hide a configured topology or replace authentication.
## What It Looks Like ## What It Looks Like
| Network view with ethernet labels | Hosts & VMs view with VMs expanded | Modal editing a machine | | Network view with ethernet labels | Hosts & VMs view with VMs expanded | Modal editing a machine |
@@ -89,6 +114,10 @@ docker rm open-network-diagram
| --------------------------------------------------------------------------- | ------------------------------------------------------------------- | -------------------------------------------------- | | --------------------------------------------------------------------------- | ------------------------------------------------------------------- | -------------------------------------------------- |
| ![Open Network Diagram IPAM panel with subnet utilization](screenshot4.png) | ![Open Network Diagram rack view with shelf items](screenshot5.png) | ![Open Network Diagram dark mode](screenshot6.png) | | ![Open Network Diagram IPAM panel with subnet utilization](screenshot4.png) | ![Open Network Diagram rack view with shelf items](screenshot5.png) | ![Open Network Diagram dark mode](screenshot6.png) |
| Zero-setup first run | Read-only live demo |
| ------------------------------------------------------------------------ | ------------------------------------------------------------ |
| ![Open Network Diagram zero-setup first-run onboarding](screenshot7.png) | ![Open Network Diagram read-only live demo](screenshot8.png) |
## Docker Compose Option ## Docker Compose Option
If you prefer compose: If you prefer compose:
@@ -97,10 +126,11 @@ If you prefer compose:
services: services:
open-network-diagram: open-network-diagram:
image: jcreek23/open-network-diagram:latest image: jcreek23/open-network-diagram:latest
user: '${OND_UID:-1000}:${OND_GID:-1000}'
ports: ports:
- '8080:3000' - '8080:3000'
volumes: volumes:
- ./ond-data:/app/data - ./data:/app/data
environment: environment:
NETWORK_DATA_FILE: /app/data/network.json NETWORK_DATA_FILE: /app/data/network.json
NETWORK_BACKUP_DIR: /app/data/.backups NETWORK_BACKUP_DIR: /app/data/.backups
@@ -110,9 +140,40 @@ services:
Start it with: Start it with:
```bash ```bash
docker compose up -d mkdir -p data
OND_UID="$(id -u)" OND_GID="$(id -g)" docker compose up -d
``` ```
`OND_UID` and `OND_GID` are used only by Compose to select the container process identity; they are not application environment variables. The defaults suit common Linux installations. Docker Desktop users can remove the `user` line if their file-sharing configuration requires the image's built-in user.
### Using a Docker Named Volume
If host-directory permissions are inconvenient, Docker can manage the storage volume instead:
```bash
docker run -d \
--name open-network-diagram \
--restart unless-stopped \
-p 8080:3000 \
-e NETWORK_DATA_FILE=/app/data/network.json \
-e NETWORK_BACKUP_DIR=/app/data/.backups \
-v ond-data:/app/data \
jcreek23/open-network-diagram:latest
```
Docker creates the `ond-data` volume automatically. Use `docker volume inspect ond-data` to inspect its location or `docker cp open-network-diagram:/app/data/network.json ./network.json` to copy the data file out.
### Versioning Your Topology
`data/network.json` and `data/.backups` are ignored by Git because topology data may expose host names, addresses, MAC addresses, and other sensitive infrastructure details. If you have reviewed and sanitized the data and deliberately want to track it, opt in with:
```bash
git add -f data/network.json
git commit -m "Track network topology"
```
Once the file is tracked, normal Git commands include later changes. Backup files remain ignored.
## For Developers ## For Developers
### Local Development ### Local Development
@@ -139,6 +200,8 @@ pnpm run icons:manifest # regenerate local vendor icon manifest
- API endpoint: `GET/PUT /api/network-data` - API endpoint: `GET/PUT /api/network-data`
- Writes are enabled unless `NETWORK_READ_ONLY=true` - Writes are enabled unless `NETWORK_READ_ONLY=true`
- Writable deployments automatically initialize a missing or whitespace-only data file with a blank network.
- Read-only deployments never initialize or modify the configured data file and continue to use the bundled demo when the API data is unavailable.
- When writes are unavailable, API responses include `writableReason` for diagnostics. - When writes are unavailable, API responses include `writableReason` for diagnostics.
- Writes are persisted atomically to the configured data file - Writes are persisted atomically to the configured data file
- Rolling backups are kept in the backup directory (last 5) - Rolling backups are kept in the backup directory (last 5)
@@ -232,7 +295,7 @@ OpenNetworkDiagram/
├── src/lib/config/vendorIconManifest.ts # Generated local icon catalog ├── src/lib/config/vendorIconManifest.ts # Generated local icon catalog
├── static/data/network.json # Demo dataset (Netlify) ├── static/data/network.json # Demo dataset (Netlify)
├── static/icons/vendor/ # Vendored icon assets (runtime-local) ├── static/icons/vendor/ # Vendored icon assets (runtime-local)
├── data/network.json.example # Starter data template for Docker users ├── data/network.json.example # Optional starter data template
├── third_party/ # Third-party provenance + licensing ├── third_party/ # Third-party provenance + licensing
├── Dockerfile # Docker build/runtime image ├── Dockerfile # Docker build/runtime image
├── server.mjs # Node runtime server (static + API) ├── server.mjs # Node runtime server (static + API)
@@ -252,9 +315,19 @@ OpenNetworkDiagram/
1. Fork the repository. 1. Fork the repository.
2. Create a feature branch. 2. Create a feature branch.
3. Commit your changes. 3. Before committing, run:
4. Push your branch.
5. Open a pull request. ```bash
pnpm lint
pnpm test
pnpm check
pnpm run build:docker
pnpm run build:netlify
```
4. Commit your changes.
5. Push your branch.
6. Open a pull request.
## License ## License
+1
View File
@@ -4,6 +4,7 @@ services:
context: . context: .
dockerfile: Dockerfile dockerfile: Dockerfile
image: open-network-diagram:local image: open-network-diagram:local
user: '${OND_UID:-1000}:${OND_GID:-1000}'
ports: ports:
- '8080:3000' - '8080:3000'
volumes: volumes:
+2 -1
View File
@@ -7,9 +7,10 @@
"dev": "vite dev", "dev": "vite dev",
"build": "vite build", "build": "vite build",
"build:docker": "DEPLOY_TARGET=docker vite build", "build:docker": "DEPLOY_TARGET=docker vite build",
"build:netlify": "NETWORK_READ_ONLY=true DEPLOY_TARGET=netlify vite build", "build:netlify": "NETWORK_READ_ONLY=true OND_STATIC_DEMO=true DEPLOY_TARGET=netlify vite build",
"start": "node server.mjs", "start": "node server.mjs",
"preview": "vite preview", "preview": "vite preview",
"test": "node --test tests/*.test.mjs",
"prepare": "svelte-kit sync || echo ''", "prepare": "svelte-kit sync || echo ''",
"check": "svelte-kit sync && svelte-check --tsconfig ./tsconfig.json", "check": "svelte-kit sync && svelte-check --tsconfig ./tsconfig.json",
"check:watch": "svelte-kit sync && svelte-check --tsconfig ./tsconfig.json --watch", "check:watch": "svelte-kit sync && svelte-check --tsconfig ./tsconfig.json --watch",
BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 37 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 56 KiB

+15 -4
View File
@@ -5,7 +5,7 @@ import { fileURLToPath } from 'node:url';
import { validateNetworkData } from './src/lib/shared/networkSchemaCore.mjs'; import { validateNetworkData } from './src/lib/shared/networkSchemaCore.mjs';
import { import {
getWritableState, getWritableState,
readNetworkFile, readOrInitializeNetworkFile,
writeNetworkFile writeNetworkFile
} from './src/lib/shared/networkPersistenceCore.mjs'; } from './src/lib/shared/networkPersistenceCore.mjs';
@@ -59,12 +59,17 @@ function resolveReadOnlyErrorMessage(reason) {
async function serveApi(request, response) { async function serveApi(request, response) {
if (request.method === 'GET') { if (request.method === 'GET') {
try { try {
const [payload, writableState] = await Promise.all([readNetworkFile(), getWritableState()]); const [payload, writableState] = await Promise.all([
readOrInitializeNetworkFile(),
getWritableState()
]);
const validation = validateNetworkData(payload.data); const validation = validateNetworkData(payload.data);
if (!validation.valid) { if (!validation.valid) {
sendJson(response, 500, { sendJson(response, 500, {
error: 'Stored network data is invalid', error: 'Stored network data is invalid',
details: validation.errors details: validation.errors,
writable: false,
writableReason: writableState.reason
}); });
return; return;
} }
@@ -77,8 +82,14 @@ async function serveApi(request, response) {
updatedAt: payload.updatedAt updatedAt: payload.updatedAt
}); });
} catch (error) { } catch (error) {
const writableState = await getWritableState().catch(() => ({
writable: false,
reason: null
}));
sendJson(response, 500, { sendJson(response, 500, {
error: error instanceof Error ? error.message : 'Failed to read network data' error: error instanceof Error ? error.message : 'Failed to read network data',
writable: false,
writableReason: writableState.reason
}); });
} }
return; return;
+8
View File
@@ -1,6 +1,14 @@
// See https://svelte.dev/docs/kit/types#app.d.ts // See https://svelte.dev/docs/kit/types#app.d.ts
// for information about these interfaces // for information about these interfaces
declare global { declare global {
interface ImportMetaEnv {
readonly OND_STATIC_DEMO: boolean;
}
interface ImportMeta {
readonly env: ImportMetaEnv;
}
namespace App { namespace App {
// interface Error {} // interface Error {}
// interface Locals {} // interface Locals {}
+13 -1
View File
@@ -4,6 +4,7 @@
export let isSampleData = false; export let isSampleData = false;
export let writable = false; export let writable = false;
export let hasMachines = false; export let hasMachines = false;
export let storageReady = false;
const dispatch = createEventDispatcher<{ const dispatch = createEventDispatcher<{
addmachine: void; addmachine: void;
@@ -15,7 +16,11 @@
<div class="first-run"> <div class="first-run">
<h2>Map your network</h2> <h2>Map your network</h2>
{#if isSampleData} {#if storageReady}
<p class="storage-note">
Your network data file is ready. Add your first machine to start mapping.
</p>
{:else if isSampleData}
<p class="sample-note">This is sample data — replace it with your own machines.</p> <p class="sample-note">This is sample data — replace it with your own machines.</p>
{/if} {/if}
<div class="actions"> <div class="actions">
@@ -82,6 +87,13 @@
color: var(--status-warn); color: var(--status-warn);
} }
.storage-note {
margin: 0;
font-size: 12.5px;
line-height: 1.5;
color: var(--text-2);
}
.actions { .actions {
display: flex; display: flex;
flex-direction: column; flex-direction: column;
+70 -43
View File
@@ -57,6 +57,8 @@
| null; | null;
const autosaveDelayMs = 800; const autosaveDelayMs = 800;
const staticDemoMode = import.meta.env.OND_STATIC_DEMO;
const staticDemoReason = 'Static demo deployment; changes are not persisted.';
let container: HTMLDivElement; let container: HTMLDivElement;
let diagramStage: HTMLDivElement; let diagramStage: HTMLDivElement;
@@ -299,13 +301,19 @@
let subnetEditorOpen = false; let subnetEditorOpen = false;
let showRackManager = false; let showRackManager = false;
// First-run card: shown for an empty dataset or the untouched example // Writable empty datasets use a session-only dismissal so onboarding
// file, until dismissed (persisted) or the user makes their first edit. // returns after reload until the user adds a real entity. Sample-data
// dismissal remains persisted for returning demo users.
const firstRunDismissKey = 'ond-firstrun-dismissed'; const firstRunDismissKey = 'ond-firstrun-dismissed';
let firstRunDismissed = let firstRunDismissed =
typeof localStorage !== 'undefined' && localStorage.getItem(firstRunDismissKey) === '1'; typeof localStorage !== 'undefined' && localStorage.getItem(firstRunDismissKey) === '1';
let emptyFirstRunDismissedThisSession = false;
function dismissFirstRun() { function dismissFirstRun() {
if (writable && totalEntities === 0) {
emptyFirstRunDismissedThisSession = true;
return;
}
firstRunDismissed = true; firstRunDismissed = true;
try { try {
localStorage.setItem(firstRunDismissKey, '1'); localStorage.setItem(firstRunDismissKey, '1');
@@ -329,9 +337,9 @@
].every((name) => name.trim().toLowerCase().startsWith('example')); ].every((name) => name.trim().toLowerCase().startsWith('example'));
$: showFirstRun = $: showFirstRun =
hasLoadedInitialData && hasLoadedInitialData &&
!firstRunDismissed &&
!isLoadingData && !isLoadingData &&
(totalEntities === 0 || (totalEntities <= 4 && allExampleNames)); ((writable && totalEntities === 0 && !emptyFirstRunDismissedThisSession) ||
(!firstRunDismissed && totalEntities <= 4 && allExampleNames));
let copiedIp: string | null = null; let copiedIp: string | null = null;
let copiedIpTimer: ReturnType<typeof setTimeout> | null = null; let copiedIpTimer: ReturnType<typeof setTimeout> | null = null;
let pendingSubnetRemoval: number | null = null; let pendingSubnetRemoval: number | null = null;
@@ -695,6 +703,13 @@
return `Read-only: ${reason}`; return `Read-only: ${reason}`;
} }
function resolveStaticFallbackNotice(reason: string | null): string {
if (!reason) {
return 'Read-only: API unavailable; using bundled static data.';
}
return `Read-only: ${reason} Using bundled static data.`;
}
// Nodes are invisible hit targets (interaction, layout, edge anchoring); // Nodes are invisible hit targets (interaction, layout, edge anchoring);
// the visible cards live in NodeCardLayer. Only edges are canvas-drawn, // the visible cards live in NodeCardLayer. Only edges are canvas-drawn,
// styled from the same CSS tokens as the chrome. // styled from the same CSS tokens as the chrome.
@@ -1257,6 +1272,7 @@
return; return;
} }
firstRunDismissed = true; firstRunDismissed = true;
emptyFirstRunDismissedThisSession = true;
saveError = null; saveError = null;
saveState = 'unsaved'; saveState = 'unsaved';
if (!writable) { if (!writable) {
@@ -1898,46 +1914,56 @@
isLoadingData = true; isLoadingData = true;
loadError = null; loadError = null;
saveError = null; saveError = null;
let apiFallbackReason: string | null = staticDemoMode ? staticDemoReason : null;
try { if (!staticDemoMode) {
const response = await fetch('/api/network-data', { try {
headers: { Accept: 'application/json' } const response = await fetch('/api/network-data', {
}); headers: { Accept: 'application/json' }
if (!response.ok) { });
throw new Error(`API responded with ${response.status}`); if (!response.ok) {
} const body = (await response.json().catch(() => ({}))) as {
const body = (await response.json()) as { error?: string;
data: NetworkData; writableReason?: string | null;
writable: boolean; };
writableReason?: string | null; apiFallbackReason =
source: string; body.writableReason ?? body.error ?? `API responded with ${response.status}`;
updatedAt: string; throw new Error(apiFallbackReason);
}; }
const validation = validateNetworkData(body.data); const body = (await response.json()) as {
if (!validation.valid || !validation.data) { data: NetworkData;
throw new Error('API returned invalid network data'); writable: boolean;
} writableReason?: string | null;
source: string;
updatedAt: string;
};
const validation = validateNetworkData(body.data);
if (!validation.valid || !validation.data) {
throw new Error('API returned invalid network data');
}
networkData = cloneNetworkData(validation.data); networkData = cloneNetworkData(validation.data);
ensureSelectedTargetValid(); ensureSelectedTargetValid();
writable = body.writable; writable = body.writable;
readOnlyNotice = body.writable readOnlyNotice = body.writable
? defaultReadOnlyNotice ? defaultReadOnlyNotice
: resolveReadOnlyNotice(body.writableReason); : resolveReadOnlyNotice(body.writableReason);
dataSourceLabel = body.source; dataSourceLabel = body.source;
lastSavedSnapshot = JSON.stringify(networkData); lastSavedSnapshot = JSON.stringify(networkData);
saveState = 'saved'; saveState = 'saved';
connectionDraftByKey = {}; connectionDraftByKey = {};
revalidateDraft(); revalidateDraft();
refreshGraph(); refreshGraph();
hasLoadedInitialData = true; hasLoadedInitialData = true;
isLoadingData = false; isLoadingData = false;
return; return;
} catch (apiError) { } catch (apiError) {
console.warn( apiFallbackReason ??= resolveErrorMessage(apiError);
'[OpenNetworkDiagram] API load unavailable, falling back to static JSON', console.warn(
apiError '[OpenNetworkDiagram] API load unavailable, falling back to static JSON',
); apiError
);
}
} }
try { try {
@@ -1945,7 +1971,7 @@
networkData = cloneNetworkData(fallbackData); networkData = cloneNetworkData(fallbackData);
ensureSelectedTargetValid(); ensureSelectedTargetValid();
writable = false; writable = false;
readOnlyNotice = 'Read-only: API unavailable; using bundled static data.'; readOnlyNotice = resolveStaticFallbackNotice(apiFallbackReason);
dataSourceLabel = jsonPath; dataSourceLabel = jsonPath;
lastSavedSnapshot = JSON.stringify(networkData); lastSavedSnapshot = JSON.stringify(networkData);
saveState = 'saved'; saveState = 'saved';
@@ -2501,6 +2527,7 @@
isSampleData={allExampleNames} isSampleData={allExampleNames}
{writable} {writable}
hasMachines={networkData.machines.length > 0} hasMachines={networkData.machines.length > 0}
storageReady={writable && totalEntities === 0}
on:addmachine={addMachine} on:addmachine={addMachine}
on:connectport={openFirstMachineEditor} on:connectport={openFirstMachineEditor}
on:openipam={() => (showIpamPanel = true)} on:openipam={() => (showIpamPanel = true)}
+10
View File
@@ -2,6 +2,7 @@ import {
checkWritableState as checkWritableStateCore, checkWritableState as checkWritableStateCore,
getWritableState as getWritableStateCore, getWritableState as getWritableStateCore,
isWriteEnabled as isWriteEnabledCore, isWriteEnabled as isWriteEnabledCore,
readOrInitializeNetworkFile as readOrInitializeNetworkFileCore,
readNetworkFile as readNetworkFileCore, readNetworkFile as readNetworkFileCore,
writeNetworkFile as writeNetworkFileCore writeNetworkFile as writeNetworkFileCore
} from '../shared/networkPersistenceCore.mjs'; } from '../shared/networkPersistenceCore.mjs';
@@ -18,6 +19,11 @@ export interface WritableState {
reason: string | null; reason: string | null;
} }
export interface NetworkFileBootstrapResult extends NetworkFileMetadata {
data: NetworkData;
initialized: boolean;
}
export function isWriteEnabled(): boolean { export function isWriteEnabled(): boolean {
return isWriteEnabledCore(); return isWriteEnabledCore();
} }
@@ -26,6 +32,10 @@ export async function readNetworkFile(): Promise<{ data: NetworkData } & Network
return (await readNetworkFileCore()) as { data: NetworkData } & NetworkFileMetadata; return (await readNetworkFileCore()) as { data: NetworkData } & NetworkFileMetadata;
} }
export async function readOrInitializeNetworkFile(): Promise<NetworkFileBootstrapResult> {
return (await readOrInitializeNetworkFileCore()) as NetworkFileBootstrapResult;
}
export async function writeNetworkFile(data: NetworkData): Promise<NetworkFileMetadata> { export async function writeNetworkFile(data: NetworkData): Promise<NetworkFileMetadata> {
return (await writeNetworkFileCore(data)) as NetworkFileMetadata; return (await writeNetworkFileCore(data)) as NetworkFileMetadata;
} }
+92 -5
View File
@@ -16,12 +16,19 @@ import path from 'node:path';
* @typedef {{ machines: unknown[]; devices: unknown[] }} NetworkData * @typedef {{ machines: unknown[]; devices: unknown[] }} NetworkData
* @typedef {{ source: string; updatedAt: string }} NetworkFileMetadata * @typedef {{ source: string; updatedAt: string }} NetworkFileMetadata
* @typedef {{ data: NetworkData } & NetworkFileMetadata} NetworkFileReadResult * @typedef {{ data: NetworkData } & NetworkFileMetadata} NetworkFileReadResult
* @typedef {{ status: 'ready'; data: NetworkData; source: string; updatedAt: string } | { status: 'missing' | 'blank'; source: string; updatedAt: string | null }} NetworkFileReadState
* @typedef {NetworkFileReadResult & { initialized: boolean }} NetworkFileBootstrapResult
* @typedef {{ writable: boolean; reason: string | null }} WritableState * @typedef {{ writable: boolean; reason: string | null }} WritableState
* @typedef {{ createBackup?: boolean }} WriteNetworkFileOptions
*/ */
const NETWORK_READ_ONLY = process.env.NETWORK_READ_ONLY === 'true'; const NETWORK_READ_ONLY = process.env.NETWORK_READ_ONLY === 'true';
const DATA_FILE_DEFAULT = path.resolve(process.cwd(), 'data/network.json'); const DATA_FILE_DEFAULT = path.resolve(process.cwd(), 'data/network.json');
const BACKUP_DIR_DEFAULT = path.resolve(process.cwd(), 'data/.backups'); const BACKUP_DIR_DEFAULT = path.resolve(process.cwd(), 'data/.backups');
const EMPTY_NETWORK_DATA = Object.freeze({
machines: Object.freeze([]),
devices: Object.freeze([])
});
/** /**
* @returns {string} * @returns {string}
@@ -86,20 +93,57 @@ async function readUpdatedAt(dataFilePath) {
} }
/** /**
* @returns {Promise<NetworkFileReadResult>} * @returns {Promise<NetworkFileReadState>}
*/ */
export async function readNetworkFile() { export async function readNetworkFileState() {
const source = resolveDataFilePath(); const source = resolveDataFilePath();
const raw = await readFile(source, 'utf8'); let raw;
try {
raw = await readFile(source, 'utf8');
} catch (error) {
if (formatFileSystemErrorCode(error) === 'ENOENT') {
return {
status: 'missing',
source,
updatedAt: null
};
}
throw error;
}
if (raw.trim().length === 0) {
return {
status: 'blank',
source,
updatedAt: await readUpdatedAt(source)
};
}
const parsed = JSON.parse(raw); const parsed = JSON.parse(raw);
const updatedAt = await readUpdatedAt(source); const updatedAt = await readUpdatedAt(source);
return { return {
status: 'ready',
data: parsed, data: parsed,
source, source,
updatedAt updatedAt
}; };
} }
/**
* @returns {Promise<NetworkFileReadResult>}
*/
export async function readNetworkFile() {
const state = await readNetworkFileState();
if (state.status !== 'ready') {
throw new Error(`Network data file "${state.source}" is ${state.status}.`);
}
return {
data: state.data,
source: state.source,
updatedAt: state.updatedAt
};
}
/** /**
* @param {string} dataFilePath * @param {string} dataFilePath
* @param {string} backupDirectory * @param {string} backupDirectory
@@ -149,14 +193,21 @@ export async function trimBackups(backupDirectory, baseName, keep = 5) {
/** /**
* @param {NetworkData} data * @param {NetworkData} data
* @param {WriteNetworkFileOptions} [options]
* @returns {Promise<NetworkFileMetadata>} * @returns {Promise<NetworkFileMetadata>}
*/ */
export async function writeNetworkFile(data) { export async function writeNetworkFile(data, options) {
if (!isWriteEnabled()) {
throw new Error('Writes disabled by NETWORK_READ_ONLY=true.');
}
const source = resolveDataFilePath(); const source = resolveDataFilePath();
const backupDirectory = resolveBackupDirectory(); const backupDirectory = resolveBackupDirectory();
const directory = path.dirname(source); const directory = path.dirname(source);
await mkdir(directory, { recursive: true }); await mkdir(directory, { recursive: true });
await createBackupIfPresent(source, backupDirectory); if (options?.createBackup ?? true) {
await createBackupIfPresent(source, backupDirectory);
}
const temporaryPath = `${source}.tmp-${Date.now()}-${process.pid}`; const temporaryPath = `${source}.tmp-${Date.now()}-${process.pid}`;
const jsonPayload = `${JSON.stringify(data, null, '\t')}\n`; const jsonPayload = `${JSON.stringify(data, null, '\t')}\n`;
@@ -244,3 +295,39 @@ export async function getWritableState() {
export async function checkWritableState() { export async function checkWritableState() {
return (await getWritableState()).writable; return (await getWritableState()).writable;
} }
/**
* Reads configured network data, creating a valid blank dataset only when the
* file is missing/blank and the deployment has confirmed writable storage.
*
* @returns {Promise<NetworkFileBootstrapResult>}
*/
export async function readOrInitializeNetworkFile() {
const state = await readNetworkFileState();
if (state.status === 'ready') {
return {
data: state.data,
source: state.source,
updatedAt: state.updatedAt,
initialized: false
};
}
const writableState = await getWritableState();
if (!writableState.writable) {
const suffix = writableState.reason ? ` ${writableState.reason}` : '';
throw new Error(`Network data file "${state.source}" is ${state.status}.${suffix}`);
}
const data = {
machines: [...EMPTY_NETWORK_DATA.machines],
devices: [...EMPTY_NETWORK_DATA.devices]
};
const metadata = await writeNetworkFile(data, { createBackup: false });
return {
data,
source: metadata.source,
updatedAt: metadata.updatedAt,
initialized: true
};
}
+45 -6
View File
@@ -3,11 +3,14 @@ import { json } from '@sveltejs/kit';
import { validateNetworkData } from '$lib/data/networkSchema'; import { validateNetworkData } from '$lib/data/networkSchema';
import { import {
getWritableState, getWritableState,
readNetworkFile, readOrInitializeNetworkFile,
writeNetworkFile writeNetworkFile
} from '$lib/server/networkPersistence'; } from '$lib/server/networkPersistence';
import type { RequestHandler } from './$types'; import type { RequestHandler } from './$types';
const staticDemoMode = import.meta.env.OND_STATIC_DEMO;
const staticDemoReason = 'Static demo deployment; changes are not persisted.';
function serializeValidationErrors(errors: Array<{ path: string; message: string }>) { function serializeValidationErrors(errors: Array<{ path: string; message: string }>) {
return errors.map((issue) => ({ return errors.map((issue) => ({
path: issue.path, path: issue.path,
@@ -22,10 +25,35 @@ function resolveReadOnlyErrorMessage(reason: string | null): string {
return `Write API unavailable: ${reason}`; return `Write API unavailable: ${reason}`;
} }
async function resolveFailedReadPayload(error: unknown) {
const writableState = await getWritableState().catch(() => ({
writable: false,
reason: null
}));
const message =
error instanceof Error && error.message ? error.message : 'Failed to read network data file';
return {
error: message,
writable: false,
writableReason: writableState.reason
};
}
export const GET: RequestHandler = async () => { export const GET: RequestHandler = async () => {
if (staticDemoMode) {
return json(
{
error: staticDemoReason,
writable: false,
writableReason: staticDemoReason
},
{ status: 503 }
);
}
try { try {
const [{ data, source, updatedAt }, writableState] = await Promise.all([ const [{ data, source, updatedAt }, writableState] = await Promise.all([
readNetworkFile(), readOrInitializeNetworkFile(),
getWritableState() getWritableState()
]); ]);
const validation = validateNetworkData(data); const validation = validateNetworkData(data);
@@ -33,7 +61,9 @@ export const GET: RequestHandler = async () => {
return json( return json(
{ {
error: 'Stored network data is invalid', error: 'Stored network data is invalid',
details: serializeValidationErrors(validation.errors) details: serializeValidationErrors(validation.errors),
writable: false,
writableReason: writableState.reason
}, },
{ status: 500 } { status: 500 }
); );
@@ -47,13 +77,22 @@ export const GET: RequestHandler = async () => {
updatedAt updatedAt
}); });
} catch (error) { } catch (error) {
const message = return json(await resolveFailedReadPayload(error), { status: 500 });
error instanceof Error && error.message ? error.message : 'Failed to read network data file';
return json({ error: message }, { status: 500 });
} }
}; };
export const PUT: RequestHandler = async ({ request }) => { export const PUT: RequestHandler = async ({ request }) => {
if (staticDemoMode) {
return json(
{
error: resolveReadOnlyErrorMessage(staticDemoReason),
writable: false,
writableReason: staticDemoReason
},
{ status: 403 }
);
}
const writableStateBeforeWrite = await getWritableState(); const writableStateBeforeWrite = await getWritableState();
if (!writableStateBeforeWrite.writable) { if (!writableStateBeforeWrite.writable) {
return json( return json(
+163
View File
@@ -0,0 +1,163 @@
import assert from 'node:assert/strict';
import { chmod, mkdir, mkdtemp, readFile, readdir, rm, stat, writeFile } from 'node:fs/promises';
import os from 'node:os';
import path from 'node:path';
import test from 'node:test';
let moduleSequence = 0;
function restoreEnvironment(name, value) {
if (value === undefined) {
delete process.env[name];
return;
}
process.env[name] = value;
}
async function withPersistenceEnvironment(run, options = {}) {
const root = await mkdtemp(path.join(os.tmpdir(), 'ond-persistence-'));
const source = path.join(root, 'data', 'network.json');
const backupDirectory = path.join(root, 'backups');
const previousDataFile = process.env.NETWORK_DATA_FILE;
const previousBackupDirectory = process.env.NETWORK_BACKUP_DIR;
const previousReadOnly = process.env.NETWORK_READ_ONLY;
process.env.NETWORK_DATA_FILE = source;
process.env.NETWORK_BACKUP_DIR = backupDirectory;
process.env.NETWORK_READ_ONLY = options.readOnly ? 'true' : 'false';
moduleSequence += 1;
try {
const persistence = await import(
`../src/lib/shared/networkPersistenceCore.mjs?test=${moduleSequence}`
);
await run({ persistence, root, source, backupDirectory });
} finally {
restoreEnvironment('NETWORK_DATA_FILE', previousDataFile);
restoreEnvironment('NETWORK_BACKUP_DIR', previousBackupDirectory);
restoreEnvironment('NETWORK_READ_ONLY', previousReadOnly);
await rm(root, { recursive: true, force: true });
}
}
test('initializes a missing writable data file with a blank network', async () => {
await withPersistenceEnvironment(async ({ persistence, source, backupDirectory }) => {
const result = await persistence.readOrInitializeNetworkFile();
assert.equal(result.initialized, true);
assert.deepEqual(result.data, { machines: [], devices: [] });
assert.equal(result.source, source);
assert.deepEqual(JSON.parse(await readFile(source, 'utf8')), {
machines: [],
devices: []
});
assert.deepEqual(await readdir(backupDirectory), []);
});
});
async function assertInitializesBlankFile(contents) {
await withPersistenceEnvironment(async ({ persistence, source }) => {
await persistence.writeNetworkFile({ machines: [], devices: [] }, { createBackup: false });
await writeFile(source, contents, 'utf8');
const result = await persistence.readOrInitializeNetworkFile();
assert.equal(result.initialized, true);
assert.deepEqual(JSON.parse(await readFile(source, 'utf8')), {
machines: [],
devices: []
});
});
}
test('initializes zero-byte and whitespace-only writable data files', async () => {
await assertInitializesBlankFile('');
await assertInitializesBlankFile(' \n\t');
});
test('does not rewrite an existing valid empty network', async () => {
await withPersistenceEnvironment(async ({ persistence, source }) => {
await persistence.writeNetworkFile({ machines: [], devices: [] }, { createBackup: false });
const before = await stat(source);
const result = await persistence.readOrInitializeNetworkFile();
const after = await stat(source);
assert.equal(result.initialized, false);
assert.deepEqual(result.data, { machines: [], devices: [] });
assert.equal(after.mtimeMs, before.mtimeMs);
});
});
test('does not replace malformed or schema-invalid non-empty JSON', async () => {
await withPersistenceEnvironment(async ({ persistence, source }) => {
await persistence.writeNetworkFile({ machines: [], devices: [] }, { createBackup: false });
await writeFile(source, '{not json', 'utf8');
await assert.rejects(persistence.readOrInitializeNetworkFile(), SyntaxError);
assert.equal(await readFile(source, 'utf8'), '{not json');
});
await withPersistenceEnvironment(async ({ persistence, source }) => {
await persistence.writeNetworkFile({ machines: [], devices: [] }, { createBackup: false });
await writeFile(source, '{}\n', 'utf8');
const result = await persistence.readOrInitializeNetworkFile();
assert.equal(result.initialized, false);
assert.deepEqual(result.data, {});
assert.equal(await readFile(source, 'utf8'), '{}\n');
});
});
test('never initializes missing or blank files in explicit read-only mode', async () => {
await withPersistenceEnvironment(
async ({ persistence, source }) => {
await assert.rejects(
persistence.readOrInitializeNetworkFile(),
/Writes disabled by NETWORK_READ_ONLY=true/
);
await assert.rejects(stat(source), { code: 'ENOENT' });
},
{ readOnly: true }
);
await withPersistenceEnvironment(
async ({ persistence, source }) => {
await mkdir(path.dirname(source), { recursive: true });
await writeFile(source, ' \n', 'utf8');
await assert.rejects(
persistence.readOrInitializeNetworkFile(),
/Writes disabled by NETWORK_READ_ONLY=true/
);
assert.equal(await readFile(source, 'utf8'), ' \n');
},
{ readOnly: true }
);
});
test(
'unwritable blank files remain untouched and report the permission failure',
{ skip: process.platform === 'win32' },
async () => {
await withPersistenceEnvironment(async ({ persistence, source }) => {
await persistence.writeNetworkFile({ machines: [], devices: [] }, { createBackup: false });
await writeFile(source, '', 'utf8');
await chmod(source, 0o400);
try {
const writableState = await persistence.getWritableState();
assert.equal(writableState.writable, false);
assert.match(writableState.reason, /not readable and writable/);
await assert.rejects(
persistence.readOrInitializeNetworkFile(),
/not readable and writable/
);
assert.equal(await readFile(source, 'utf8'), '');
} finally {
await chmod(source, 0o600);
}
});
}
);
+5
View File
@@ -2,6 +2,11 @@ import tailwindcss from '@tailwindcss/vite';
import { sveltekit } from '@sveltejs/kit/vite'; import { sveltekit } from '@sveltejs/kit/vite';
import { defineConfig } from 'vite'; import { defineConfig } from 'vite';
const staticDemoMode = process.env.OND_STATIC_DEMO === 'true';
export default defineConfig({ export default defineConfig({
define: {
'import.meta.env.OND_STATIC_DEMO': JSON.stringify(staticDemoMode)
},
plugins: [tailwindcss(), sveltekit()] plugins: [tailwindcss(), sveltekit()]
}); });