diff --git a/README.md b/README.md index ef7c8ce..d8297b9 100644 --- a/README.md +++ b/README.md @@ -8,7 +8,7 @@ **A declarative, self-hosted containerised tool for visualising and managing home lab & network architecture diagrams.** -Open Network Diagram helps you document your infrastructure in a visual UI while keeping a real JSON source of truth you can version, back up, and reuse. +Open Network Diagram helps you document your infrastructure in a visual UI while keeping a real JSON source of truth you can back up, reuse, and optionally version after reviewing it for sensitive information. - Homelab-friendly: run it in minutes with Docker. - Practical: edit in the UI and autosave to `network.json`. @@ -47,11 +47,10 @@ Open Network Diagram helps you document your infrastructure in a visual UI while This is the fastest way to run Open Network Diagram for a home lab. -1. Create a local data folder and seed your first `network.json`: +1. Create a local data folder: ```bash -mkdir -p ond-data -curl -fsSL https://raw.githubusercontent.com/jcreek/OpenNetworkDiagram/main/data/network.json.example -o ond-data/network.json +mkdir -p data ``` 2. Run the published Docker image: @@ -61,15 +60,18 @@ docker run -d \ --name open-network-diagram \ --restart unless-stopped \ -p 8080:3000 \ + --user "$(id -u):$(id -g)" \ -e NETWORK_DATA_FILE=/app/data/network.json \ -e NETWORK_BACKUP_DIR=/app/data/.backups \ - -v "$(pwd)/ond-data:/app/data" \ + -v "$(pwd)/data:/app/data" \ jcreek23/open-network-diagram:latest ``` +On first access, Open Network Diagram creates a valid blank `data/network.json` and shows the getting-started card. + 3. Open the app at `http://localhost:8080`. -4. Edit your topology in the UI. Changes persist to `ond-data/network.json`. +4. Add your first machine. Changes and rolling backups remain directly accessible in the local `data` directory across container updates and restarts. Useful follow-up commands: @@ -79,6 +81,29 @@ docker stop open-network-diagram docker rm open-network-diagram ``` +The `--user` option runs the container with your host UID and GID so files in the bind mount remain writable and owned by your account. Docker Desktop commonly handles bind-mount permissions without this mapping, so macOS users can omit `--user` if their Docker Desktop configuration requires the image's built-in user. If storage is not writable, the app leaves it untouched, loads the bundled read-only demo, and reports the exact permission problem in the UI. + +### Optional Sample Data + +The application starts with a blank network by default. To explore the sample topology instead, download it before starting the container: + +```bash +curl -fsSL https://raw.githubusercontent.com/jcreek/OpenNetworkDiagram/main/data/network.json.example \ + -o data/network.json +``` + +The sample is optional and is never required for startup. + +## Deployment Security + +Open Network Diagram does not include authentication or TLS. The quick-start port mapping `-p 8080:3000` listens on all host interfaces, so any device that can reach the host can view the topology and, in writable mode, change it. + +Do not expose a writable deployment directly to the internet. Restrict access with a firewall, LAN or VPN, or place the application behind an authenticated reverse proxy that terminates TLS. + +To make the service reachable only from the Docker host, replace the quick-start mapping with `-p 127.0.0.1:8080:3000`. + +For a public demonstration, add `-e NETWORK_READ_ONLY=true` to `docker run`, or add `NETWORK_READ_ONLY: 'true'` under Compose's `environment`. Read-only mode prevents changes and uses the bundled demo if the configured data file is missing, but it does not hide a configured topology or replace authentication. + ## What It Looks Like | Network view with ethernet labels | Hosts & VMs view with VMs expanded | Modal editing a machine | @@ -89,6 +114,10 @@ docker rm open-network-diagram | --------------------------------------------------------------------------- | ------------------------------------------------------------------- | -------------------------------------------------- | | ![Open Network Diagram IPAM panel with subnet utilization](screenshot4.png) | ![Open Network Diagram rack view with shelf items](screenshot5.png) | ![Open Network Diagram dark mode](screenshot6.png) | +| Zero-setup first run | Read-only live demo | +| ------------------------------------------------------------------------ | ------------------------------------------------------------ | +| ![Open Network Diagram zero-setup first-run onboarding](screenshot7.png) | ![Open Network Diagram read-only live demo](screenshot8.png) | + ## Docker Compose Option If you prefer compose: @@ -97,10 +126,11 @@ If you prefer compose: services: open-network-diagram: image: jcreek23/open-network-diagram:latest + user: '${OND_UID:?Set OND_UID to your host user ID}:${OND_GID:?Set OND_GID to your host group ID}' ports: - '8080:3000' volumes: - - ./ond-data:/app/data + - ./data:/app/data environment: NETWORK_DATA_FILE: /app/data/network.json NETWORK_BACKUP_DIR: /app/data/.backups @@ -110,9 +140,40 @@ services: Start it with: ```bash -docker compose up -d +mkdir -p data +OND_UID="$(id -u)" OND_GID="$(id -g)" docker compose up -d ``` +`OND_UID` and `OND_GID` are required Compose interpolation values supplied by the command above; they are not application environment variables. Docker Desktop users can remove the `user` line if their file-sharing configuration requires the image's built-in user. + +### Using a Docker Named Volume + +If host-directory permissions are inconvenient, Docker can manage the storage volume instead: + +```bash +docker run -d \ + --name open-network-diagram \ + --restart unless-stopped \ + -p 8080:3000 \ + -e NETWORK_DATA_FILE=/app/data/network.json \ + -e NETWORK_BACKUP_DIR=/app/data/.backups \ + -v ond-data:/app/data \ + jcreek23/open-network-diagram:latest +``` + +Docker creates the `ond-data` volume automatically. Use `docker volume inspect ond-data` to inspect its location or `docker cp open-network-diagram:/app/data/network.json ./network.json` to copy the data file out. + +### Versioning Your Topology + +`data/network.json` and `data/.backups` are ignored by Git because topology data may expose host names, addresses, MAC addresses, and other sensitive infrastructure details. If you have reviewed and sanitized the data and deliberately want to track it, opt in with: + +```bash +git add -f data/network.json +git commit -m "Track network topology" +``` + +Once the file is tracked, normal Git commands include later changes. Backup files remain ignored. + ## For Developers ### Local Development @@ -139,6 +200,8 @@ pnpm run icons:manifest # regenerate local vendor icon manifest - API endpoint: `GET/PUT /api/network-data` - Writes are enabled unless `NETWORK_READ_ONLY=true` +- Writable deployments automatically initialize a missing or whitespace-only data file with a blank network. +- Read-only deployments never initialize or modify the configured data file and continue to use the bundled demo when the API data is unavailable. - When writes are unavailable, API responses include `writableReason` for diagnostics. - Writes are persisted atomically to the configured data file - Rolling backups are kept in the backup directory (last 5) @@ -232,7 +295,7 @@ OpenNetworkDiagram/ ├── src/lib/config/vendorIconManifest.ts # Generated local icon catalog ├── static/data/network.json # Demo dataset (Netlify) ├── static/icons/vendor/ # Vendored icon assets (runtime-local) -├── data/network.json.example # Starter data template for Docker users +├── data/network.json.example # Optional starter data template ├── third_party/ # Third-party provenance + licensing ├── Dockerfile # Docker build/runtime image ├── server.mjs # Node runtime server (static + API) @@ -252,9 +315,19 @@ OpenNetworkDiagram/ 1. Fork the repository. 2. Create a feature branch. -3. Commit your changes. -4. Push your branch. -5. Open a pull request. +3. Before committing, run: + +```bash +pnpm lint +pnpm test +pnpm check +pnpm run build:docker +pnpm run build:netlify +``` + +4. Commit your changes. +5. Push your branch. +6. Open a pull request. ## License diff --git a/docker-compose.yml b/docker-compose.yml index 2400de4..9c344d8 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -4,6 +4,7 @@ services: context: . dockerfile: Dockerfile image: open-network-diagram:local + user: '${OND_UID:?Set OND_UID to your host user ID}:${OND_GID:?Set OND_GID to your host group ID}' ports: - '8080:3000' volumes: diff --git a/package.json b/package.json index 5050675..d3e69bd 100644 --- a/package.json +++ b/package.json @@ -7,9 +7,10 @@ "dev": "vite dev", "build": "vite build", "build:docker": "DEPLOY_TARGET=docker vite build", - "build:netlify": "NETWORK_READ_ONLY=true DEPLOY_TARGET=netlify vite build", + "build:netlify": "NETWORK_READ_ONLY=true OND_STATIC_DEMO=true DEPLOY_TARGET=netlify vite build", "start": "node server.mjs", "preview": "vite preview", + "test": "node --test tests/*.test.mjs", "prepare": "svelte-kit sync || echo ''", "check": "svelte-kit sync && svelte-check --tsconfig ./tsconfig.json", "check:watch": "svelte-kit sync && svelte-check --tsconfig ./tsconfig.json --watch", diff --git a/screenshot7.png b/screenshot7.png new file mode 100644 index 0000000..7ded9f3 Binary files /dev/null and b/screenshot7.png differ diff --git a/screenshot8.png b/screenshot8.png new file mode 100644 index 0000000..ae67128 Binary files /dev/null and b/screenshot8.png differ diff --git a/server.mjs b/server.mjs index 81cac72..825ddf7 100644 --- a/server.mjs +++ b/server.mjs @@ -5,7 +5,7 @@ import { fileURLToPath } from 'node:url'; import { validateNetworkData } from './src/lib/shared/networkSchemaCore.mjs'; import { getWritableState, - readNetworkFile, + readOrInitializeNetworkFile, writeNetworkFile } from './src/lib/shared/networkPersistenceCore.mjs'; @@ -59,12 +59,17 @@ function resolveReadOnlyErrorMessage(reason) { async function serveApi(request, response) { if (request.method === 'GET') { try { - const [payload, writableState] = await Promise.all([readNetworkFile(), getWritableState()]); + const [payload, writableState] = await Promise.all([ + readOrInitializeNetworkFile(), + getWritableState() + ]); const validation = validateNetworkData(payload.data); if (!validation.valid) { sendJson(response, 500, { error: 'Stored network data is invalid', - details: validation.errors + details: validation.errors, + writable: false, + writableReason: writableState.reason }); return; } @@ -77,8 +82,14 @@ async function serveApi(request, response) { updatedAt: payload.updatedAt }); } catch (error) { + const writableState = await getWritableState().catch(() => ({ + writable: false, + reason: null + })); sendJson(response, 500, { - error: error instanceof Error ? error.message : 'Failed to read network data' + error: error instanceof Error ? error.message : 'Failed to read network data', + writable: false, + writableReason: writableState.reason }); } return; diff --git a/src/app.d.ts b/src/app.d.ts index da08e6d..5cbf5a0 100644 --- a/src/app.d.ts +++ b/src/app.d.ts @@ -1,6 +1,14 @@ // See https://svelte.dev/docs/kit/types#app.d.ts // for information about these interfaces declare global { + interface ImportMetaEnv { + readonly OND_STATIC_DEMO: boolean; + } + + interface ImportMeta { + readonly env: ImportMetaEnv; + } + namespace App { // interface Error {} // interface Locals {} diff --git a/src/lib/components/FirstRunCard.svelte b/src/lib/components/FirstRunCard.svelte index bdd22e9..2654571 100644 --- a/src/lib/components/FirstRunCard.svelte +++ b/src/lib/components/FirstRunCard.svelte @@ -4,6 +4,7 @@ export let isSampleData = false; export let writable = false; export let hasMachines = false; + export let storageReady = false; const dispatch = createEventDispatcher<{ addmachine: void; @@ -15,7 +16,11 @@

Map your network

- {#if isSampleData} + {#if storageReady} +

+ Your network data file is ready. Add your first machine to start mapping. +

+ {:else if isSampleData}

This is sample data — replace it with your own machines.

{/if}
@@ -82,6 +87,13 @@ color: var(--status-warn); } + .storage-note { + margin: 0; + font-size: 12.5px; + line-height: 1.5; + color: var(--text-2); + } + .actions { display: flex; flex-direction: column; diff --git a/src/lib/components/NetworkDiagram.svelte b/src/lib/components/NetworkDiagram.svelte index b8844a4..b7c38bf 100644 --- a/src/lib/components/NetworkDiagram.svelte +++ b/src/lib/components/NetworkDiagram.svelte @@ -57,6 +57,8 @@ | null; const autosaveDelayMs = 800; + const staticDemoMode = import.meta.env.OND_STATIC_DEMO; + const staticDemoReason = 'Static demo deployment; changes are not persisted.'; let container: HTMLDivElement; let diagramStage: HTMLDivElement; @@ -299,13 +301,22 @@ let subnetEditorOpen = false; let showRackManager = false; - // First-run card: shown for an empty dataset or the untouched example - // file, until dismissed (persisted) or the user makes their first edit. + $: totalEntities = networkData.machines.length + networkData.devices.length; + $: emptyWritableDataset = writable && totalEntities === 0; + + // Writable empty datasets use a session-only dismissal so onboarding + // returns after reload until the user adds a real entity. Sample-data + // dismissal remains persisted for returning demo users. const firstRunDismissKey = 'ond-firstrun-dismissed'; let firstRunDismissed = typeof localStorage !== 'undefined' && localStorage.getItem(firstRunDismissKey) === '1'; + let emptyFirstRunDismissedThisSession = false; function dismissFirstRun() { + if (emptyWritableDataset) { + emptyFirstRunDismissedThisSession = true; + return; + } firstRunDismissed = true; try { localStorage.setItem(firstRunDismissKey, '1'); @@ -320,7 +331,6 @@ } } - $: totalEntities = networkData.machines.length + networkData.devices.length; $: allExampleNames = totalEntities > 0 && [ @@ -329,9 +339,9 @@ ].every((name) => name.trim().toLowerCase().startsWith('example')); $: showFirstRun = hasLoadedInitialData && - !firstRunDismissed && !isLoadingData && - (totalEntities === 0 || (totalEntities <= 4 && allExampleNames)); + ((emptyWritableDataset && !emptyFirstRunDismissedThisSession) || + (!firstRunDismissed && totalEntities <= 4 && allExampleNames)); let copiedIp: string | null = null; let copiedIpTimer: ReturnType | null = null; let pendingSubnetRemoval: number | null = null; @@ -695,6 +705,15 @@ return `Read-only: ${reason}`; } + function resolveStaticFallbackNotice(reason: string | null): string { + const normalizedReason = reason?.trim(); + if (!normalizedReason) { + return 'Read-only: API unavailable; using bundled static data.'; + } + const separator = /[.!?]$/.test(normalizedReason) ? ' ' : '. '; + return `Read-only: ${normalizedReason}${separator}Using bundled static data.`; + } + // Nodes are invisible hit targets (interaction, layout, edge anchoring); // the visible cards live in NodeCardLayer. Only edges are canvas-drawn, // styled from the same CSS tokens as the chrome. @@ -1257,6 +1276,7 @@ return; } firstRunDismissed = true; + emptyFirstRunDismissedThisSession = true; saveError = null; saveState = 'unsaved'; if (!writable) { @@ -1898,46 +1918,56 @@ isLoadingData = true; loadError = null; saveError = null; + let apiFallbackReason: string | null = staticDemoMode ? staticDemoReason : null; - try { - const response = await fetch('/api/network-data', { - headers: { Accept: 'application/json' } - }); - if (!response.ok) { - throw new Error(`API responded with ${response.status}`); - } - const body = (await response.json()) as { - data: NetworkData; - writable: boolean; - writableReason?: string | null; - source: string; - updatedAt: string; - }; - const validation = validateNetworkData(body.data); - if (!validation.valid || !validation.data) { - throw new Error('API returned invalid network data'); - } + if (!staticDemoMode) { + try { + const response = await fetch('/api/network-data', { + headers: { Accept: 'application/json' } + }); + if (!response.ok) { + const body = (await response.json().catch(() => ({}))) as { + error?: string; + writableReason?: string | null; + }; + apiFallbackReason = + body.writableReason ?? body.error ?? `API responded with ${response.status}`; + throw new Error(apiFallbackReason); + } + const body = (await response.json()) as { + data: NetworkData; + writable: boolean; + writableReason?: string | null; + source: string; + updatedAt: string; + }; + const validation = validateNetworkData(body.data); + if (!validation.valid || !validation.data) { + throw new Error('API returned invalid network data'); + } - networkData = cloneNetworkData(validation.data); - ensureSelectedTargetValid(); - writable = body.writable; - readOnlyNotice = body.writable - ? defaultReadOnlyNotice - : resolveReadOnlyNotice(body.writableReason); - dataSourceLabel = body.source; - lastSavedSnapshot = JSON.stringify(networkData); - saveState = 'saved'; - connectionDraftByKey = {}; - revalidateDraft(); - refreshGraph(); - hasLoadedInitialData = true; - isLoadingData = false; - return; - } catch (apiError) { - console.warn( - '[OpenNetworkDiagram] API load unavailable, falling back to static JSON', - apiError - ); + networkData = cloneNetworkData(validation.data); + ensureSelectedTargetValid(); + writable = body.writable; + readOnlyNotice = body.writable + ? defaultReadOnlyNotice + : resolveReadOnlyNotice(body.writableReason); + dataSourceLabel = body.source; + lastSavedSnapshot = JSON.stringify(networkData); + saveState = 'saved'; + connectionDraftByKey = {}; + revalidateDraft(); + refreshGraph(); + hasLoadedInitialData = true; + isLoadingData = false; + return; + } catch (apiError) { + apiFallbackReason ??= resolveErrorMessage(apiError); + console.warn( + '[OpenNetworkDiagram] API load unavailable, falling back to static JSON', + apiError + ); + } } try { @@ -1945,7 +1975,7 @@ networkData = cloneNetworkData(fallbackData); ensureSelectedTargetValid(); writable = false; - readOnlyNotice = 'Read-only: API unavailable; using bundled static data.'; + readOnlyNotice = resolveStaticFallbackNotice(apiFallbackReason); dataSourceLabel = jsonPath; lastSavedSnapshot = JSON.stringify(networkData); saveState = 'saved'; @@ -2501,6 +2531,7 @@ isSampleData={allExampleNames} {writable} hasMachines={networkData.machines.length > 0} + storageReady={emptyWritableDataset} on:addmachine={addMachine} on:connectport={openFirstMachineEditor} on:openipam={() => (showIpamPanel = true)} diff --git a/src/lib/server/networkPersistence.ts b/src/lib/server/networkPersistence.ts index 66e8164..27a598c 100644 --- a/src/lib/server/networkPersistence.ts +++ b/src/lib/server/networkPersistence.ts @@ -2,6 +2,7 @@ import { checkWritableState as checkWritableStateCore, getWritableState as getWritableStateCore, isWriteEnabled as isWriteEnabledCore, + readOrInitializeNetworkFile as readOrInitializeNetworkFileCore, readNetworkFile as readNetworkFileCore, writeNetworkFile as writeNetworkFileCore } from '../shared/networkPersistenceCore.mjs'; @@ -18,6 +19,11 @@ export interface WritableState { reason: string | null; } +export interface NetworkFileBootstrapResult extends NetworkFileMetadata { + data: NetworkData; + initialized: boolean; +} + export function isWriteEnabled(): boolean { return isWriteEnabledCore(); } @@ -26,6 +32,10 @@ export async function readNetworkFile(): Promise<{ data: NetworkData } & Network return (await readNetworkFileCore()) as { data: NetworkData } & NetworkFileMetadata; } +export async function readOrInitializeNetworkFile(): Promise { + return (await readOrInitializeNetworkFileCore()) as NetworkFileBootstrapResult; +} + export async function writeNetworkFile(data: NetworkData): Promise { return (await writeNetworkFileCore(data)) as NetworkFileMetadata; } diff --git a/src/lib/shared/networkPersistenceCore.mjs b/src/lib/shared/networkPersistenceCore.mjs index 4854508..95b9111 100644 --- a/src/lib/shared/networkPersistenceCore.mjs +++ b/src/lib/shared/networkPersistenceCore.mjs @@ -16,12 +16,25 @@ import path from 'node:path'; * @typedef {{ machines: unknown[]; devices: unknown[] }} NetworkData * @typedef {{ source: string; updatedAt: string }} NetworkFileMetadata * @typedef {{ data: NetworkData } & NetworkFileMetadata} NetworkFileReadResult + * @typedef {{ status: 'ready'; data: NetworkData; source: string; updatedAt: string } | { status: 'missing' | 'blank'; source: string; updatedAt: string | null }} NetworkFileReadState + * @typedef {NetworkFileReadResult & { initialized: boolean }} NetworkFileBootstrapResult * @typedef {{ writable: boolean; reason: string | null }} WritableState + * @typedef {{ createBackup?: boolean; beforeCommit?: () => Promise }} WriteNetworkFileOptions */ const NETWORK_READ_ONLY = process.env.NETWORK_READ_ONLY === 'true'; const DATA_FILE_DEFAULT = path.resolve(process.cwd(), 'data/network.json'); const BACKUP_DIR_DEFAULT = path.resolve(process.cwd(), 'data/.backups'); +const EMPTY_NETWORK_DATA = Object.freeze({ + machines: Object.freeze([]), + devices: Object.freeze([]) +}); +/** @type {Map>} */ +const fileWriteOperations = new Map(); +/** @type {Map>} */ +const initializationFlights = new Map(); + +class InitializationSupersededError extends Error {} /** * @returns {string} @@ -86,20 +99,57 @@ async function readUpdatedAt(dataFilePath) { } /** - * @returns {Promise} + * @returns {Promise} */ -export async function readNetworkFile() { +export async function readNetworkFileState() { const source = resolveDataFilePath(); - const raw = await readFile(source, 'utf8'); + let raw; + try { + raw = await readFile(source, 'utf8'); + } catch (error) { + if (formatFileSystemErrorCode(error) === 'ENOENT') { + return { + status: 'missing', + source, + updatedAt: null + }; + } + throw error; + } + + if (raw.trim().length === 0) { + return { + status: 'blank', + source, + updatedAt: await readUpdatedAt(source) + }; + } + const parsed = JSON.parse(raw); const updatedAt = await readUpdatedAt(source); return { + status: 'ready', data: parsed, source, updatedAt }; } +/** + * @returns {Promise} + */ +export async function readNetworkFile() { + const state = await readNetworkFileState(); + if (state.status !== 'ready') { + throw new Error(`Network data file "${state.source}" is ${state.status}.`); + } + return { + data: state.data, + source: state.source, + updatedAt: state.updatedAt + }; +} + /** * @param {string} dataFilePath * @param {string} backupDirectory @@ -149,14 +199,18 @@ export async function trimBackups(backupDirectory, baseName, keep = 5) { /** * @param {NetworkData} data + * @param {string} source + * @param {WriteNetworkFileOptions} [options] * @returns {Promise} */ -export async function writeNetworkFile(data) { - const source = resolveDataFilePath(); +async function writeNetworkFileUnlocked(data, source, options) { + if (!isWriteEnabled()) { + throw new Error('Writes disabled by NETWORK_READ_ONLY=true.'); + } + const backupDirectory = resolveBackupDirectory(); const directory = path.dirname(source); await mkdir(directory, { recursive: true }); - await createBackupIfPresent(source, backupDirectory); const temporaryPath = `${source}.tmp-${Date.now()}-${process.pid}`; const jsonPayload = `${JSON.stringify(data, null, '\t')}\n`; @@ -168,10 +222,14 @@ export async function writeNetworkFile(data) { await fileHandle.close(); } try { + await options?.beforeCommit?.(); + if (options?.createBackup ?? true) { + await createBackupIfPresent(source, backupDirectory); + } await rename(temporaryPath, source); - } catch (renameError) { + } catch (error) { await unlink(temporaryPath).catch(() => undefined); - throw renameError; + throw error; } await trimBackups(backupDirectory, path.basename(source), 5); @@ -183,6 +241,36 @@ export async function writeNetworkFile(data) { }; } +/** + * @template T + * @param {string} source + * @param {() => Promise} operation + * @returns {Promise} + */ +async function withFileWriteLock(source, operation) { + const previousOperation = fileWriteOperations.get(source) ?? Promise.resolve(); + const currentOperation = previousOperation.catch(() => undefined).then(operation); + fileWriteOperations.set(source, currentOperation); + + try { + return await currentOperation; + } finally { + if (fileWriteOperations.get(source) === currentOperation) { + fileWriteOperations.delete(source); + } + } +} + +/** + * @param {NetworkData} data + * @param {WriteNetworkFileOptions} [options] + * @returns {Promise} + */ +export function writeNetworkFile(data, options) { + const source = resolveDataFilePath(); + return withFileWriteLock(source, () => writeNetworkFileUnlocked(data, source, options)); +} + /** * @returns {Promise} */ @@ -244,3 +332,87 @@ export async function getWritableState() { export async function checkWritableState() { return (await getWritableState()).writable; } + +/** + * Reads configured network data, creating a valid blank dataset only when the + * file is missing/blank and the deployment has confirmed writable storage. + * + * @returns {Promise} + */ +async function initializeNetworkFile() { + const state = await readNetworkFileState(); + if (state.status === 'ready') { + return { + data: state.data, + source: state.source, + updatedAt: state.updatedAt, + initialized: false + }; + } + + const writableState = await getWritableState(); + if (!writableState.writable) { + const suffix = writableState.reason ? ` ${writableState.reason}` : ''; + throw new Error(`Network data file "${state.source}" is ${state.status}.${suffix}`); + } + + const data = { + machines: [...EMPTY_NETWORK_DATA.machines], + devices: [...EMPTY_NETWORK_DATA.devices] + }; + let metadata; + try { + metadata = await withFileWriteLock(state.source, () => + writeNetworkFileUnlocked(data, state.source, { + createBackup: false, + beforeCommit: async () => { + const currentState = await readNetworkFileState(); + if (currentState.status === 'ready') { + throw new InitializationSupersededError(); + } + } + }) + ); + } catch (error) { + if (!(error instanceof InitializationSupersededError)) { + throw error; + } + const currentState = await readNetworkFileState(); + if (currentState.status !== 'ready') { + throw new Error(`Network data file "${currentState.source}" is ${currentState.status}.`); + } + return { + data: currentState.data, + source: currentState.source, + updatedAt: currentState.updatedAt, + initialized: false + }; + } + return { + data, + source: metadata.source, + updatedAt: metadata.updatedAt, + initialized: true + }; +} + +/** + * @returns {Promise} + */ +export function readOrInitializeNetworkFile() { + const source = resolveDataFilePath(); + const existingFlight = initializationFlights.get(source); + if (existingFlight) { + return existingFlight; + } + + const flight = initializeNetworkFile(); + initializationFlights.set(source, flight); + const clearFlight = () => { + if (initializationFlights.get(source) === flight) { + initializationFlights.delete(source); + } + }; + flight.then(clearFlight, clearFlight); + return flight; +} diff --git a/src/routes/api/network-data/+server.ts b/src/routes/api/network-data/+server.ts index 2a487dc..7a295fc 100644 --- a/src/routes/api/network-data/+server.ts +++ b/src/routes/api/network-data/+server.ts @@ -3,11 +3,14 @@ import { json } from '@sveltejs/kit'; import { validateNetworkData } from '$lib/data/networkSchema'; import { getWritableState, - readNetworkFile, + readOrInitializeNetworkFile, writeNetworkFile } from '$lib/server/networkPersistence'; import type { RequestHandler } from './$types'; +const staticDemoMode = import.meta.env.OND_STATIC_DEMO; +const staticDemoReason = 'Static demo deployment; changes are not persisted.'; + function serializeValidationErrors(errors: Array<{ path: string; message: string }>) { return errors.map((issue) => ({ path: issue.path, @@ -22,10 +25,35 @@ function resolveReadOnlyErrorMessage(reason: string | null): string { return `Write API unavailable: ${reason}`; } +async function resolveFailedReadPayload(error: unknown) { + const writableState = await getWritableState().catch(() => ({ + writable: false, + reason: null + })); + const message = + error instanceof Error && error.message ? error.message : 'Failed to read network data file'; + return { + error: message, + writable: false, + writableReason: writableState.reason + }; +} + export const GET: RequestHandler = async () => { + if (staticDemoMode) { + return json( + { + error: staticDemoReason, + writable: false, + writableReason: staticDemoReason + }, + { status: 503 } + ); + } + try { const [{ data, source, updatedAt }, writableState] = await Promise.all([ - readNetworkFile(), + readOrInitializeNetworkFile(), getWritableState() ]); const validation = validateNetworkData(data); @@ -33,7 +61,9 @@ export const GET: RequestHandler = async () => { return json( { error: 'Stored network data is invalid', - details: serializeValidationErrors(validation.errors) + details: serializeValidationErrors(validation.errors), + writable: false, + writableReason: writableState.reason }, { status: 500 } ); @@ -47,13 +77,22 @@ export const GET: RequestHandler = async () => { updatedAt }); } catch (error) { - const message = - error instanceof Error && error.message ? error.message : 'Failed to read network data file'; - return json({ error: message }, { status: 500 }); + return json(await resolveFailedReadPayload(error), { status: 500 }); } }; export const PUT: RequestHandler = async ({ request }) => { + if (staticDemoMode) { + return json( + { + error: resolveReadOnlyErrorMessage(staticDemoReason), + writable: false, + writableReason: staticDemoReason + }, + { status: 403 } + ); + } + const writableStateBeforeWrite = await getWritableState(); if (!writableStateBeforeWrite.writable) { return json( diff --git a/tests/networkPersistenceCore.test.mjs b/tests/networkPersistenceCore.test.mjs new file mode 100644 index 0000000..bf64268 --- /dev/null +++ b/tests/networkPersistenceCore.test.mjs @@ -0,0 +1,180 @@ +import assert from 'node:assert/strict'; +import { chmod, mkdir, mkdtemp, readFile, readdir, rm, stat, writeFile } from 'node:fs/promises'; +import os from 'node:os'; +import path from 'node:path'; +import test from 'node:test'; + +let moduleSequence = 0; + +function restoreEnvironment(name, value) { + if (value === undefined) { + delete process.env[name]; + return; + } + process.env[name] = value; +} + +async function withPersistenceEnvironment(run, options = {}) { + const root = await mkdtemp(path.join(os.tmpdir(), 'ond-persistence-')); + const source = path.join(root, 'data', 'network.json'); + const backupDirectory = path.join(root, 'backups'); + const previousDataFile = process.env.NETWORK_DATA_FILE; + const previousBackupDirectory = process.env.NETWORK_BACKUP_DIR; + const previousReadOnly = process.env.NETWORK_READ_ONLY; + + process.env.NETWORK_DATA_FILE = source; + process.env.NETWORK_BACKUP_DIR = backupDirectory; + process.env.NETWORK_READ_ONLY = options.readOnly ? 'true' : 'false'; + moduleSequence += 1; + + try { + const persistence = await import( + `../src/lib/shared/networkPersistenceCore.mjs?test=${moduleSequence}` + ); + await run({ persistence, root, source, backupDirectory }); + } finally { + restoreEnvironment('NETWORK_DATA_FILE', previousDataFile); + restoreEnvironment('NETWORK_BACKUP_DIR', previousBackupDirectory); + restoreEnvironment('NETWORK_READ_ONLY', previousReadOnly); + await rm(root, { recursive: true, force: true }); + } +} + +test('initializes a missing writable data file with a blank network', async () => { + await withPersistenceEnvironment(async ({ persistence, source, backupDirectory }) => { + const result = await persistence.readOrInitializeNetworkFile(); + + assert.equal(result.initialized, true); + assert.deepEqual(result.data, { machines: [], devices: [] }); + assert.equal(result.source, source); + assert.deepEqual(JSON.parse(await readFile(source, 'utf8')), { + machines: [], + devices: [] + }); + assert.deepEqual(await readdir(backupDirectory), []); + }); +}); + +async function assertInitializesBlankFile(contents) { + await withPersistenceEnvironment(async ({ persistence, source }) => { + await persistence.writeNetworkFile({ machines: [], devices: [] }, { createBackup: false }); + await writeFile(source, contents, 'utf8'); + + const result = await persistence.readOrInitializeNetworkFile(); + + assert.equal(result.initialized, true); + assert.deepEqual(JSON.parse(await readFile(source, 'utf8')), { + machines: [], + devices: [] + }); + }); +} + +test('initializes zero-byte and whitespace-only writable data files', async () => { + await assertInitializesBlankFile(''); + await assertInitializesBlankFile(' \n\t'); +}); + +test('does not rewrite an existing valid empty network', async () => { + await withPersistenceEnvironment(async ({ persistence, source }) => { + await persistence.writeNetworkFile({ machines: [], devices: [] }, { createBackup: false }); + const before = await stat(source); + + const result = await persistence.readOrInitializeNetworkFile(); + const after = await stat(source); + + assert.equal(result.initialized, false); + assert.deepEqual(result.data, { machines: [], devices: [] }); + assert.equal(after.mtimeMs, before.mtimeMs); + }); +}); + +test('does not overwrite real data written while a missing file is being initialized', async () => { + await withPersistenceEnvironment(async ({ persistence, source }) => { + const realData = { + machines: [{ machineName: 'Router' }], + devices: [] + }; + + const bootstrap = persistence.readOrInitializeNetworkFile(); + const write = persistence.writeNetworkFile(realData, { createBackup: false }); + const [bootstrapResult] = await Promise.all([bootstrap, write]); + + assert.equal(bootstrapResult.initialized, false); + assert.deepEqual(bootstrapResult.data, realData); + assert.deepEqual(JSON.parse(await readFile(source, 'utf8')), realData); + }); +}); + +test('does not replace malformed or schema-invalid non-empty JSON', async () => { + await withPersistenceEnvironment(async ({ persistence, source }) => { + await persistence.writeNetworkFile({ machines: [], devices: [] }, { createBackup: false }); + await writeFile(source, '{not json', 'utf8'); + + await assert.rejects(persistence.readOrInitializeNetworkFile(), SyntaxError); + assert.equal(await readFile(source, 'utf8'), '{not json'); + }); + + await withPersistenceEnvironment(async ({ persistence, source }) => { + await persistence.writeNetworkFile({ machines: [], devices: [] }, { createBackup: false }); + await writeFile(source, '{}\n', 'utf8'); + + const result = await persistence.readOrInitializeNetworkFile(); + + assert.equal(result.initialized, false); + assert.deepEqual(result.data, {}); + assert.equal(await readFile(source, 'utf8'), '{}\n'); + }); +}); + +test('never initializes missing or blank files in explicit read-only mode', async () => { + await withPersistenceEnvironment( + async ({ persistence, source }) => { + await assert.rejects( + persistence.readOrInitializeNetworkFile(), + /Writes disabled by NETWORK_READ_ONLY=true/ + ); + await assert.rejects(stat(source), { code: 'ENOENT' }); + }, + { readOnly: true } + ); + + await withPersistenceEnvironment( + async ({ persistence, source }) => { + await mkdir(path.dirname(source), { recursive: true }); + await writeFile(source, ' \n', 'utf8'); + + await assert.rejects( + persistence.readOrInitializeNetworkFile(), + /Writes disabled by NETWORK_READ_ONLY=true/ + ); + assert.equal(await readFile(source, 'utf8'), ' \n'); + }, + { readOnly: true } + ); +}); + +test( + 'unwritable blank files remain untouched and report the permission failure', + { skip: process.platform === 'win32' }, + async () => { + await withPersistenceEnvironment(async ({ persistence, source }) => { + await persistence.writeNetworkFile({ machines: [], devices: [] }, { createBackup: false }); + await writeFile(source, '', 'utf8'); + await chmod(source, 0o400); + + try { + const writableState = await persistence.getWritableState(); + assert.equal(writableState.writable, false); + assert.match(writableState.reason, /not readable and writable/); + await assert.rejects( + persistence.readOrInitializeNetworkFile(), + /not readable and writable/ + ); + assert.equal(await readFile(source, 'utf8'), ''); + } finally { + await chmod(source, 0o600); + } + }); + } +); diff --git a/vite.config.ts b/vite.config.ts index 2d35c4f..d967383 100644 --- a/vite.config.ts +++ b/vite.config.ts @@ -2,6 +2,11 @@ import tailwindcss from '@tailwindcss/vite'; import { sveltekit } from '@sveltejs/kit/vite'; import { defineConfig } from 'vite'; +const staticDemoMode = process.env.OND_STATIC_DEMO === 'true'; + export default defineConfig({ + define: { + 'import.meta.env.OND_STATIC_DEMO': JSON.stringify(staticDemoMode) + }, plugins: [tailwindcss(), sveltekit()] });