Configuration Reference

This page describes Darling v3.13.0. The source of this page is on GitHub at that version.

Audience: operators.

This page covers the keys of darling.json that set up the store and the servers, plus peers, heartbeat and reads. It moved here from the Darling operator guide. The other blocks have their own pages: Alert and notification settings, Agents and alert sources, MCP configuration, Web dashboard configuration and Collector schedules and switches.

All sections except postgres and servers are optional — omit a section (or any key) to get the defaults listed here. Defaults deliberately mirror a fresh Lite install.

Where each setting lives: first-run seed or file only

darling.json does two jobs. Some keys are read from the file at every start. Others are only a first-run seed. The first time the service starts against a store with no settings, it copies them into the store. After that the store holds the only copy that counts, and the service replaces the file’s value with the store’s on every load. If you change a seeded key in the file and restart, nothing changes. Each seeded section is copied once, only while its own store table is empty.

Keys Where they live Where to change them after the first start
alerts (every key), analysis (enabled, intervalMinutes, notificationsEnabled, notifySeverity) First-run seed The Viewer’s Settings window, or the MCP tool update_alert_settings
smtp (including emailSendResolutions), and webhooks (the Teams, Slack, generic and PagerDuty keys, including slackSendResolutions, teamsSendResolutions and genericSendResolutions) First-run seed The Viewer’s Settings > Notifications, or the MCP tool update_notification_channels. The send-resolutions switches are also set by update_alert_settings
capturePlans, queryStoreBackfillEnabled, queryStoreTextBudgetMb (default 64), queryStoreFetchTopN (default 100), maxConcurrentSweeps (default 4) First-run seed The Viewer’s Settings window, or the MCP tool update_service_flags
planContentRetentionDays (default 21), composeStatementTimeoutSeconds (default 60), planXmlCompression (default "gzip") First-run seed Neither the Viewer nor MCP edits them. Change the column in config.config_service (plan_content_retention_days, compose_statement_timeout_seconds, plan_xml_compression). The statement timeout reaches the viewer and mcp roles on the next service start
mcp.enabled, mcp.port, web.enabled, web.port First-run seed The Viewer’s Settings window, live and with no restart. On a box with no Viewer, --enable-mcp and --enable-web
servers First-run seed, copied once while the server registry is empty. A server added to the file later is ignored The Viewer, the web dashboard’s Admin page, or the MCP tools add_servers and edit_server. The one exception is the remediation* and hypotheticalIndex* login keys, which the service syncs from the file at every start
analysis.uncorroboratedRoute Both. The store’s value wins when it is set. Otherwise the file’s value is used The store value, from the same places as the rest of analysis. Clear it to hand the choice back to the file
postgres, collectSchemaChangeEvents, rawChunkIntervalReconcileEnabled (default true), dailySummaryWarmerEnabled (default true), queryTextTrigramIndexEnabled (default false), procedureStatsPlanCycleInterval, xeRingBufferMaxEvents, queryStatsDeferredPlanFetch, procedureStatsDeferredPlanFetch, allowedAwsRoles, passwordKey, peers, frontDoor, heartbeat, health, reads, forcePlanBot, analyzer, and every other key in mcp and web (including mcp.network and web.network) File only Edit darling.json and restart the service

queryTextTrigramIndexEnabled

Off by default. When true, the service builds two indexes. The first is a trigram index on the plan-cache text (collect.query_text_dim.query_text), which finds the matching texts, so a search with no server_name can match the whole text table. The second is a sample index on collect.query_stats (query_text_digest, collection_time DESC), which finds where those texts ran on uncompressed chunks. The search does not wait for the second: once the matching texts are known, their digests are bound to the sample read (up to 1,000 of them; a pattern more statements than that contain joins the whole raw window instead), and the sample index turns that read’s scan of an uncompressed chunk into an index scan. On a seeded large store (40 servers, 3 million sample rows over four days, 300,000 texts, the older half of the window compressed the way the service runs it), a pattern that matched one text took 0.73 seconds cold through the window join, 0.33 seconds through the digest read with no sample index, and 0.11 seconds with it; a pattern that a third of the statements contain takes about 2 seconds either way and on a larger store can reach the 4 second budget and report stopped. On the largest store measured, the trigram index took 3.1 GB, which is 52% of the text table, took about 17 minutes to build and added about 0.5 ms to each row written. The sample index holds one entry per sample row that carries a digest in the uncompressed chunks (about 65 bytes a row on the seeded store), and a compressed chunk keeps no rows in it. That is why both are off until you ask for them.

  • The service builds them in the background about 20 minutes after it starts: the trigram index with CREATE INDEX CONCURRENTLY, then the sample index. On a TimescaleDB store query_stats is a hypertable, which refuses CONCURRENTLY, so the sample index is built one chunk per transaction (timescaledb.transaction_per_chunk), which locks only the chunk being built for the length of its scan. Collection keeps running and an upgrade never waits for either. Neither is part of a migration.
  • It needs the pg_trgm extension, and the service tries CREATE EXTENSION IF NOT EXISTS pg_trgm. If the store login is refused, the service stops asking. get_store_metrics then shows the state WaitingOnExtension and the statement to run as a role that can create it. Once the extension exists, the next pass builds the index.
  • A build that fails or is cut short leaves its index half built. The service drops it and tries again after 30 minutes, then 1 hour, then twice as long each time, up to 12 hours.
  • Setting the key back to false and restarting the service drops both indexes.
  • get_store_metrics has a query_text_index block with the state (Off, DropPending, Dropping, Pending, Building, Ready, Failed or WaitingOnExtension), each index’s size (size_bytes for the trigram index, sample_index_size_bytes for the sample index), and the last error. Ready means the trigram index exists and is valid. The sample index’s own state is reported beside it (sample_index_exists, sample_index_valid, sample_index_size_bytes, and the note), and the all-servers search is served with or without it.
  • The trigram index is what lets find_queries_by_text search every server at once. Leave server_name out and the tool searches the plan-cache text of all enabled servers, but only while the state is Ready. In any other state the tool refuses that search with a sentence that names this key and the state. Query Store text has no index and is still searched one server at a time. Searching one server, with or without a database, never uses the trigram index and is not slower for having it.
  • get_store_metrics shows the query_text_index block even before the first self-metrics snapshot, so the build can be watched on a new store.
  • An edit takes effect when the service restarts.

postgres

Two mutually exclusive modes — setting managed: true together with connectionString (or with either login setting below) is a validation error:

Key Default Notes
managed false true runs the bundled PostgreSQL + TimescaleDB (Windows only; see Managed Bundled PostgreSQL). The connection string is derived, never configured.
port 5641 Managed mode only: the loopback port the bundled server listens on. Deliberately uncommon so it coexists with any PostgreSQL (5432) already on the machine.
dataDirectory (null) Managed mode only: the cluster’s data directory. null means %ProgramData%\PerformanceMonitorDarling\pg.
connectAs "admin" Managed mode only: which least-privilege role the Viewer connects as — "admin" (reads everything + manages mute rules and dismisses alerts) or "viewer" (read-only; those write actions are hidden/disabled). See Security & Least-Privilege Roles. Ignored in bring-your-own mode (the connection string picks the role).
connectionString (required unless managed) Npgsql connection string for a store you provision yourself, e.g. Host=localhost;Port=5432;Username=darling;Password=...;Database=darling. You own that cluster’s settings: if it has TimescaleDB, size its background workers — managed mode does this for you, this mode does not.
webConnectionString (unset) Bring-your-own only: the web dashboard’s store login, normally the web role provision-roles.sql creates, e.g. Host=localhost;Port=5432;Username=web;Password=...;Database=darling. Same forms as connectionString (a literal, or an env:NAME / file:/path reference), resolved when the dashboard starts, so an unreadable reference keeps the dashboard down with the reason logged rather than stopping collection. Unset, the dashboard connects as the owner (connectionString) and loses the web role’s secret-column carve, narrow write grants and statement_timeout backstop; the service warns at startup. Not needed on the compose distribution’s own store, where the service provisions the role itself, and a validation error in managed mode (#3914).
mcpConnectionString (unset) The same for the MCP server: normally the mcp role provision-roles.sql creates; unset, MCP connects as the owner, with the same warning.

servers (array)

The list can be empty. A new install starts with "servers": [], and you add the first server in the Viewer.

Key Default Notes
name "" Display name; falls back to host
host (required) Server/instance to monitor
engine "sqlserver" "sqlserver" or "postgres" (postgresql / pg / aurora-postgresql also accepted). Configuration rather than something probed, because it decides which driver builds the connection string before there is a connection to ask. An omitted or unrecognized value means SQL Server, so every existing darling.json keeps its exact present behaviour — see PostgreSQL targets
port (driver default) PostgreSQL targets on a non-default port. SQL Server carries its port in the host as host,1433 instead, and that convention is left alone
database (none) Azure SQL Database only: the one database this entry monitors (also part of the server’s storage identity). PostgreSQL targets connect to the maintenance database and read cluster-wide catalogs
auth "integrated" "integrated", "sql", "serviceprincipal" or "managedidentity"
username (none) Required for "sql" (the login) and "serviceprincipal" (the application or client id). For "managedidentity", only a user-assigned identity sets it (its client id)
encryptedPassword (none) DPAPI blob from --encrypt-password (preferred). For "serviceprincipal" it holds the client secret
password (none) A literal (dev only, warned on every use) or an env:NAME / file:/path reference (#1804) — references are the supported non-Windows shape and are not warned
readOnlyIntent false Route to a readable AG secondary (ApplicationIntent=ReadOnly)
queryStoreReplica (none) SQL Server only: a readable AG secondary (host or host,port) or a listener name that Darling reads Query Store from, instead of the server itself, so the primary carries none of that read. Empty (the default) reads the primary and changes nothing. Azure SQL Database never reads a replica. The replica host gets the same login and password as the server, so the replica must accept them. Changing it counts as a change of how the server is reached, so the web dashboard, the MCP edit_server tool and the desktop viewer ask for the password again. Collection health shows the newest Query Store run’s read_source (for example replica 12, primary 2) beside the collector, only when a run has one
trustServerCertificate false true skips the check of the SQL Server’s certificate: the connection is still encrypted, but any certificate is accepted. The compose sample sets it to true so a first run reaches a server with a self-signed certificate. Keep verification on (false) for production targets
encryptMode "Mandatory" Mandatory / Strict / Optional; unknown values fail closed to Mandatory
multiSubnetFailover false
excludedDatabases [] Databases excluded from collection
xeSessionDatabases [] Azure SQL Database only (#7639): the databases that get Extended Events sessions (deadlock, blocked process, client timeout and long query completions), each a database-scoped session that Azure caps per pool. Empty means every database. When set, only these get a session, and excludedDatabases still applies and wins. A server-scoped session on a box SQL Server covers every database and ignores this. A session this install made in a database left off the list is left in place. Also set from the MCP set_server_xe_session_databases tool; stored in config.config_monitored_servers.xe_session_databases (V247)
monthlyCostUsd 0 The server’s monthly cost in USD. Every cost figure on the FinOps tab is proportional math on this number. 0 hides the cost columns and cards
alertDeliveryModeOverride (none) "Summary" or "PerEvent" for this server’s deadlock and blocking alerts. With none, the server follows alerts.deliveryMode
awsRoleArn (none) PostgreSQL targets on Amazon RDS or Aurora only: the IAM role ARN (such as arn:aws:iam::123456789012:role/darling-monitor) Darling assumes for this server’s AWS calls (the host check, CPU and host memory from Performance Insights, and the log reads behind plan capture, deadlocks and log events). With none, the process’s own AWS credentials are used. The role must be allowed: a role named in servers[] is allowed by being here, and the web, MCP, desktop viewer and --add-server can set only roles listed in allowedAwsRoles. See the runbook for the trust policy
awsExternalId (none) Optional, needs awsRoleArn: the external ID sent when the role is assumed, to match the trust policy’s sts:ExternalId condition. 2 to 1224 characters. Never returned by the web or MCP and never written to a log
remediationUsername (none) Optional second credential, for operator-initiated remediation (#2138) and the Level 3 server actions an agent can run (#5632). The credential above stays read-only forever and is never used for a write. Omit both remediation keys — the default — and a server you add has no remediation login. For a server that is already registered, leaving the keys out changes nothing: the store keeps the login it has. Both halves are required; one alone counts as unarmed and the service says so at connect. Always SQL auth when set, so there is no remediationAuth key: an integrated remediation identity would be the service account, which is the monitoring identity
remediationEncryptedPassword (none) The remediation credential’s secret: an env:NAME / file:/path reference, or a DPAPI blob from --encrypt-password. Only a reference works everywhere. A DPAPI blob still works for operator-initiated remediation (#2138), but the agent server actions (plan force, plan eviction and session kill) accept only an env: or file: reference and refuse a blob; --validate-config and the service log at start warn about a blob. There is deliberately no plaintext variant of this one — a wrong monitoring password fails a read, a wrong remediation password fails a write against a production server. Unlike the other per-server keys, both remediation keys are re-read from darling.json on every start: an entry with both keys sets or replaces that registered server’s remediation login. Anything else leaves the login alone. An entry with neither key never clears a login that was set in the store, and removing the keys does not disarm a server. To stop agent actions on a server, turn off its action switches, or turn off agent changes service-wide; the switches cover agent actions only, not operator remediation from the Viewer. To remove a server’s remediation login entirely, so neither agents nor operators can use it, first take both keys out of darling.json (an entry with both keys would set the login again on the next start), then run this as the store owner: UPDATE config_monitored_servers SET remediation_username = NULL, remediation_encrypted_password = NULL, modified_at = now() AT TIME ZONE 'UTC' WHERE name = 'alpha-sql-01';. An entry with only one key is logged and changes nothing. A server that is not in darling.json is never touched. The login is written only to the registered server that pairs with the entry by id (else by name, among rows no other entry claimed by id), and only when that server’s host, port, engine and other connection settings are the entry’s: otherwise it is skipped and the log names the server. A password that is none of an env:/file: reference, a sealed value or a DPAPI blob is not written (the log gives its shape, never the value). Services that share one store should declare the same logins, because each one copies its own darling.json when it starts. The log says when a login is set or changed
hypotheticalIndexUsername (none) PostgreSQL targets only. The login that the “Test an index” check runs under (#6170). The check never runs under the monitoring login. With no login set, the check is refused and the message says what to set up. See Testing an index on PostgreSQL
hypotheticalIndexEncryptedPassword (none) The password of that login, as an env:NAME or file:/path reference. Darling reads it on its own host when the check runs. A stored password is refused, and so is a reference to one of Darling’s own configuration files or secrets (unless this same reference is in darling.json for this server). The login needs a SCRAM-SHA-256 password: the check signs in with SCRAM-SHA-256 only, as the remediation login does

Least privilege for the remediation credential. Grant ALTER on each database you intend to force a plan in — that is what sp_query_store_force_plan / sp_query_store_unforce_plan need. ALTER SERVER STATE is needed only to evict a plan (evict_cached_plan, #5632), because that is what a targeted DBCC FREEPROCCACHE(plan_handle) requires, and it is the expensive grant. It is genuinely optional: without it an eviction is refused as evict_permission_denied, a named reason and never silent, and nothing is run. The service does not guess per platform: before it evicts (and on a dry run) it asks the server with a read-only HAS_PERMS_BY_NAME check, which answers for the credential’s effective permissions however the grant arrived. Azure SQL Database has no plan handle eviction at all, which no grant changes, and that is refused as its own reason, evict_unsupported_on_platform, rather than as a permission problem. An eviction has no undo, and it also needs that server’s evict_plan switch; no server has it on after an upgrade.

ALTER ANY CONNECTION is a strong grant, and it is needed only to kill (kill_session, #5632). It lets the remediation login kill ANY session on that server, not only the one an agent names, so grant it only on a server where you want an agent to be able to kill. Darling checks that the session id and its sign-in time still match what the agent read from get_blocking before it runs KILL, and that the session’s last request ended at the same time as when it was read (if the session ran something new since, the answer is session_changed and nothing is killed), and refuses with kill_permission_denied (a named reason, nothing run) when the login lacks the grant. A kill also needs that server’s kill_session switch, which is off after an upgrade. A person turns it on the way they turn on evict_plan: the store owner sets enabled to true for that server and the action kill_session in config.config_server_actions. There is no switch screen for it today. Agent dispatch gives the statement. Azure SQL Database (General Purpose and Hyperscale). A session there is reached only from inside its own database, so kill_session takes database_name (the database_name of the get_blocking row) and connects to that database, never to master, read-write. The database must be the server entry’s database or one Darling collected a blocking row from in the last day, and not one the entry excludes; anything else is refused before a connection opens (database_required, database_excluded, database_not_monitored). The remediation login needs a user in each database it may kill in, either a contained user or a user for its login, with exactly these grants: CREATE USER [remediation] WITH PASSWORD = '...'; GRANT VIEW DATABASE STATE TO [remediation]; GRANT KILL DATABASE CONNECTION TO [remediation];. A missing grant answers view_state_permission_denied or kill_permission_denied and names the grant. One batch checks the engine, that the connection is in the named database (wrong_database), that the database is read-write there (not_primary), the session id and its login_time (session_reused), that the session’s database_id is this database (session_in_other_database), and refuses system sessions and any session of the monitoring or remediation login (tool_login_session). On Hyperscale the kill reaches only the primary: a session on a named replica or a read-only secondary answers session_gone (not found on the primary) and nothing elsewhere is touched. A serverless database that is paused or resuming is retried on the open twice and then answers database_unavailable with nothing sent; an Azure transient error after the batch was sent answers outcome_unknown, and the batch is never sent again.

Testing an index on PostgreSQL

The server tab can test an index on a PostgreSQL server. It plans one stored statement twice, with and without an index that does not exist. The index is a hypothetical one from the hypopg extension. Nothing is built.

Planning a statement can run a function. PostgreSQL runs a function while it plans when the function is marked IMMUTABLE and its arguments are constants, and also when it is marked STABLE and its arguments are constants, because the planner calls it to estimate how many rows a condition matches. The function runs under the login that plans. So the check does not use the monitoring login, which usually holds pg_monitor or pg_read_all_stats. It uses its own login.

Set up the login the way you set up the remediation login:

  1. Create a login on the server. Give it no pg_monitor, pg_read_all_stats, pg_read_all_data or superuser role.
  2. Give it CONNECT on the database.
  3. Give it USAGE on each schema that holds a table you want to test, and on the schema hypopg is installed in.
  4. Give it SELECT on each table that the statements read. PostgreSQL checks table privileges when it starts an EXPLAIN, so a table the login cannot read stops the check with permission denied.
  5. Give it a SCRAM-SHA-256 password (password_encryption = 'scram-sha-256' when you set it), and a scram-sha-256 line in pg_hba.conf. The check signs in with SCRAM-SHA-256 only. A server that asks for a password in clear text, MD5, or no password at all is refused before the password is sent. Unless the server entry verifies the server’s certificate in full (encryption not Optional and Trust server certificate off), the check also needs SCRAM channel binding, so TLS must be on at the server (ssl = on).
  6. Set hypotheticalIndexUsername to the login name and hypotheticalIndexEncryptedPassword to an env:NAME or file:/path reference to its password.

For a server that is already in the store, there are two ways. If you set BOTH keys in darling.json for that server, Darling applies them on every start (it sets the login, or replaces it when the file’s differ), and the log says which server and the shape of the secret, never the login or the reference. Only an env: or file: reference is written, and only to the server whose connection settings match the entry. Leaving the keys out of darling.json never clears a login the server already has, and neither does a single key. Services that share one store should declare the same logins, because each one copies its own darling.json when it starts. The other way is to set them on the store row, as the store’s owner role (the web dashboard, the MCP edit_server tool and the desktop viewer cannot set either pair, by design). For example, in psql as the owner:

UPDATE config.config_monitored_servers
SET hypothetical_index_username = 'pm_index_check',
    hypothetical_index_encrypted_password = 'env:PM_INDEX_CHECK_PASSWORD',
    modified_at = (now() AT TIME ZONE 'UTC')
WHERE name = 'my-pg-server';

The service picks the change up when it next reloads its server list. Once the password column is set, the web dashboard, the MCP edit_server tool and the desktop viewer refuse to change how that server is reached (host, port, database, authentication, username, encryption), as they do for a remediation login: change it on the service host, with the same UPDATE as the owner. Removing the two columns (SET ... = NULL) lifts that, unless darling.json still has both keys for the server: the next start puts them back.

A bring-your-own store needs one step after the service upgrades the store to V203. Run tools/provision-roles.sql again, so config.edit_monitored_server and the password-rules trigger know the new password column. The service replaces both on each start when its login owns them, and warns once, naming the script, when it does not. The compose store and managed provisioning do this on their own.

A bring-your-own store needs one step after the service upgrades the store to V213 (the Query Store replica setting). Run tools/provision-roles.sql again. The service replaces the three config.edit_monitored_server functions on each start when its login owns them, but the new 18-argument one is a new function that only the script grants to the web and mcp roles, and only the script grants them the new query_store_replica column; until then a web or MCP edit of a server fails with permission denied. The compose store and managed provisioning do this on their own.

A bring-your-own store needs one step after the service upgrades the store to V214 (known patterns). Run tools/provision-roles.sql again. It creates the three functions that add, change and delete a known pattern and grants them to the admin and web roles. Until you run it, adding, changing or deleting a known pattern fails. Reading them keeps working. The compose store and managed provisioning do this on their own.

What a role that can create functions can read through this check. The check runs EXPLAIN on a stored statement. If a role can create a function in that database, and a statement that calls it with constant arguments is among the statements the check plans, that function runs while the statement is planned, as this login. So a role that can create functions in that database can read, through the check, whatever this login can SELECT. Grant the login SELECT only on the tables you want to test, and treat the right to create functions in that database accordingly. The transaction is read-only, so the function cannot write.

The login needs no write grants. The check tests SELECT statements and WITH statements only. An INSERT, UPDATE or DELETE is refused with a plain message, because planning one needs a write grant on its table.

With no login set, the check is refused. The message names the login, the grants and the two keys. Darling never falls back to the monitoring login.

Ending a PostgreSQL session

On a PostgreSQL server the remediation login ends one session with pg_terminate_backend (kill_session, #5632). The agent names the session by its backend_id and its pid. get_pg_blocking returns them as root_backend_id and root_pid, and get_pg_session_states as backend_id and pid. The backend_id holds the session’s start time in whole seconds, then its pid in seven digits. The two must agree, or Darling refuses.

Darling checks the pid and the start second in the same statement that ends the session. The statement also ends the session only if its transaction start and last state change are still the ones Darling read earlier. If the session ran something new since, the answer is session_changed and nothing is ended. A pid given to a session that started in another second is never ended. This is not a full guarantee. On Linux a pid comes back only after many others have been handed out. On Windows a pid can come back within about a second, so a new session there can have the same pid and the same start second.

If the monitoring login lacks pg_read_all_stats, the collector cannot read another role’s start time, and it stores the server’s start time in backend_id instead. Such an id names no session, so Darling refuses it as backend_id_has_no_session_start. Grant pg_read_all_stats (or pg_monitor) to the monitoring login; the ids collected after that work.

The login needs two grants: pg_signal_backend, and pg_read_all_stats (or pg_monitor, which includes it). Without pg_read_all_stats the server hides another role’s start time. Darling then cannot check the session and refuses. Darling reads what the login can do before it tries. It refuses with not_permitted and a message that names the missing grant, and it does not try the kill anyway.

Darling also refuses a session that belongs to a superuser. PostgreSQL lets only a superuser end one, and pg_signal_backend does not change that.

Darling ends client sessions only. A background process or a replication sender is refused as not_user_session, and Darling’s own connection is refused as own_session. Ending a session rolls back its open transaction. A session that is waiting for a synchronous standby has already committed on the server, so its commit stays. There is no undo.

pg_terminate_backend only sends the signal. After a kill Darling reads the session once more and says in still_there whether it was still there. When that read fails, or the caller cancels during it, still_there is null and the kill still counts as done. On SQL Server the same holds for rollback. If the reply to the kill is lost, for example because the connection drops or the server ends the kill’s own connection, the answer is outcome_unknown: the session may have ended. The agent changes list then shows the kill as “result unknown”, not failed. The SQL Server kill answers the same way. A kill also needs that server’s kill_session switch, which is off after an upgrade.

How the remediation login signs in to PostgreSQL

The remediation connection always opens the postgres database. It does not use the database that the server entry names. pg_stat_activity and pg_terminate_backend cover the whole cluster, so that database is enough. If the login cannot connect to postgres, Darling refuses with sign_in_failed and names that database. A person can run GRANT CONNECT ON DATABASE postgres TO <remediation login>.

The connection sets search_path to pg_catalog, pg_temp when it signs in. A setting made with ALTER DATABASE or ALTER ROLE does not change it. If a connection pooler drops that setting, Darling reads search_path before it runs anything, refuses with search_path_not_pinned, and names the value it found. The server entry then needs to point at the PostgreSQL server directly. Each statement that Darling runs also names its views, functions and operators with the pg_catalog schema. Darling never uses a function or an operator of the same name in another schema.

The remediation login signs in with SCRAM-SHA-256 only. If the server asks for a password in clear text or as MD5, or asks for no password, Darling refuses with sign_in_not_strong_enough. It does not send the password. If the server entry does not verify the server’s certificate in full, the sign-in also needs SCRAM channel binding. Channel binding ties the sign-in to the TLS session. The sign-in then fails when that TLS session does not end at the server that holds the login. The monitoring connection does not change.

Before a kill or a dry run can sign in, do these steps:

  1. Set password_encryption = 'scram-sha-256' on the server. It is the default from PostgreSQL 14.
  2. Set the password of the remediation login again, so that the server stores it as SCRAM.
  3. Give the remediation login a scram-sha-256 line in pg_hba.conf.
  4. Turn TLS on at the server (ssl = on). Channel binding needs TLS, and a verified certificate needs TLS too.
  5. If you do not want channel binding, verify the certificate. In the server entry, set an encryption mode other than Optional and turn off Trust server certificate.

Managed PostgreSQL

None of the three providers’ pages says whether its admin role holds pg_signal_backend. Check your own login: a dry run asks the server and ends nothing. These are the facts each page states.

  • Amazon RDS and Aurora: rds_superuser can stop connections with pg_terminate_backend. The internal rdsadmin role does the work of a superuser (AWS).
  • Azure Database for PostgreSQL flexible server: the server admin belongs to azure_pg_admin, a pseudo-superuser role. It does not belong to azuresu, the role with superuser privileges (Microsoft).
  • Google Cloud SQL: cloudsqlsuperuser holds limited superuser privileges. cloudsqladmin is a system user with superuser privileges (Google).

On any of them, a login that is not a superuser cannot end a superuser’s session. Darling refuses it as not_permitted.

What the remediation credential is used for. It is the login the Level 3 agent server actions sign in with: force_query_store_plan and unforce_query_store_plan, evict_cached_plan, kill_session, recompile_module, create_plan_guide and drop_plan_guide, and the operator-initiated remediation the Viewer offers (#2138). The monitoring credential is never used for any of them. Every agent action is a dry run unless the agent passes dry_run as false, and it runs only when all of these hold: the service switch config_service.agent_actions_enabled is on, the request is Level 3, the server’s own switch for that action is on, and the server has a remediation login whose password is an env: or file: reference. mcp.allowWrites never skips these checks, and on a route with action_approval set to required the tool stores a proposal that a person must approve before anything runs. See Level 3 server actions.

allowedAwsRoles (array, optional)

Key Default Notes
allowedAwsRoles [] The AWS roles the web, MCP, desktop viewer and --add-server may set on a server. Each entry is an IAM role ARN (matched exactly) or a 12-digit AWS account id (every role in that account in the aws partition; write aws-cn:123456789012 or aws-us-gov:123456789012 for an account in the China or GovCloud partition). The service runs a role only if it is listed here or a servers[] entry in this file names it in awsRoleArn. With no list, the web and MCP cannot set a role. Read at start: restart the service after you change it. Example: [ "arn:aws:iam::123456789012:role/darling-monitor", "123456789012" ]

capturePlans (boolean, optional)

Key Default Notes
capturePlans true Capture execution plans into query_stats.query_plan_xml and query_store_stats.query_plan_text. PostgreSQL TOAST compresses the plan text transparently (LZ4 on the managed store) and TimescaleDB chunk compression squeezes it further, so plans are cheap to keep — unlike Lite, which stores to DuckDB/Parquet and deliberately never captures them. Set false to skip plan capture (e.g. to shave storage across a very large fleet).

collectSchemaChangeEvents (boolean, optional)

Key Default Notes
collectSchemaChangeEvents true Record Object:Created / Object:Altered / Object:Deleted schema-change (DDL) events in the built-in default-trace collector. Set false on a noisy or benchmark box where a create/drop-happy workload floods the viewer’s System Events > Default Trace tab — e.g. HammerDB’s TPC-H Query 15 creates and drops a revenue view thousands of times, and the collector faithfully records every create/delete. Only the Object DDL slice is suppressed; file auto-grow/shrink, ErrorLog, and security-audit events are still collected. The shared collector’s equivalent of the full Dashboard’s @include_object_events. A file-only knob (not stored in the control plane): edit and restart.

procedureStatsPlanCycleInterval (integer, optional)

Key Default Notes
procedureStatsPlanCycleInterval 4 How many collection cycles pass between execution-plan captures for procedure_stats. 1 captures on every cycle and is byte-identical to releases before this knob existed; 4 captures on one cycle in four. Clamped to [1,60], and a nonsense value (0 or negative) degrades to 1 rather than to the default, so a bad setting never silently starts skipping plans. Why it exists: procedure_stats is normally the most expensive collector against a monitored server, and nearly all of that cost is plan XML — a controlled decomposition put 73.8% of its read loop in RENDERING the plans and 26.0% in shipping them, against 0.2% for the same query with no plan apply at all. The render happens inside sys.dm_exec_text_query_plan, a server-side TVF, so it is CPU on the monitored server rather than on the collector host; skipping a cycle omits the apply entirely rather than rendering and discarding. What you trade: plan freshness — a stored plan can be up to this many cycles old, and 4 keeps the worst case inside the collector’s own ten-minute candidate window. Runtime statistics (execution counts, CPU, duration, reads) are collected every cycle regardless, and the collector stays on its 1-minute schedule. The capture cycle is offset per server, so a fleet spreads across the interval instead of every server rendering together. Only procedure_stats is gated; query_stats, query_store, deadlocks and blocked_process_report still capture plans every cycle. A file-only knob (not stored in the control plane): edit and restart.

xeRingBufferMaxEvents (integer, optional)

Key Default Notes
xeRingBufferMaxEvents 1000 The max_events_limit of the ring buffer in the deadlock, blocked-process and long-query Extended Events sessions the service creates. A ring buffer holds at most this many events or 4 MB (max_memory, fixed at 4096 KB), whichever fills first. SQL Server’s own default is 1000, and a session at its limit evicts its oldest events before a collection reads them. A larger value holds more events between collections. Clamped to [1000,10000]: a whole number outside that range, however large up to the 64-bit limit, is replaced by the nearest bound, and the service logs one warning at start naming the key, the value given and the value used. A value with a fraction or an exponent, or past the 64-bit limit, is a darling.json error, like a value of the wrong type. A file-only knob (not stored in the control plane): edit and restart.

Memory often fills first. max_memory stays at 4096 KB whatever the setting. A deadlock event carries its whole deadlock graph XML, so with large graphs the deadlock session’s buffer fills by memory before it holds 1,000 events, and a higher limit adds little there. The setting mostly helps the blocked-process and long-query sessions. When memory was the cap, the loss note says so.

It applies when a session is created. A session keeps the limit it was created with, so a changed value reaches only sessions created afterwards. How an existing session gets the new value:

  • Long query: automatic. When a long_query_completions read finds this install’s own session running below the setting, the service recreates that session with the setting right after the read’s rows are stored: it drops the session, creates it with the setting and starts it. On Azure SQL Database this is done per database, and only in the databases the long-query sessions are created in: a session in a database left off the server’s xeSessionDatabases list is left alone. A recreate that completes is not repeated while the service runs, for that session and that setting value, so an engine that keeps a lower limit is recreated once, not on every run; a restart or a new value tries once more. A failed attempt is logged as a warning, the next collection sweep creates the session if it is missing, and the recreate is tried again an hour later. Where a registration with read-only intent finds its replica refusing the start because it does not show the new definition yet, the next collection sweeps try the start again for up to an hour while the replica catches up, then report the fault. Events that reach the session between that read and the recreate are lost. Disabling the collector and enabling it again also recreates the session with the current value. The service never drops or alters the shared deadlock and blocked-process sessions for this.
  • Deadlock and blocked-process: run --drop-xe-sessions <server-name>. The service creates a missing session when it next checks the server, which is within the hour or at its next restart. The verb drops every Darling session on the server, so it also drops the attention session and this install’s long-query session. Other installs that watch the server share the deadlock and blocked-process sessions by name, so the drop removes theirs too, and another install that creates a session first creates it at its own limit.

The deadlock and blocked-process sessions are shared by name across installs, so when two installs with different settings watch one server, whichever created the session wins. The loss note shows the session’s actual limit.

Where a loss shows. When a collection finds that events were evicted from a ring buffer before the read, or that the session dropped events before they reached its targets, collection health shows a WARNING on the run that found the loss, with a note that says how many, over what window, and what to do about it. The WARNING clears on the next clean run, because collection health reads only a collector’s newest run; the MCP window answers keep the record for the whole window they cover. The same note reaches the ring_buffer_loss key of the MCP tools get_deadlocks, get_deadlock_trend, get_blocking, get_blocking_trend, get_blocking_stats and get_long_query_completions when the window they cover lost events (an empty answer carries it inside hints, and then does not call the window an all-clear). Read it like this:

  • Evicted is the number of events that arrived and were pushed out of a full buffer before a collection read them. Dropped is a per-session number: events the session discarded from its own full buffers before they reached any target. For you they are the same loss. A larger event limit helps only with evicted events, so for dropped events the note advises collecting more often.
  • “At least N” means the previous read was truncated, so some events it held were never counted.
  • After a session restart the note says the loss cannot be counted, and it gives no number: a window whose only lossy runs were restarts has evicted_events and dropped_events null, not 0, and unknown_spans counts the restarts. The window of a note names the runs that found the loss; each loss happened in the time before its run. A restart is routine, so this alone does not raise the WARNING.
  • The note advises raising xeRingBufferMaxEvents only when the event limit was the cap and the read was not truncated. When the buffer was full at its 4 MB memory cap, or the read was already truncated, a higher limit would not help: collect more often instead.
  • When the setting is above the session’s actual limit, the note says so: the session was created before the setting changed, or by another install. It also gives the steps above: this install’s own long-query session is recreated with the setting once after its next read (see above), and a shared deadlock or blocked-process session takes the setting once it is dropped and the next ensure creates it again.
  • A value that makes collection health report a truncated ring buffer is too high. SQL Server cuts a ring buffer’s XML at about 4 MB when it is read, so events held past that cut are not read.
  • Azure SQL Database registrations of the logical server (no database, or master) read deadlocks from a telemetry file and report no ring-buffer loss for them.
  • get_blocking and get_blocking_trend with a database_name filter carry the whole session’s loss, not the filtered database’s: the buffer holds every database’s events, and the loss then has a database_filter_note saying so.

queryStatsDeferredPlanFetch (boolean, optional)

Key Default Notes
queryStatsDeferredPlanFetch true Whether query_stats fetches execution-plan XML only for the plans this collector host has not already stored. On (the default), the main query reads the statistics with no plan, and a second query on the same connection renders just the plans whose identity (plan handle, creation_time, plan_generation_num) the host has not committed; a row for a plan it has stored reuses the stored digest, so every reader sees the same rows and plans as before. Off restores the inline capture, which renders and ships every row’s plan on every run. Why it exists: on one large store the inline capture rendered 54 GB of plan XML an hour on the monitored servers and shipped 29 GB of it, against 1.1 GB an hour of distinct plans, about 47 times the plans there are. The render is CPU on the monitored server (sys.dm_exec_text_query_plan), so this is load taken off the servers you watch. Why you would turn it off: to get the plan re-rendered on every run (for example to compare against the old behavior while testing), or if a server is so constrained that the second round trip is unwelcome. What you trade by leaving it on: a plan’s text is fetched once per identity, so a plan the engine changes in place without a recompile is not re-read until it leaves the cache (entries unused for an hour are dropped; a restart empties it and each server renders its plans once more). A statement recompile between the two queries can pair the new plan with the old identity for that one run’s row; the old identity leaves sys.dm_exec_query_stats at once, so it is not read again. Azure SQL Database keeps the inline capture regardless of this setting, because its per-database collection path has no second query to defer to. Only meaningful while capturePlans is on. The run records plans_rendered, plans_rendered_bytes and plan_fetch_ms in its collection_log note. A file-only knob (not stored in the control plane): edit and restart.

procedureStatsDeferredPlanFetch (string, optional)

Key Default Notes
procedureStatsDeferredPlanFetch "on" Whether procedure_stats fetches execution-plan XML only for the module plans this collector host has not already stored. Three values; anything else is read as off and logs one warning. off keeps the inline capture and is the way to turn the deferred fetch off: every captured module plan is rendered and shipped, as before. shadow does exactly what off does, and every stored row is the same as off would store; in addition it recognizes each module plan by its identity (plan handle, cached_time, and a fingerprint of the module’s statements read from sys.dm_exec_query_stats: how many, the latest creation_time, the sum of plan_generation_num) and checks that identity against the plans this host has already stored. It writes nothing different. What it measures lands in the run’s collection_log note, beside plans_rendered: deferred_would_hit (identities it would have skipped rendering), deferred_false_hit, deferred_same_shape and deferred_miss. Of the would-hits that rendered different bytes, deferred_false_hit counts the plans whose shape changed, or whose shape could not be compared. This is the number that must stay 0. deferred_same_shape counts the plans whose bytes changed but whose shape held, for example a memory grant that SQL Server adjusted in place. on treats that as a correct hit. Each false hit logs one warning. It names the identity and says why. One reason is that the plan’s shape changed. The other reasons mean the shapes were not compared: the cached entry has no shape yet, the rendered plan is over the size cap, or its XML did not parse. The note also splits deferred_false_hit by reason into deferred_false_hit_shape, deferred_false_hit_no_shape, deferred_false_hit_over_cap and deferred_false_hit_unparsed, which add up to it, so a real shape change reads apart from a plan that was not compared (an over-cap plan is still compared by its byte size, so a memory grant change can count there). on (the default since 3.10) skips the render for a recognized plan and sends its stored digest instead; the others are rendered in one second query on the same connection, at most 150 per run (the main query is already TOP (150), so the limit is a guard, and deferred_over_cap = 0 is expected). The run records plans_rendered, plans_rendered_bytes, plan_fetch_ms, deferred_hit and deferred_miss. procedureStatsPlanCycleInterval still decides which runs may render plans: a run between captures sends the digest for a recognized plan and renders nothing. An identity is trusted for four capture cycles, then rendered again. Why a shadow mode: a module plan changes in place when one statement recompiles, with the same plan handle and cached_time; shadow measures the identity against the plans it just rendered before on is allowed to trust it. Move from shadow to on only after deferred_false_hit has stayed 0 over a representative period. A rising deferred_same_shape is not a reason to wait. Azure SQL Database stays off regardless of this setting. Only meaningful while capturePlans is on. A file-only knob (not stored in the control plane): edit and restart. The plan cache starts empty under whichever mode the service starts in.

passwordKey (string, optional)

Key Default Notes
passwordKey (unset) Where the service reads its password key from, instead of the credentials directory (#8006): a file:/path or env:NAME reference, such as file:/run/secrets/password_key. As a compose secret, that file is owned by user 1654 on the host, mode 0400 or 0600 (#8141, see secrets/README.md). A literal value is refused, and so is a file: path inside the credentials directory. Unset, nothing changes: the service makes and keeps the key in the credentials directory. The key is read where it is used, not when darling.json loads, and --validate-config prints its fingerprint or a warning. With the setting the service never makes, writes or retires a key file. See the password key for the key’s format, making one with --new-password-key, and moving an existing key. A file-only key: edit and restart.

health (object, optional)

Key Default Notes
health.enabled false Turns on the health port (#8184), for Kubernetes liveness and readiness probes.
health.port 5154 The port. It must be 1 to 65535 and different from web.port and mcp.port, whether or not those are enabled. The check compares against the web.port and mcp.port written in darling.json (or their defaults when the file omits them), not a port set later from the Viewer or the web dashboard.
health.listen (empty) The address to bind. Empty means 0.0.0.0 inside a container and 127.0.0.1 outside one. It must be an IP address.

The port has no token, no allowFrom and no TLS, and serves only /livez and /readyz, each answering one word. See The health port. Nothing is checked while health.enabled is false. A bad value is not a configuration problem: it turns off the health port only. The service logs one Critical line that names the setting, does not open the port, and keeps collecting. A file-only block: the control plane never writes it, and a change needs a restart.

peers

Declared peer stores — optional, and only relevant when the fleet is split across several Darling boxes, one store each (SQL Server primaries on one box, their readable replicas on another, PostgreSQL on a third). Each box’s MCP server answers over its own store only, so a server monitored by a sibling resolves as not-found — which an agent cannot tell apart from “nobody monitors this server.” Declaring the siblings fixes that at the three places an agent forms its picture of the fleet.

Disclosure only. There is no address and no credential in this block, and nothing behind it: the service never contacts a peer, cannot read a peer’s data, and cannot tell whether a peer is even running. A peer is a name plus a sentence, so an agent (or its human) can pick the right endpoint. Everything here is sent verbatim to every connected MCP client, so the service refuses to start if any peer text looks like a connection string or credential.

Key Default Notes
storeName "" Opt-in. A short label for THIS store itself; when set, every store self-alert fires under it instead of Monitor Store — see below, setting it re-keys the self-alert fingerprints
thisStoreCovers "" One sentence naming what THIS store monitors — the anchor the peer list is relative to
stores[].name — Required. Whatever an operator would recognize (the box name, “the use1 store”)
stores[].covers "" A short sentence naming what that store monitors. Human prose — never parsed, only shown
stores[].matches [] Optional server-name substrings that store monitors, case-insensitive. The only machine-checked field
"peers": {
  "storeName": "prod-sql-use1-monitor-01",
  "thisStoreCovers": "the 42 us-east-1 SQL Server primaries",
  "stores": [
    {
      "name": "prod-sql-use2-monitor-01",
      "covers": "the readable replicas of those same 42 primaries, in-region from us-east-2",
      "matches": ["use2"]
    },
    { "name": "prod-sql-pg-monitor-01", "covers": "the Aurora PostgreSQL clusters", "matches": ["-aurora-"] }
  ]
}

What it changes, with peers declared:

  • The MCP instructions gain a Fleet Coverage section, high enough that an agent reads which store it is talking to before it reads the tool census.
  • list_servers gains this_store_covers, a peer_fleets array, and a peer_note. Both are always present: an empty peer_fleets has two very different meanings (this really is the only store, or nobody declared the siblings) and the service cannot tell them apart, so peer_note says exactly that rather than letting an empty array read as “this is the whole fleet.” An empty registry answers in prose rather than JSON, and carries the peer list too — a store with nothing registered is a fresh or just-restarted box, which is the worst place to drop the disclosure.
  • The server-resolution miss appends the disclosure to the existing “Could not resolve server. Available servers:” listing, naming the peer whose declared coverage matches — so not monitored here stops looking like not monitored anywhere.

storeName names the store on its own self-alerts. Every fleet-level self-alert — Store Disk Pressure, Retention Held, the Collector Cost Digest, the Fleet Sweep Rollup, all twelve families and their resolution edges — fires under the constant Monitor Store, which is unambiguous with exactly one store and collides with exactly two: on a shared channel the alert cannot say which host it is about, and the delivery fingerprint downstream automation dedups on is identical on both stores, so the second store’s incident lands on the first store’s ticket as a recurrence rather than opening its own. Set storeName on each store and its self-alerts carry that label instead — into the cards, the alert history, the mute-rule matching and the fingerprint, the same way a monitored server’s display name travels. Setting it is accepting a re-key: the label is part of the fingerprint, so open work items keyed on the old one are orphaned and fresh ones start beside them. That is why it is opt-in with the constant as the default (unset is byte-identical to every earlier release), and why there is deliberately no machine-hostname fallback — a fallback would perform that re-key on every existing install at upgrade, silently. Two couplings to plan for: mute rules match the alert row’s spelling, so re-scope any self-alert mute from Monitor Store to the new label; and do not reuse a monitored server’s display name — the two would be indistinguishable on every shared surface including the triage page, and nothing polices it, because the registry is store-authoritative after seeding and config validation cannot see it.

matches is deliberately plain substrings, no globbing and no regex: it exists to answer “which region/role prefix is this name?”, and a pattern language would be a config surface with its own failure modes. Blank entries are dropped — an empty substring matches every name, which would make one peer claim the whole fleet. A peer with no matches is still disclosed everywhere; it just cannot be singled out on a miss, and the miss message says so instead of implying the server is unmonitored.

A file-only block (not seeded into the control plane): it describes the deployment topology of this box, which must not be editable from a peer’s Viewer. An edit takes effect on the next service restart. There is deliberately no cross-store connectivity here — actual federated reads (auth between stores, latency, partial failures) are a much larger surface, and may never be worth building if disclosure alone makes the split legible.

Declaring nothing changes nothing, with one exception worth knowing about on upgrade. The instructions, the resolution-miss message, and list_servers‘ empty-registry sentence are byte-for-byte what they were. But list_servers‘ JSON envelope carries this_store_covers, peer_fleets and peer_note on every response, declared or not — so a script comparing that tool’s exact shape sees three new keys even if you never write a peers block. That is deliberate: an empty peer_fleets means either “this is the only store” or “nobody declared the siblings”, and a note that only appeared when peers were declared would say nothing in precisely the case that produces the wrong conclusion.

A peers block that fails validation is refused whole, and nothing is disclosed — not the valid subset. An unfinished block that asserts coverage which may be wrong is worse than no block, and the service logs each problem at Critical. The check runs inside the publish rather than only in config validation, because the MCP host loads its own config and deliberately never validates it (its fail-closed checks are host-local), so validation alone would leave the one path that actually broadcasts uncovered.

frontDoor (optional)

Turns this service into a front door: one MCP address and one web address in front of several Darling stores. It is off unless frontDoor.enabled is true. The block is file-only, holds the access token of every store, and is never returned by a page or an MCP tool. It is not part of peers, because the peers text goes to every MCP client. In front-door mode the service has no database of its own, so postgres is not required, and no collector runs. Every field, the checks at start and the setup of each store are in The front door.

heartbeat

An optional outbound ping to a dead-man’s-switch URL, off unless url is set. File-only: the web Admin and MCP cannot edit it, and no page returns it. See Watching the service from outside.

reads (optional)

reads.maxHeavyReads (default 4) is how many long reads the MCP server and the web dashboard run at once. A read is long when the 90th percentile of its tool or page is above 2 seconds at that window size over the last 7 days (with no history yet, a 7-day or longer window counts as long). Up to 16 more long reads wait, oldest first, for up to 45 seconds; past that a read is refused: busy on MCP and HTTP 429 with Retry-After on the web. Short reads, the health reads (get_read_latency, get_slow_reads, get_collection_health, get_collection_log, get_store_metrics, get_store_log, list_servers) and MCP write tools never wait. 0 turns the limit off. The limit is per service process: the WPF viewer reads the store directly and is not counted. The value is read when each server starts.

An identical long read is answered from memory for a short time: 15 seconds for a read with no window or a window up to 1 hour, 30 seconds up to 24 hours, 60 seconds up to 7 days and 120 seconds beyond. One read is cached whatever its cost: get_resource_use_by_tag for the whole fleet (no server named) with a window of 24 hours or more, for 5 minutes, because it takes about a minute when nothing is stored and a page reload or a second user would pay that again. The plan cache, procedures and blocking sections of that answer come from 1-minute collectors, so a blocked process report or a deadlock that landed in the last 5 minutes can be missing from the fleet answer until the entry expires. The window is the length asked for, not its band, so a fleet read of 2 to 23 hours (the page’s Past 4 hours) is not held for 5 minutes. A window under 24 hours, and a read of one server, keep the times above. The 5 minutes count from when the answer is stored, not from a fixed clock boundary. A Custom View or alert notebook read panel of get_resource_use_by_tag loads through the same read as the page, so with no server and a window of 24 hours or more it is held for 5 minutes too, with the same lag. Only a composed panel (one with a source) keeps the times above, and this read is not a compose source. A read answered from memory takes no slot. The cache is per service process too.

get_read_latency (and its grid on the web) counts these per route and window size. count and the percentiles cover only the reads that ran. queued is the reads that waited for a slot, queue_wait_ms their summed wait and queue_wait_p95_ms the 95th percentile of that wait. rejected is the reads answered busy or 429, cache_hits the reads answered from memory and cancelled the reads whose caller left. wait_total_ms stays the wait for a store connection only. The admission object is the live state of this process since it started: the limit, the long reads running and waiting now, and per route the reads admitted, queued, refused, answered from memory and missed by the cache. It is not stored, so it starts again at zero after a restart.