This page describes Darling v3.13.0. The source of this page is on GitHub at that version.
Audience: operators.
This page covers PostgreSQL servers as monitored targets: how to connect, the permissions, what gets collected and what is not done yet. It moved here from the Darling operator guide. For a first target step by step, read the PostgreSQL first-target runbook.
Darling monitors PostgreSQL alongside SQL Server. For the ordered procedure with a proof point at each step, see the first-target runbook; this section is the reference for what each piece does.
Add "engine": "postgres" to a servers entry and that target is collected by the PostgreSQL collectors instead of the T-SQL ones:
{
"name": "orders-prod",
"engine": "postgres",
"host": "orders-prod.cluster-abc123.us-east-1.rds.amazonaws.com",
"auth": "sql",
"username": "darling_monitor",
"encryptedPassword": "<DPAPI blob from --encrypt-password>"
}
Which path registers a target depends on the store, not the file. darling.json seeds config.config_monitored_servers once, when it is empty; after that the registry is authoritative and a darling.json edit adds nothing. So a fresh install declares its PostgreSQL targets in the file, and an existing one adds them with the add_servers tool (or the Viewer’s Add Server dialog), which takes effect within one collection sweep without a restart. The registry carries engine and port per row, so a target keeps its engine across restarts and reloads.
auth must be "sql" — PostgreSQL has no integrated-authentication path here, and an entry asking for it fails --test-connection rather than waiting to fail at first connect. add_servers enforces the same rule, and unlike the file parser it REFUSES an unrecognized engine rather than resolving it to SQL Server: the file’s leniency keeps one bad line from stopping the whole fleet at startup, while onboarding is a single deliberate act where a silent fallback would surface as a connection failure against the wrong port. Password handling is identical to a SQL Server entry: --encrypt-password produces the DPAPI blob, and the env:NAME / file:/path references work the same way. TLS defaults to full certificate verification (SslMode=VerifyFull); trustServerCertificate relaxes it to Require, which is the setting Aurora usually needs since it presents an RDS CA a stock trust store does not know, and "encryptMode": "Optional" relaxes it further to Prefer.
One store, both engines. The PostgreSQL collectors write to the same store as the SQL Server ones, into their own tables, on the same naive-UTC contract and the same server_id identity. Nothing is partitioned by engine — a mixed fleet is one store, one viewer, one MCP endpoint.
A collector table must never be named after a pg_catalog object. pg_catalog is searched implicitly and first, ahead of every entry in search_path, so an unqualified reference to such a name resolves to the system object no matter what the store holds. It fails loudly in one place — CREATE INDEX on a view is 42809, which aborts the migration and leaves the store unusable — and silently everywhere else: a reader’s FROM <name> would return the monitoring store’s own system view instead of collected history, so the tool reports nothing and any alert behind it never fires. That is why the slot collector stores into pg_replication_slot_stats while still being named pg_replication_slots after the view it reads (the same split as query_store → query_store_stats). A live-store test asserts no collector table shadows a catalog object, against the real catalog rather than a hardcoded reserved list.
A collector never runs against the wrong engine. Every definition declares its TargetEngine, and both SKUs check it before dispatch, so a PostgreSQL target is never sent T-SQL and a SQL Server target never sees pg_stat_statements. A store monitoring only SQL Server still carries the 29 PostgreSQL tables, empty; nothing else about it changes.
Connect to the server, not to a pooler
A PostgreSQL target must be a direct connection to the server. When the service signs in, it sets search_path to pg_catalog, pg_temp for its own session. That way every name in its queries means the built-in object. It sends this setting in the startup options parameter, and connection poolers do not pass it on.
PgBouncer refuses the sign-in with “unsupported startup parameter: options”. If options is listed in its ignore_startup_parameters, PgBouncer drops the setting instead. RDS Proxy accepts only a short list of startup parameters, and options is not on it. In each case the target fails to connect with a precondition message, and nothing is collected from it. Point the target at the server’s own host and port. On AWS, use the instance or cluster endpoint, not the RDS Proxy endpoint.
Permissions on a PostgreSQL target
pg_monitor covers most collectors. One further grant and one extension decide whether three of them return anything, and neither absence reports as a permission error:
CREATE ROLE darling_monitor WITH LOGIN PASSWORD '<password>';
GRANT pg_monitor TO darling_monitor;
GRANT pg_read_all_data TO darling_monitor; -- PostgreSQL 14+, see below
pg_monitor is the standard PostgreSQL monitoring role — it bundles pg_read_all_stats, pg_read_all_settings, and pg_stat_scan_tables. Without it the statistics views still return rows, but only for the connecting user’s own backends, which silently turns fleet monitoring into self-monitoring. On Amazon Aurora and RDS the same grant works: GRANT pg_monitor TO darling_monitor; as an rds_superuser. No superuser is needed, and nothing is created on the monitored server — unlike a SQL Server target, there are no Extended Events sessions to provision and no server setting to bootstrap.
pg_read_all_data is what makes the statistics-based reads work, and without it they succeed and store nothing. pg_stats filters every row through has_column_privilege, and pg_monitor confers no SELECT on user tables — so a pg_monitor-only role reads pg_stats as EMPTY rather than as denied, which is why nothing logs a PERMISSIONS skip. THREE collectors change behaviour: pg_column_stats stores zero rows on every run, pg_table_bloat_stats still collects sizes and dead-tuple counts but suppresses the bloat ESTIMATE that surface exists for (estimate_unavailable), and since #3234 pg_index_bloat does the same — it records every index at its size with a stated reason and no estimate. That last one reverses what this paragraph used to say, and the reversal is the point. Index bloat was measured-only at every grant level, so the grant genuinely did not affect it; #2561 had considered a statistics-based estimator and rejected it precisely because pg_stats is invisible to pg_monitor. #3234 wrote that estimator and made it the scheduled collector, which moves index bloat onto exactly the dependency the earlier text ruled out. The exact function still works without this grant, which is what the per-row exact_measurement_command is for. On Amazon Aurora and RDS the same grant works as an rds_superuser. The role is PostgreSQL 14+. On 13 and older it does not exist, so grant per schema instead — GRANT USAGE ON SCHEMA public TO darling_monitor;, GRANT SELECT ON ALL TABLES IN SCHEMA public TO darling_monitor;, and ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO darling_monitor; so tables created later are covered too. docs/postgres-first-target-runbook.md carries the same fallback.
Self-hosted targets add one more pair, for the three collectors that read the server log. pg_deadlocks, pg_plan_capture and pg_log_events read it with pg_read_file(), which pg_monitor does not cover: the login needs the pg_read_server_files role AND an explicit GRANT EXECUTE ON FUNCTION pg_read_file(text), pg_read_file(text, bigint, bigint), pg_read_file(text, bigint, bigint, boolean) — measured on #2566, the role alone does not carry it, because the function’s ACL is postgres=X/postgres. The EXECUTE half lives in each database’s own catalog, so run both grants in the database Darling connects to; issued in a different database on the same cluster they change nothing and the failure looks identical (#3239). Unlike the pg_stats gap above this one fails LOUDLY — every cycle records a PERMISSIONS skip whose message names this exact pair and the database to run it in. Grant pg_read_binary_file(text, bigint, bigint) in the same statement (#4046): pg_read_file() fails the WHOLE 4 MB tail read when one byte in it is not valid UTF-8. A failed login can plant such a byte with nothing more than a bad role or database name. pg_read_binary_file() has no such check. Once it is granted, all three collectors switch to it on their own, with no restart. The switch happens on the next cycle after a read that failed on such a byte, and otherwise within an hour. The bytes are then decoded in the connected database’s own server_encoding — UTF8, SQL_ASCII, every LATIN and ISO_8859 variant, WIN866/WIN874/WIN1250–1258, KOI8R/KOI8U, and EUC_JP/EUC_CN/EUC_KR — so a database’s own non-ASCII text comes back as itself rather than as replacement characters. Only EUC_TW, EUC_JIS_2004, LATIN6, LATIN8, LATIN10, MULE_INTERNAL, and any encoding this runtime cannot resolve stay on pg_read_file(); the grant does not help on those. One server log file mixes lines from backends in different databases, each in its own encoding — decoding with the connecting database’s encoding can still mis-decode (never fail) a line from a database with a different encoding, exactly as pg_read_file()‘s text route already can. On Aurora/RDS none of these three grants applies or is grantable: there is no filesystem, and all three collectors pull the log through the RDS API instead (#2538), authorized by the monitoring host’s IAM role rather than any database grant. docs/postgres-first-target-runbook.md step 1 carries the sequence and its proof point.
pgstattuple is no longer required by any scheduled collector, and is the ON-REQUEST route to an exact index measurement. Since #3234 pg_index_bloat estimates from catalog statistics and reads no index page, so a database without the extension collects normally rather than failing for that database — which it previously did, observed on the postgres maintenance database of the first production target. Every row still reports whether the extension is present, because that decides whether the exact_measurement_command it carries needs a CREATE EXTENSION pgstattuple; in front of it. That command calls pgstatindex, which reads every page of the index and is the analogue of sys.dm_db_index_physical_stats in DETAILED mode — run it deliberately on the one index you are about to act on rather than on a schedule. It is also the ONLY route for a PARTIAL or a DEDUPLICATED index, neither of which any statistics model can reach: PostgreSQL 13+ stores duplicate keys once in a posting list, so real storage is denser than per-tuple arithmetic allows and a correct model still over-predicts. The extension grants EXECUTE to pg_stat_scan_tables, which pg_monitor already carries — which is why the exact path works on a role the estimate does not.
pg_stat_statements must be present for pg_statement_stats, which means the extension in shared_preload_libraries (a restart, or a parameter-group change plus reboot on Aurora/RDS) and CREATE EXTENSION pg_stat_statements; in the database Darling connects to. The extension tracks all databases in the cluster keyed by dbid, so one installation in the connect database covers the whole instance. The collectors that need something installed, named rather than counted: pg_statement_stats (pg_stat_statements, above), pg_buffer_usage (pg_buffercache), pg_kernel_stats (pg_stat_kcache, which sits on top of pg_stat_statements), pg_predicate_stats (pg_qualstats), and pg_wait_sampling (the pg_wait_sampling module). Every other collector needs nothing installed: they read core catalogs and Aurora’s built-in functions. Needing shared_preload_libraries and a server restart (a parameter-group change plus a reboot on Aurora/RDS), and inert until then whatever else is installed: pg_stat_statements, pg_stat_kcache, pg_qualstats and pg_wait_sampling. Additionally per database, because the collectors that read them run per database and CREATE EXTENSION in one database does not create them in another: pg_qualstats. pgstattuple was on this list until #3234 and is now needed only for the on-request exact measurement, by no scheduled collector. Every one of them degrades identically when its dependency is absent — the query raises 42P01 or 42883, which classifies as ObjectMissing and, because the dependency is declared, is recorded as a non-fatal EXTENSION_MISSING skip that names the extension (#3240; never as PERMISSIONS, whose remedy is a grant) — and pg_extension_availability reports which install would light up every one of them except pg_wait_sampling. It deliberately does NOT cover pg_wait_sampling, which was on that roster and was taken off (#2564, fixed in #2584). The roster is a reportable-absence list rather than a list of what collectors depend on, and those are different sets: it carries extensions no collector reads and omits this one, which a collector does read — which is why a dependency is declared on the collector rather than read off the roster. Check pg_wait_sampling with SHOW shared_preload_libraries; instead.
What gets collected
| Collector | Source | Cadence / retention | Why it exists |
|---|---|---|---|
pg_wait_stats |
aurora_stat_system_waits() |
1 min / 30 d | Aurora only. Core PostgreSQL has no cumulative wait counters at all — pg_stat_activity.wait_event is an instantaneous sample — so there is no equivalent to sys.dm_os_wait_stats to read on a non-Aurora target |
pg_statement_stats |
pg_stat_statements; aurora_stat_statements() on Aurora |
1 min / 30 d | Per-query-shape totals, matching query_stats‘ cadence, wherever pg_stat_statements is installed — since #2625 it reads BOTH sources, chosen at query-build time. Aurora’s function adds the storage-vs-cache I/O split (storage_blks_read / orcache_blks_hit) and per-statement peak memory, which core PostgreSQL does not expose; on the vanilla path those columns are written NULL, not zero |
pg_wraparound_stats |
pg_database, pg_class |
5 min / 90 d | XID and MultiXact freeze headroom per database. The highest-consequence signal PostgreSQL has and one with no SQL Server counterpart: run out of transaction IDs and the server stops accepting writes. Freeze headroom moves in autovacuum-sized steps rather than continuously, so 5 minutes is ample and 90 days shows the age trend against the actual freeze threshold |
pg_xmin_horizon |
pg_stat_activity, pg_replication_slots, pg_stat_replication, pg_prepared_xacts |
1 min / 30 d | Why vacuum is reclaiming nothing. Four unrelated causes produce an identical symptom and need completely different fixes, so this attributes the specific holder instead of reporting the number. Per-minute because a holder is the fast-moving leading indicator — the useful answer is which session or slot appeared minutes ago |
pg_replication_slots → collect.pg_replication_slot_stats |
pg_replication_slots |
1 min / 90 d | Slot health and retained WAL. An abandoned slot retains WAL without bound by default, filling the volume and stopping the server, and it grows at whatever rate the server writes WAL — hours on a busy writer, not days |
pg_autovacuum_stats |
pg_stat_user_tables, pg_class |
60 min / 90 d | Writers only, and per database. Per-table autovacuum state. Stores each table’s own computed trigger threshold beside its dead-tuple count, honouring per-table reloptions overrides rather than only the GUCs — without the threshold a dead-tuple count is not actionable, since the same count is routine on a large table and urgent on a small one |
pg_io_stats |
pg_stat_io |
1 min / 30 d | I/O attributed to a (backend_type, object, context) triple rather than to a file — who did it, to what, and why. PostgreSQL 16+; valid on a standby. Every counter column is nullable on purpose: PostgreSQL uses NULL for “does not apply to this combination”, and on Aurora the whole write side is NULL because backends there do not write data files. The read reports whether write counters are TRACKED, so absent writes cannot be misread as zero writes |
pg_blocking → collect.pg_blocking_edges |
pg_stat_activity, pg_blocking_pids() |
1 min / 30 d | Who is blocked, by whom, and what state each side was in — stored as an edge list, one row per (blocked, blocking) pair, so the read layer can assemble chains and name the root. This is a SAMPLE, not an event log, and that is the one thing to carry away: SQL Server’s blocked-process report is written by the engine when blocking crosses a threshold, whereas PostgreSQL records nothing unless something asks, so blocking shorter than the interval is never seen. Valid on a standby, where recovery conflicts are blocking that happens nowhere else. One minute is the floor worth paying for: pg_blocking_pids() takes ShareLock on the lock manager partitions per call, so it is evaluated only for backends already waiting on a lock |
pg_database_stats |
pg_stat_database |
1 min / 30 d | Four questions off one cluster-wide view: temp-file spills (temp_files / temp_bytes), cache hit ratio (blks_hit / blks_read), a server-recorded deadlocks count, and the commit/rollback split. Every major and every target shape — the newest column selected arrived in PostgreSQL 9.2, it is core rather than Aurora, and a standby is included deliberately, because a sort on a read replica spills the way it does on a writer. Stored PER DATABASE rather than aggregated, because stats_reset is per database: an aggregate would have to pick one reset timestamp for rows that legitimately disagree, and one database’s reset would corrupt the cluster’s delta with nothing left in the data to say so. stats_reset is captured so the read can report a reset AS a reset. Since V133 the row also carries numbackends — the client backends connected to the database at the instant of the read, the one LEVEL in a row of counters and the honest numerator for “how full is max_connections“; written from that rung; read by the PG_CONNECTION_SATURATION fact (summed at one instant, the fraction’s numerator) and surfaced per database as get_pg_database_stats‘ peak_numbackends |
pg_index_usage_stats |
pg_stat_user_indexes, pg_statio_user_indexes, pg_index |
24 h / 90 d | Per-index scan counts, sizes, and the constraint/replica-identity/validity facts that decide droppability. Writers only, and per database — a replica reports its OWN scan counts rather than the writer’s, so an index that is load-bearing on the primary reads as unused there, which is the single worst answer this surface can give. last_idx_scan is PostgreSQL 16+ and is substituted with a typed NULL below it so the row shape does not change across a mixed-version fleet. Daily rather than hourly, matching index_object_stats: “has anything scanned this index” is a structural question, and an hourly sample would record the same catalog facts 24 times a day at 24x the fan-out connections. 90 days of retention because the RETENTION WINDOW IS THE EVIDENCE — an index can only be called unused for as long as we have been watching it, and 30 days cannot clear a monthly report. Indexes below 64 KB are not collected (nothing to model), except invalid ones, which are a finding at any size |
pg_table_bloat_stats |
pg_class, pg_stats, pg_stat_user_tables |
60 min / 90 d | The statistics-based per-table bloat ESTIMATE, plus measured heap/TOAST/index sizes and the dead-tuple counts. Never reads the relation, which is what makes it affordable on a cadence: measured at 44 ms for a 2,001-table database against 860 ms for pgstattuple over the same tables, and that 19x is a floor on the ratio because those tables were tiny and fully cached. Hourly to match pg_autovacuum_stats deliberately — this measures the DAMAGE whose CAUSE that one measures, and correlating them needs a common grain. Writers only, and per database. Tables below 1 MB are not collected. The estimate is suppressed rather than captioned when its inputs cannot be trusted; see V85 |
pg_session_states |
pg_stat_activity |
1 min / 30 d | Which sessions are holding a transaction open, for how long, and — the part that decides whether it matters — whether that transaction is what pins the xmin horizon. idle in transaction is NOT automatically a horizon holder, which is the whole reason this stores an age rather than a state: measured on live PostgreSQL 16.15, a READ COMMITTED transaction that only read and one whose UPDATE matched zero rows both sit idle in transaction indefinitely with backend_xmin AND backend_xid NULL, pinning nothing at all. horizon_age is the GREATER of the two ages and is -1 when the session holds neither — not 0, which would read as “holds the newest possible xid”. No raw statement text is stored: pg_stat_activity.query carries literal parameter values, so the normalised query_id and a whitelisted command keyword are stored instead. A SAMPLE, like pg_blocking — a transaction that opened and closed between two cycles is invisible, and the read carries its own capture counts so that is visible rather than assumed. Valid on a standby, deliberately: a standby holds its own transactions, and with hot_standby_feedback on its xmin propagates to the primary |
pg_server_config |
pg_settings |
60 min / 365 d | The configuration history — nothing else in the stack can reconstruct one that was never recorded. Hourly because configuration changes when a person changes it, and a year because “this got slow sometime last quarter” is asked long after the fact; get_pg_server_config_changes reads consecutive snapshots, so the change and its timing survive the person who made it. source is stored so the read can state which rows are server configuration and which are the collector’s own session |
pg_deadlocks |
the server log via pg_read_file(); the RDS log API on Aurora/RDS |
5 min / 90 d | The full deadlock reports PostgreSQL writes to its log — previously only the COUNT (pg_stat_database.deadlocks) was collected, a number that goes up. deadlock_hash is the identity: the self-hosted route re-reads an overlapping log tail deliberately, so one deadlock lands many times and every read groups on the hash. graph_text is stored whole beside the parsed columns, so a lock shape the parser does not break out still arrives readable by a person; every query in it, and victim_statement, is normalized first (#4005): literals become ?, and a query that cannot be read to its end is withheld. Reports stored before that, the deadlock alerts fired from them and the analysis findings that quoted them are rewritten in place the same way by a bounded pass each hour (#4012), so none of those three keeps a raw query or raw-graph hash for a direct SELECT to read; an alert whose report is gone keeps its key only as an HMAC under the store’s log-hash key. Not rewritten: a deadlock’s ERROR line in pg_log_events stored before #4020 keeps its unkeyed raw_line_hash until that table’s 30-day retention drops it, as #4004 ruled, since re-keying it would split its dedupe identity. Needs nothing enabled at default settings, but log_error_verbosity = terse and a non-English lc_messages both silently empty it — the readiness collector’s message_locale facet judges the latter |
pg_plan_capture |
auto_explain JSON in the server log via pg_read_file(); the RDS log API on Aurora/RDS |
60 min / 14 d | Captured execution plans, redacted before storage: query text is dropped outright (identity is query_id, whose normalised text pg_statement_stats carries) and every remaining string literal is scrubbed, with plan_hash taken of the REDACTED plan so a shape recurs to one hash whatever values it ran with. The self-hosted read is a fixed 4 MB tail — #2565 measured 772 MB of log in twenty seconds at capture-everything, so no fixed number bounds the worst case — and a server logging faster than the window covers should lower the cadence per server |
pg_log_events |
the server log via pg_read_file(); the RDS log API on Aurora/RDS |
5 min / 30 d | The log, CLASSIFIED (#3601): every line at WARNING or worse as an error event, log_connections / log_disconnections lines as connection, log_lock_waits reports as lock_wait, log_temp_files lines as temp_file with the spill’s exact bytes lifted into a column (#3602), log_autovacuum_min_duration completion reports as autovacuum with the run’s relation, duration, pages, tuples, buffers and WAL lifted into columns (#3603, V130 — get_pg_autovacuum_health shows them per table as recent_runs), and checkpoint lines recognised and stored under their family. The same 4 MB tail and the same logging_collector gate as pg_deadlocks, classified in C# by one PgLogEventClassifier both transports feed — the body crosses the wire so no family’s recogniser is spelled twice. message / detail / context are stored as PostgreSQL wrote them, with the SQL in a detail or context (a deadlock’s queries, a function’s statement) normalized, literals replaced by ?, and the statement is never stored, only fingerprinted; raw_line_hash is the identity every read dedupes on, keyed with the store’s log-hash key and never returned (#4004). A family whose log_* setting is off on the target is empty by construction; the read names the setting |
pg_plan_capture_readiness |
pg_settings (the auto_explain and logging GUCs) |
60 min / 365 d | Whether the target could capture plans at all, one row per facet in causal order — could it run auto_explain, is it loaded, is it capturing, in what format, and can a plan be attributed to a statement — because “no plans” has several unrelated causes and each carries its own remedy on the row. Hourly since every facet is a parameter-group setting; a year because “when did plan capture get turned on” is asked months later |
pg_write_stats |
pg_stat_checkpointer, pg_stat_bgwriter, pg_stat_wal |
1 min / 30 d | Checkpoints, background writing and WAL as ONE collector because they are one story: a high share of REQUESTED checkpoints means WAL filled max_wal_size before the interval elapsed. A union schema with version-conditional columns — PostgreSQL 17 moved the checkpoint counters out of pg_stat_bgwriter, so a major that lacks a column reports NULL rather than the collector splitting into per-version twins. PostgreSQL 14+ (pg_stat_wal); all three views are cluster-wide singletons, so a snapshot is one row. Each row also carries postmaster_start_time (V139, #3955): a clean restart keeps the counters and PostgreSQL counts its shutdown checkpoint as REQUESTED, and the shutdown checkpoint’s own write and sync work lands in the same counters, so the reads compare consecutive rows’ start times and treat a restart-spanning interval’s checkpoint figures as unknown, never as WAL or I/O pressure — get_pg_write_stats returns the requested count, its share, the checkpoint write and sync time and the checkpoint buffer counts as null with postmaster_restarted: true for a window that holds a restart, and PG_CHECKPOINT_PRESSURE leaves those intervals out of its requested, write, sync and buffer sums (postmaster_restart_count says how many) |
pg_lock_stats |
pg_locks |
1 min / 30 d | Lock state by type, mode and relation, granted or waiting. Does NOT duplicate pg_blocking_edges, which stores blocked/blocker PAIRS — that answers who is stuck behind whom, and cannot answer what lock, on what object, is being waited on. Aggregated by (database, locktype, mode, granted, relation), so row count is bounded by CONTENTION rather than concurrency — and it is a SAMPLE, so the cadence is the resolution |
pg_replication_stats |
pg_stat_replication |
1 min / 30 d | The CONNECTED standbys — four byte distances from pg_current_wal_lsn() and three time lags. The counterpart of the slot collector and a different question: a slot describes what the primary RETAINS and exists even when nothing is attached (the disk-filling case); this describes replicas actually attached, and one that disappears here while its slot persists is the dangerous combination. The lag columns keep growing during a stall rather than freezing, which is what makes them usable as an alerting signal |
pg_buffer_usage |
pg_buffercache |
60 min / 30 d | What is resident in the shared buffer pool, per relation — which objects the cache is SPENT on, a different question from which are read most. Joins pg_buffercache.relfilenode to pg_relation_filenode(c.oid), not to c.oid, which is the join every published example gets wrong, and scopes to the connected database because the pool is cluster-wide while pg_class is not. Hourly: the view is a scan of every buffer (6.1 ms at 512 MB, linear to ~780 ms at 64 GB), and a server without the extension records a non-fatal skip naming the one-line CREATE EXTENSION fix |
pg_extension_availability |
pg_available_extensions, pg_extension |
24 h / 365 d | Four states, not a boolean — installed, outdated, available, absent — because available is a CREATE EXTENSION away and absent is not. Absence is derived against an enumerated roster; auto_explain and pg_wait_sampling are deliberately off it, being preload-only modules that never appear in pg_available_extensions even where they are loaded and working. Per database as of V95, because installed/outdated are per-database facts |
pg_wait_sampling |
the pg_wait_sampling extension’s profile, or — without the extension — a service-side pg_stat_activity sampler (#3604) |
5 min / 30 d | Wait events attributed to the query that waited — wait analysis existed ONLY on Aurora before this, which was backwards, because self-hosted is where the extension story is richest. Two arms, one table, one instrument per target: with the extension created in the monitored database the collector reads its 10 ms in-engine profile; without it the collector polls pg_stat_activity once a second for a 30-second window each cycle and accumulates the tallies itself, so a stock first install gets a wait profile (a floor, not parity) rather than an empty chart. Not on Aurora, which has pg_wait_stats and cannot load the module. Every read of this table discloses instrument (extension_sampled / service_sampled). sample_count is a tally of observations, not a duration, and profile_period_ms travels beside it because the count is uninterpretable alone. Since V133 the sampler arm also stores sampled_ms — the milliseconds of the interval its 30-second window actually observed — because samples × period over the five-minute interval understates that arm’s wait rate by the duty cycle (~10×); the extension arm stores NULL there, having watched the whole interval. Written from that rung, read by nothing yet. Cluster-wide with no database attribution, deliberately: the profile carries none, and inventing one would be the scope error V95 removed elsewhere |
pg_kernel_stats |
pg_stat_kcache |
60 min / 30 d | The kernel’s own numbers per queryid — user/system CPU, bytes that reached the DEVICE, page faults. pg_stat_statements reports elapsed time, which cannot separate a query that was WAITING from one burning processor; these compose with it to make that split. exec_read_bytes = 0 does not mean nothing was read: a read served by the OS page cache is genuinely zero. Top-level statements only, because track = 'all' would double-count every function body on the server |
pg_predicate_stats |
pg_qualstats |
60 min / 30 d | Which columns are filtered on, how selectively, and where the planner’s estimate was wrong. pg_index_usage_stats records indexes that were USED; this records predicates that were EVALUATED, including the ones with no index behind them — the index-candidate signal, invisible to everything else here. sample_rate is stored per row and is usually not 1 (the extension defaults to 1/max_connections), so counts are never read as complete. Per database, because the OIDs its rows carry only resolve in their own database |
pg_column_stats |
pg_stats |
24 h / 365 d | The planner inputs behind a misestimate — n_distinct, null_frac, avg_width, correlation, and the frequency of the single most common value. The SHAPE of the skew, never the values: most_common_vals and histogram_bounds hold raw customer data and are deliberately not stored. Per database. Daily and kept a year because these move when ANALYZE runs and are asked about retrospectively. Needs pg_read_all_data — without it the collector succeeds and stores zero rows, the quiet failure the paragraph above exists for |
pg_index_bloat |
pg_stats, pg_class, pg_index |
24 h / 90 d | The complete btree census with no size floor, ESTIMATED from catalog statistics with no page reads since #3234, ranked by reclaimable bytes and never by percentage. A row with a skipped_reason has NO answer rather than a healthy one, and every row carries the pgstatindex call for its own index as the on-request exact route. Writers only, and per database. Daily to match pg_index_usage_stats — the two answer halves of one question: is this index earning its keep, and is it wasting space doing it |
pg_cpu_utilization |
AWS Performance Insights (os.cpuUtilization) |
5 min / 30 d | Instance-level CPU for an Aurora target. This collector has no SQL route at all — PostgreSQL exposes no instance-level CPU signal — so every row arrives through the RDS/Performance Insights API, authorized by the monitoring host’s AWS identity rather than any database grant. Five minutes rather than one because each cycle is an outbound GetResourceMetrics call per server, and each call asks for the last several 1-minute points, so a missed cycle backfills rather than gaps. Since V136 (#3691) the same row also carries the host’s memory — os.memory.total/free/cached/buffers/active as bytes and configured_memory_bytes (the minute’s Serverless v2 capacity × 2 GiB per ACU; NULL on a provisioned class) — from the same call: six more metric names, no second request. A stock PostgreSQL target has no row here at all, and that absence is what a memory consumer must read as unavailable, never as “no memory”. Read by nothing yet |
pg_database_size_stats |
pg_database, pg_database_size() |
60 min / 365 d | The per-database size SERIES (#3691, V136), the source the object-growth and disk-free questions never had: size_bytes per database and total_bytes for the instance on every row, templates included and flagged. Hourly because size moves in checkpoint- and autovacuum-sized steps; a year because “how has this grown since last year” is what a size series is for (~6,000 rows a day fleet-wide). A database this role may not size — the vendor-owned one on a managed target — is a row with size_bytes NULL, never 0 and never a failed collection, and the instance total goes NULL with it rather than becoming a partial sum. Read by nothing yet |
The two Aurora-only collectors — pg_wait_stats and pg_cpu_utilization, the Performance Insights CPU read — are gated on Aurora detection, not on configuration: the connect probe looks for aurora_version() and the gate follows what it finds. Point Darling at self-managed PostgreSQL and the other collectors run while those two sit out.
pg_autovacuum_stats additionally gates OFF on a standby (pg_is_in_recovery()), and the reason is worth knowing because it is not a permissions or availability problem. pg_stat_user_tables reads fine on a replica and reports all zeros: measured on Aurora 17.7, the same cluster, database and 15 tables, the writer reported 13,654,458 dead tuples and 150,790,506 live tuples while the reader reported 0 for every tuple counter. Those are the writer’s stats-collector numbers and they are not replicated. Ungated, a replica target would return no rows, the activity filter would read that as “nothing has pending work”, and you would get a confident report of perfect autovacuum health for a cluster 13 million dead tuples behind. For the same reason, treat an empty get_pg_replication_slots result from a replica as per-instance rather than cluster-wide — slots live on the writer.
Cadences and retention are the shared defaults, with no knobs, exactly as for SQL Server.
Supported PostgreSQL versions
Darling has three support tiers for a PostgreSQL target.
- Supported: PostgreSQL 14 to 18, the versions the PostgreSQL project still fixes, plus Amazon RDS, Aurora and Azure Database for PostgreSQL.
- Best effort: PostgreSQL 10 to 13. The collectors change their queries to fit the version, and a column that the version does not have is stored as NULL. Bugs on these versions are fixed when they are reported.
- Not supported: PostgreSQL below 10.
Oldest PostgreSQL version per collector
Each collector sends a query that works on the version of the target it connects to. A column that a version does not have is stored as NULL. A collector never sends a query that fails on the target, so an older target writes no error lines to its log for these collectors.
| Collector | Oldest version | Older versions |
|---|---|---|
pg_statement_stats |
10 | pg_stat_statements before 1.8 (PostgreSQL 12 and older) has total_time, min_time, max_time and mean_time. The collector stores them in the same columns. The WAL columns are NULL. The collector reads the extension’s own version from pg_extension, so a newer server with an older extension is handled too. |
pg_replication_slots |
10 | wal_status and safe_wal_size_bytes need 13 and are NULL below it. two_phase needs 14. |
pg_session_states |
10 | Parallel workers are left out by leader_pid from 13. Below 13 they are left out by backend_type, which names them from 11. On 10 they can not be told apart from other background workers, so they add to the session total only. |
pg_deadlocks, pg_log_events, pg_plan_capture |
10 | The server log query leaves out MATERIALIZED below 12, because older servers already materialize those CTEs. |
pg_write_stats |
10 | The WAL columns need 14 and are NULL below it. The checkpoint and background writer columns are collected on every version. |
pg_io_stats |
16 | pg_stat_io does not exist before 16. The collector does not run below 16, and get_pg_io_stats answers not_collected with the server’s version. |
pg_wait_sampling |
10 | query_id needs 14 and is NULL below it, which every read shows as not reported and never as 0 (0 means the wait belongs to no query). Rows that an earlier build stored for these targets hold 0. |
| Every other PostgreSQL collector | 10 | No version difference. pg_wait_stats and pg_cpu_utilization run on Aurora only. |
The queries for versions before 13 were run against real PostgreSQL 10 and 12 servers for #6206. Their rows were not read back at that time, and #6836 was the result: pg_statement_stats failed on the NULL WAL columns of every PostgreSQL 10 and 12 target. Since #6836, tests make each of these collectors send its old-version query to a current server and run the collector’s reader on the rows that come back. For pg_statement_stats, pg_replication_slots and pg_session_states the rows are also written to the store. pg_deadlocks, pg_log_events and pg_plan_capture are run on every log route (stderr, csvlog and jsonlog) with the keyword MATERIALIZED left out, which a current server accepts. Their reader is run on what the log holds, and their rows are not written to the store. pg_write_stats below 17 reads columns that a current server does not have, so its test uses the Aurora form of the query, which sends the same NULL WAL columns. pg_wait_sampling has no live case for the extension, because the throwaway PostgreSQL runtime does not have it. Only the service’s own sampler is run live: its query for 10 to 13 never names query_id, so a current server runs it, and its rows are written to the store and read back. PostgreSQL 11 and 13 are checked against the documentation only.
Three of the 29 are outage predictors rather than performance metrics, which is deliberate: PostgreSQL’s most damaging failures are quiet, slow, and fully predictable days ahead, and nothing in the engine raises its hand about them. They are read through the MCP tools (see the tool list), a web tab and — since #2530 — a WPF viewer tab: a PostgreSQL target gets seven inner tabs (Overview, Activity, Vacuum, Waits, I/O, Replication, Storage) in place of the nineteen SQL Server ones, chosen from collect.servers.engine_kind. The placement is derived from the collector catalog in both directions, so a new PostgreSQL collector cannot ship without a screen — or without the coverage pin’s explicit, dated exemption, which is where pg_database_size_stats sits until its consumer lanes land (#3691).
What it does not do yet
Plan capture shipped (collect.pg_plan_capture — auto_explain output read from the server log on self-hosted targets and through the AWS log API on Aurora/RDS; #2566, #2538, #2692), closing the largest gap this section used to open with. Scheduled analysis now runs for a PostgreSQL target too: the pass routes by the registry’s engine_kind inside the analysis service, measures its 24-hour history on pg_database_stats and scores the PostgreSQL-target fact vocabulary (PG_* / CONFIG_PG_* / ANOMALY_PG_*, sources pg_*), and the v1 detector families (#3542) have landed, so a PostgreSQL pass can return findings. The Darling viewer charts four PostgreSQL trends and shows deadlock graphs (#6385, #6387). The other PostgreSQL panels on both UIs are tables over the window, and there is no correlated timeline and no drill-down from a blocking root to the sessions behind it. The three Tier 0 outage predictors DO alert. Wraparound risk, a blocked vacuum horizon and replication-slot retention are evaluated on the alert cadence and delivered through the same deliverer, history and mute rules as every SQL Server alert, so they land in the same places and obey the same suppression. They ride alongside the shared engine rather than inside it, via a separate IPostgresAlertReadAdapter consulted only for PostgreSQL targets — Lite has no PostgreSQL target, and extending the shared adapter would have left it implementing three methods that can only return empty. Their thresholds are derived from the server’s own settings (wraparound grades against that cluster’s autovacuum_freeze_max_age, not a constant) and carry no knob by decision — see docs/design/postgres-alerting-design-note.md — but do not read that as “PostgreSQL alerting is not configurable”: five more alert families fire on a PostgreSQL target under the SAME names and the same alerts switches as SQL Server (Deadlocks Detected, Blocking Detected, Long-Running Query, Poison Wait, High CPU; #2711, #2719), and the two count alerts have PostgreSQL-specific thresholds — pgDeadlockCountThreshold and pgBlockingCountThreshold in the table above, deadlocks.pg_count_threshold / blocking.pg_count_threshold on get_alert_settings (#3444, V122). The first-target runbook‘s step 9 lays all eight out.
Scheduled ANALYSIS runs for a PostgreSQL target through the PostgreSQL-target engine; its finding families are the #3542 v1 build, which has landed.
Collector FAILURES are classified, though. A PostgreSQL fault is routed through the same ITargetProvider.Classify the engine seam exposes, so a persistent, operator-actionable condition records as a non-fatal skip with an explanation instead of logging ERROR every cycle — a pg_stat_statements view that was never created (SQLSTATE 42P01; since #3240 that records as EXTENSION_MISSING naming the extension, because the collector declares the dependency), a source Aurora does not implement (0A000), a feature switched off in the parameter group (55006). For the cases that stay in the store’s non-fatal PERMISSIONS bucket the message says which kind it is, because none of them is a missing grant either. Connection-level failures (the 08 class, 57P0x) still force a reconnect and reprobe; a cancelled statement (57014) deliberately does not, since dropping the connection over a slow query would turn a tuning problem into a reconnect storm. PostgreSQL raises 57014 for the target’s own statement_timeout, for a pg_cancel_backend() and for the client cancel Npgsql sends on its own deadline, so the classification covers all three; which side’s deadline fired is read from the server’s own message text, and a wording this tool does not recognise is reported as unproven rather than attributed to a machine.
The per-database fan-out itself is done — pg_autovacuum_stats is the collector that exercises it. Worth knowing what it costs: a SQL Server collector can reach another database without reconnecting (EXECUTE [db].sys.sp_executesql), while a PostgreSQL connection is bound to one database for its lifetime, so a per-database PostgreSQL collector is necessarily one connection per database per cycle. That is why its cadence is hourly and why new per-database collectors should be added deliberately rather than by default.