MM-68722 - Set higher statistics target on posts.rootid and posts.channelid (#36506)

* MM-68722 - Set higher statistics target on posts.rootid and posts.channelid

  The default PostgreSQL column statistics target (100) is too coarse for
  posts.rootid: most rows have an empty rootid and the remainder is a long
  tail of distinct thread IDs, so the planner cannot accurately estimate
  selectivity for a specific non-empty rootid. As a result, queries like
  the per-thread MAX(createat) issued from updateThreadsFromPosts get
  planned to scan idx_posts_create_at backward and filter by rootid,
  scanning tens of millions of rows per call instead of using the
  idx_posts_root_id_delete_at index. This was observed during a large DM
  import where 16 workers were each stuck on the same query for 20 to 70
  seconds, dropping import throughput from a few thousand posts per minute
  to ~155 per minute.

  Raising the statistics target to 5000 on rootid (and on channelid for
  the same reason) lets ANALYZE record enough most-common-values that the
  planner correctly recognizes specific non-empty values as highly
  selective and picks the right index. Verified locally: post-fix the
  query runs in 0.058 ms with 11 buffer hits, vs 20725 ms with 25.8M
  buffer hits before.

  This is a metadata-only change. ALTER TABLE ... SET STATISTICS takes a
  SHARE UPDATE EXCLUSIVE lock and does not block concurrent DML.

* MM-68722 - Run ANALYZE in migration so new statistics target takes effect immediately

  ALTER TABLE ... SET STATISTICS only changes the target; the planner keeps
  using the existing sample in pg_statistic until ANALYZE runs. On a busy
  Posts table autoanalyze fires after ~10 percent of rows change, and on a
  quiet site it may not fire at all before the operator runs the import that
  motivates this change. Running ANALYZE in the migration ensures the new
  sample is collected at deploy time rather than at an indeterminate later
  point.

  Scoped to (rootid, channelid) since those are the only columns whose
  statistics target changed. Both are gathered in a single table scan.
  ANALYZE takes SHARE UPDATE EXCLUSIVE and does not block DML. No
  corresponding change in the down migration: resetting the target to -1
  leaves the existing samples valid for continued planner use, so no
  rollback-time ANALYZE is needed.
This commit is contained in:
Doug Lauder
2026-05-12 08:45:11 -04:00
committed by GitHub
parent 69f30c21e9
commit 0530d60e4a
3 changed files with 7 additions and 0 deletions
@@ -343,3 +343,5 @@ channels/db/migrations/postgres/000172_add_recaps_viewed_at.down.sql
channels/db/migrations/postgres/000172_add_recaps_viewed_at.up.sql
channels/db/migrations/postgres/000173_create_recaps_user_id_viewed_at_index.down.sql
channels/db/migrations/postgres/000173_create_recaps_user_id_viewed_at_index.up.sql
channels/db/migrations/postgres/000174_set_posts_statistics_targets.down.sql
channels/db/migrations/postgres/000174_set_posts_statistics_targets.up.sql
@@ -0,0 +1,2 @@
ALTER TABLE posts ALTER COLUMN rootid SET STATISTICS -1;
ALTER TABLE posts ALTER COLUMN channelid SET STATISTICS -1;
@@ -0,0 +1,3 @@
ALTER TABLE posts ALTER COLUMN rootid SET STATISTICS 5000;
ALTER TABLE posts ALTER COLUMN channelid SET STATISTICS 5000;
ANALYZE posts (rootid, channelid);