Files
mattermost/server/platform/services
Doug LauderandMattermost Build 540ccc599b MM-68179: Run sendLoop workers on all HA nodes (#35909)
* MM-68179: Run sendLoop workers on all HA nodes

  In HA clusters, sendLoop worker goroutines only ran on the leader node.
  When an API request to send a channel invite landed on a non-leader node,
  SendMsg enqueued the task to a local in-memory channel but no goroutine
  consumed it, silently losing the message. Fix by starting sendLoop workers
  in Start() on all nodes, independent of the leader-only ping lifecycle.

  - Separate sendLoop lifecycle (Start/Shutdown) from ping lifecycle
    (pingStart/pingStop on leader change)
  - Rename resume/pause to pingStart/pingStop for clarity
  - Change Active() to mean "service started" via atomic.Bool
  - Remove SetActive (no longer needed; tests use Start())

* address review comment

* Added idempotency guard to Start()

* Start() and Shutdown(): CompareAndSwap instead of Load/Store — eliminates races where concurrent calls could both proceed. Only the winner of the CAS executes; the loser returns nil
  immediately.

Ping test: replaced time.Sleep with assert.Never/assert.Eventually — no more brittle fixed sleeps. Uses assert.Never to verify no pings fire on non-leader, and assert.Eventually to
  verify pings stop after losing leadership (snapshot-then-compare pattern).

* make unit tests parallel capable

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2026-04-07 18:49:54 +02:00
..
2025-06-10 15:04:57 -03:00