mirror of
https://github.com/mattermost/mattermost.git
synced 2026-09-19 02:06:37 +08:00
85acba42e1f9ae8a6bfb2f724bbf819b21a45c7a
251
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
85acba42e1 |
Skip flaky TestExtractConcurrency (#37834)
Automatic Merge |
||
|
|
0b034ca9a7 |
Fix flaky TestExtractConcurrency (#37527)
Automatic Merge |
||
|
|
9de0027868 |
[MM-69982] Fix intermittent remote cluster ping failures from stale keep-alive connection reuse (#37694)
* [MM-69982] Fix intermittent remote cluster ping failures from stale keep-alive reuse The remote cluster HTTP transport set IdleConnTimeout to 90s while pings fire every PingFreq (60s) and peers close idle keep-alive connections after their own IdleTimeout (default 60s). Because 90s > 60s, the pooled ping connection outlived the peer's reaping, so each 60s ping reused a connection the peer had already closed, racing the FIN and failing intermittently with EOF / connection reset. Set IdleConnTimeout to PingFreq/2 so the pool always discards the connection before the next ping fires, making the reuse race structurally impossible while keeping the invariant (IdleConnTimeout < PingFreq) explicit and tied to PingFreq. Co-authored-by: mattermost-code <matty-code@mattermost.com> * chore: retrigger Server CI after check-style Docker Hub flake Server CI check-style failed pulling buildenv (Docker Hub Client.Timeout / missing buildenv-image artifact); unrelated to IdleConnTimeout changes. Enterprise CI check-style already passed on the same commit. Co-authored-by: mattermost-code <matty-code@mattermost.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: mattermost-code <matty-code@mattermost.com> |
||
|
|
9230eb6f6a |
MM-69725: Propagate context through Extractor.Extract using new mattermost/pdf fork (#37579)
* Propagate context through Extractor.Extract Adds ctx context.Context to the Extractor interface and all implementations so callers can cancel in-flight extractions. pdfExtractor passes the context down to GetPlainText(ctx), which checks ctx.Done() at each page boundary. When extractWithTimeout fires (deadline exceeded or parent cancel), it cancels the derived context via defer cancel(), and the PDF goroutine stops at the next page rather than running to completion, eliminating the heap growth that caused OOM on complex documents. For non-context-aware extractors (docconv, plain, archive) the context is accepted but not yet acted on; they continue to run to completion on a detached goroutine after the timeout, as before. The call site in channels/app/file.go passes rctx.Context() so request cancellation (e.g. HTTP disconnect) also propagates. * Use new mattermost/pdf fork * Update notice reflecting the new fork * Fail archiveExtractor.Extract early on ctx errors * Bump mattermost/pdf to cd8a834041c4 --------- Co-authored-by: Mattermost Build <build@mattermost.com> |
||
|
|
a8f55e5327 |
[MM-69885] Allow integrations to update mm_blocks_actions on their own posts (#37583)
* [MM-69885] Allow integrations to update mm_blocks_actions on their own posts * prune mm_blocks_actions on post updates to match remaining buttons |
||
|
|
7778bb79d1 |
MM-69792: Recover shared channel sync after a brief remote outage (#37499)
* MM-69792: Recover shared channel sync after a brief remote outage When a remote went offline for less than the 5-minute IsOnline() threshold, sync recovery never ran. IsOnline() never flipped, so no connection-state-change event fired and ForceSyncForRemote was never triggered. Posts created during the outage stayed unsynced until the next organic change in the channel. The root cause was that send failures are reported asynchronously through a callback that sendSyncMsgToRemote discarded. On a failed send syncForRemote still returned nil, so processTask saw success: the retry loop and the failure-notification hook were effectively dead code for delivery failures. Changes: - sendSyncMsgToRemote now propagates the delivery error reported via the callback instead of swallowing it. A failed send is retried rather than silently dropped. This applies to every sync type (posts, users, memberships, reactions, statuses); the success path is unchanged. - remotecluster tracks a per-remote "sync failed since last ping" flag (NotifySyncFailed). On the next successful ping, PingNow fires a connection-state-change event even when IsOnline() never flipped, driving a single ForceSyncForRemote on recovery. - Sync retries are now spaced by SyncRetryDelay (15s) instead of retrying immediately, giving a short outage time to recover and guaranteeing a failed ping before retries exhaust. Also adds a post/offline-recovery case to the sharedchannel-test integration tool (stop and restart Server B mid-run) and disables metrics on the two test servers so they do not collide on the metrics port when the loaded config has metrics enabled. * fix linter error * Fix Rabbit nit * fix CI * Delete pr37499-fix-findings-1-and-4.md * address review comments * MM-69792: Keep a per-remote retry task when a shared channel sync fans out A sync task with no specific remote (remoteID == "") fans out to every remote sharing the channel. On failure each remote's retry was a copy of the original task that still carried the original, remote-less task id. Because addTask merges on task id, all of those per-remote retries collapsed into a single entry, so only one failed remote was ever retried and the rest were silently dropped. Delivery failures were partly masked by the ping-driven ForceSyncForRemote recovery, but non-delivery errors for the other remotes had no path back. Give each per-remote retry a remote-specific id (task.id + remoteId) when the originating task had no remote, so every failed remote keeps its own retry task. Single-remote tasks already have a remote-specific id and are left untouched, since recomputing would grow the id on each retry. Also harden the tests: assert the retry is scheduled the full SyncRetryDelay out (no tolerance), assert the result callback is not invoked on delivery failures or unconfirmed responses (so the cursor stays un-advanced), and add a regression test covering fan-out to multiple failing remotes. * fix one more pre-existing edge case. * fix linter |
||
|
|
303b0c7ff2 |
[MM-69221] (#37301)
* [MM-69221] * Address coderabbit comments * Add tests to worker * Address coderabbit comment |
||
|
|
5433e6eef9 |
[MM-69689] Self-heal orphaned shared channel remotes for deleted remote clusters (#37346)
* [MM-69689] Self-heal orphaned shared channel remotes for deleted remote clusters When a SharedChannelRemote row is still live (DeleteAt = 0) but its RemoteCluster has been soft-deleted, the sync loop repeatedly picks up the orphaned row and logs a warning on every tick without ever cleaning it up. Soft-delete the orphaned SharedChannelRemote when the remote cluster lookup returns sql.ErrNoRows, so the row stops recurring in the sync loop. Logs a Warn (not Error) and skips the task. Co-authored-by: mattermost-code <matty-code@mattermost.com> * [MM-69689] Expand self-heal test coverage for orphaned shared channel remotes Add subtests covering the DeleteRemote failure branch and an end-to-end doSync run proving a self-healed orphan is drained and not retried, and assert DeleteRemote call counts. Co-authored-by: mattermost-code <matty-code@mattermost.com> * [MM-69689] Log already-deleted remote skip at debug instead of warn A soft-deleted RemoteCluster no longer syncing is the expected steady state, so skipping it does not warrant a warning. Downgrade the log for the already-soft-deleted SCR case to debug to avoid noise. --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: mattermost-code <matty-code@mattermost.com> |
||
|
|
9e3d8efc1a |
chore: remove deprecated shared channels feature flags (#37154)
* chore: remove deprecated shared channels feature flags * chore: remove sharedChannelsPluginsEnabled prop from channel header * chore: remove enableSharedChannelsPlugins prop from file_search_results and post_message_view * chore: drop hardcoded FeatureFlagEnableSharedChannelsPlugins from client config * fix: remove unused channel prop from file_search_result_item * fix: remove double blank line in file_search_result_item test * remove redundant fragment wrapper in channel header * Remove unused channelId prop from CodeBlock |
||
|
|
ebf41be704 |
Stop shared channel sync error spam for deleted remote clusters (#36931)
Filter GetUsersForUser against live remote clusters so orphaned user rows are no longer synced, and skip not-found remotes in processTask instead of retrying and logging an error. |
||
|
|
471fd8d1dd |
Bound document content extraction time and decouple it from uploads (MM-69098) (#36856)
* Bound document content extraction to prevent upload DoS (MM-69098) Extraction of uploaded Office documents ran with no time or size bound and acquired a shared worker-pool slot on the request goroutine, so a low-privilege user could keep cheap-to-upload but expensive-to-extract documents in flight and degrade file uploads for every user on the server. - Enforce a configurable per-extraction timeout (FileSettings.ExtractContentTimeout, default 10s) and honor the previously-ignored size argument in the document and PDF extractors to bound the work performed. - Run extraction on a dedicated, bounded worker pool with non-blocking submit (GoExtraction) so saturating it can never block the request goroutines that dispatch uploads; overflow is skipped and backfilled by the ExtractContent job. - Route the file and resumable-upload extraction dispatch sites through the new pool instead of the shared GoBuffered/Go pools. Co-authored-by: Julien Tant <JulienTant@users.noreply.github.com> * Use require.Fail instead of t.Fatal in extraction pool test The mattermost-govet tFatal analyzer (run by check-style) forbids t.Fatal in tests in favor of testify assertions. Co-authored-by: Julien Tant <JulienTant@users.noreply.github.com> * ci: retrigger workflows (transient container init failure) Co-authored-by: Julien Tant <JulienTant@users.noreply.github.com> * Use sync.WaitGroup.Go for extraction workers Satisfies the golangci-lint waitgroupgo/modernize analyzer flagged by check-style at goroutines.go:58. Co-authored-by: Julien Tant <JulienTant@users.noreply.github.com> * Fix file close race with detached extraction goroutine On timeout, extractWithTimeout returns while the converter may still be reading the input on a detached goroutine. ExtractContentFromFileInfo previously closed the file via defer as soon as Extract returned, racing with (and closing the file out from under) that goroutine. Transfer close ownership to docextractor via ExtractSettings.ReaderCloser: the reader is now closed only after extraction actually finishes - in the detached goroutine on the timeout path, or after Extract returns on the synchronous path. ExtractContentFromFileInfo no longer closes the file itself. Co-authored-by: Julien Tant <JulienTant@users.noreply.github.com> * Document that extraction timeout bounds wait time, not CPU Clarify in ExtractSettings.Timeout and extractWithTimeout that on timeout the docconv converter keeps running on a detached goroutine (docconv is not context-aware), so the timeout bounds how long an extraction occupies a worker slot, not the CPU it consumes. MaxFileSize is the primary bound on per-extraction work. Note load-shedding as a possible future improvement. Co-authored-by: Julien Tant <JulienTant@users.noreply.github.com> * Add tests for PDF size cap and ExtractContentTimeout validation - TestPdfMaxFileSize exercises the LimitedReaderWithError branch in pdf.go: a tight MaxFileSize errors out before extraction, while zero and generous limits extract normally. - TestFileSettingsExtractContentTimeout asserts ExtractContentTimeout=-1 fails IsValid while 0 (disabled) and 10 (default) pass. Co-authored-by: Julien Tant <JulienTant@users.noreply.github.com> * Harden extraction shutdown, panic safety, and timeout input validation - stopExtractionWorkers now drains the queue after signaling stop so a worker cannot dequeue and run buffered tasks during shutdown. - The detached extraction goroutine recovers panics and converts them to errors, so a panic in an extractor cannot crash the server. - Admin console enforces a client-side minimum of 0 for ExtractContentTimeout (validators.minValue). - Test hardening: bounded wait for the detached extractor to start, plus a test that a panic is surfaced as an error. Co-authored-by: Julien Tant <JulienTant@users.noreply.github.com> * ci: retrigger workflows (transient Docker Hub image pull timeout) Co-authored-by: Julien Tant <JulienTant@users.noreply.github.com> * Correct misleading extraction-skip logs and comments The queue-full warning logs and GoExtraction doc comments claimed skipped content would be backfilled by a 'periodic ExtractContent job'. That job is registered with a nil scheduler, so it never runs automatically -- it only runs when an admin manually triggers a content extraction job (mmctl extract). Reword the logs and comments to state that skipped files stay unsearchable until an admin runs a content extraction job, so we don't imply automatic recovery that doesn't exist. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Julien Tant <JulienTant@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
684ddb32a9 |
[MM-68988][MM-68989][MM-68990][MM-68991][MM-68997][MM-68998] Session Attributes MVF - Server-work (#36934)
* [MM-68988][MM-68989][MM-68990][MM-68991][MM-68997] Session Attributes MVF - Server-work * PR feedback * [MM-68998] Add web app hooks for Desktop App to signal a refresh of attributes/manifest * Fix types * Adjust the test to test the license first * PR feedback * Coderabbit feedback * More tests |
||
|
|
1e0bdaf068 |
MM-69057: Verify post ownership on inbound shared-channel edit/delete (#36814)
The inbound shared-channel sync handler applied edits and deletes from a remote cluster without checking that the existing post belonged to that remote, allowing a remote to modify or delete posts it did not own. Enforce the same ownership check already used for reactions and acknowledgements before editing or deleting a synced post, and add regression tests for the cross-remote cases. Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
|
03f2eaaa0b |
[MM-68400] Four plugin hooks and ChannelGuard enforcement (#36152)
* allow workflow_dispatch trigger for Server CI (for plugins CI) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * [MM-68402] MBE Phase 2: declare four generic plugin hooks (#36291) * new hooks-only phase 2 * remove ChannelWillBeMoved * remove RecapWillBeProcessed and MessageWillBeRewrittenByAI Drop the AI/recap hooks from the new-hook surface; AI-LLM paths remain uncovered in tech preview and are documented as residuals. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * [MM-68403] MBE Phase 3: ChannelGuards primitive (storage + cache + plugin API) (#36365) * phase 3 * phase 3: register ChannelGuard mock in test setup helper NewChannels' startup-time call to reloadGuardCache invokes s.ChannelGuard().GetAll(); without an expectation on the mock store, every test that sets up the server with GetMockStoreForSetupFunctions panics during init. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * phase 3: register ChannelGuard mock in retrylayer test retrylayer.New walks every store getter to wrap it; without the mock expectation on ChannelGuard, TestRetry panics during layer construction. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * use rctx properly in the store methods * phase 3: match rctx arg in testlib ChannelGuard mock GetAll now takes request.CTX, so the testify expectation must include mock.Anything; otherwise the call panics under the mocked store. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * phase 3: set api.ctx in TestChannelGuardLowercaseNormalization The test constructs PluginAPI directly without a ctx, which used to work when App.RegisterChannelGuard built its own EmptyContext. Now that the App methods take rctx from the caller, the nil ctx panics inside RequestContextWithMaster. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * [MM-68404] MBE Phase 4: App-layer plugin hook wiring (#36407) * phase 4 * Fix nil rctx in TestChannelGuardLowercaseNormalization The PluginAPI struct literal was missing ctx: rctx after a refactor moved the rctx declaration below the struct construction, leaving api.ctx as nil. This caused a nil pointer dereference in reloadGuardCache when RegisterChannelGuard called store.RequestContextWithMaster(nil). Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * Remove ChannelWillBeMoved hook call from MoveChannel (phase 4) The hook and its ID were removed from mbe-phase-2 but the call site in MoveChannel and its i18n string were not cleaned up during the rebase. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * remove channel will be moved test * Remove RecapWillBeProcessed and MessageWillBeRewrittenByAI hook calls (phase 4) The hooks and their IDs were removed from mbe-phase-2 but the call sites in ProcessRecapChannel and RewriteMessage, their i18n strings, and their tests were not cleaned up during the rebase. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Revert channel_id plumbing on rewrite endpoint (phase 4) The channel_id field on RewriteRequest was added in phase 4 to feed the synthetic post passed to MessageWillBeRewrittenByAI. With that hook removed from mbe-phase-2, channel_id has no consumer; revert the field, the api4 validation, the app.RewriteMessage parameter, and the corresponding webapp client + hook plumbing. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * [MM-68555] MBE Phase 5: Channel-guard enforcement + two-phase dispatch (#36473) * phase 5 * Bake plugin counter-file paths into source instead of env vars t.Setenv panics when an ancestor test calls t.Parallel, so the two channel-guard tests broke under ENABLE_FULLY_PARALLEL_TESTS in CI. Build each plugin source per-subtest with its temp file path embedded as a Go literal — same pattern as TestPluginUploadsAPI. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Remove guarded helpers and tests for dropped hooks (phase 5) The runGuardedRecapWillBeProcessed and runGuardedMessageWillBeRewrittenByAI helpers were never wired (their app-layer call sites were already removed in the phase-4 cleanup), and the corresponding sub-tests across panic / allow / reject / partial plugins reference hooks that no longer exist. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * [MM-68405] MBE Phase 6: fire MessagesWillBeConsumed on the edit path (#36475) --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * rebase onto master --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
|
d4fc0ecb1c |
MM-68150: Upgrade golangci-lint to v2.12.2 (#36554)
* Simplify invite_people email parsing Replace backwards in-place mutation loop with a straightforward forward filter into a new slice. Extract into parseEmailList so the logic can be unit tested directly. * MM-68150: Upgrade golangci-lint to v2.12.2 Remove //go:fix inline from NewPointer, which is a generic function not yet supported by the inline analyzer, and fix 11 slicesbackward modernize issues flagged by the new version. * MM-68150: Enable all linters by default; disable those with >20 existing issues Switch from opt-in (default: none) to opt-out (default: all) so new linters added to golangci-lint are evaluated automatically. Explicitly disable every linter that has more than 20 pre-existing violations, deferring those for later cleanup. Also disable a handful of linters whose violations are intentional patterns in this codebase (nilerr, dogsled, sqlclosecheck, iotamixing, predeclared, containedctx, iface, gocheckcompilerdirectives, promlinter, goprintffuncname, gomoddirectives). * MM-68150: Fix mirror linter issues Replace Write([]byte(s)) with WriteString(s), and FindIndex([]byte(s)) with FindStringIndex(s), to avoid unnecessary allocations. * MM-68150: Fix nosprintfhostport linter issue Use net.JoinHostPort to construct host:port strings instead of fmt.Sprintf with a manually formatted pattern. * MM-68150: Fix rowserrcheck and sqlclosecheck linter issues Check rows.Err() after iteration loops in schema_dump.go. In the sqlx_wrapper test, defer rows.Close() rather than closing inline. * MM-68150: Fix nilnesserr linter issues — wrong variable in error handlers In 11 places, a stale variable (often the outer err from a prior assignment) was used instead of the freshly-checked error variable (appErr, rowErr, jsonErr, writeErr, esErr). Each produces a typed-nil wrapped in a non-nil interface, silently discarding the real error. * MM-68150: Add i18n string for app.compile_csv_chunks.write_error --------- Co-authored-by: Mattermost Build <build@mattermost.com> |
||
|
|
e3fbf8711f |
MM-68149: Upgrade to Go 1.26.2 (#36418)
* MM-68149: upgrade to Go 1.26.2 Update go directive in go.mod and .go-version. * MM-68149: replace pointer helpers with Go 1.26 new() Go 1.26 extends the built-in new() to accept an initial value expression, making typed-pointer helpers like model.NewPointer(x), bToP(x), and boolPtr(x) redundant. Replace every call site with new(x) and remove the now-unused helper functions and their //go:fix inline directives. * MM-68149: apply go fix for reflect API and format-string changes - reflect.Ptr → reflect.Pointer (renamed in Go 1.18, deprecated alias removed in 1.26) - reflect range-over-struct: for i := 0; i < t.NumField(); i++ → for field := range t.Fields() and the equivalent for Methods() and interface types - Fix format-string concatenation and variadic-arg mismatches flagged by go vet * MM-68149: update JPEG fixtures and test infrastructure for Go 1.26 encoder Go 1.26 ships a new image/jpeg encoder that produces slightly different output. Regenerate all JPEG fixture files and switch the comparison helpers from byte-equality to pixel-level comparison with a small per-channel tolerance, so minor encoder drift across patch versions is handled automatically. Add -update-fixtures flag to make it easy to regenerate fixtures after future major Go upgrades. Document the update procedure in tests/README.md. * MM-68149: CI check that go fix ./... produces no changes * Fix real bugs flagged by CodeRabbit review - group.go: set newGroup.MemberCount not group.MemberCount (member count was populated on the wrong variable and lost before publish/return) - file_test.go: guard compareImage(GetFilePreview) on the preview slice length, not the thumbnail slice length (copy-paste error) - config_test.go: remove duplicate MinimumLength assignment * fixup! Fix real bugs flagged by CodeRabbit review |
||
|
|
ebc066e7fd |
[MM-68273] Add system messages for share / unshare events (#36032)
* [MM-68273] Add system messages for share / unshare events * Fix CI * Address feedback * Fix CI * Remove unknown prop * Add missing tests |
||
|
|
c3ab0f7f78 |
MM-68191: Add plugin Receive APIs for shared channel sync (#35962)
* Add plugin APIs for plugin to sync data into shared channels |
||
|
|
88954db3de |
[MM-63434] Use forked PDF library with parsing depth limit (#35947)
* [MM-63434] Use forked PDF library with parsing depth limit Replace github.com/ledongthuc/pdf with a fork that limits object nesting depth during parsing. Add test coverage. * Reverting incorrect merge that lost the change to msgpack The error was in merge https://github.com/mattermost/mattermost/commit/64bdff88d853e030d6e0f243cf21370a78dde6ae |
||
|
|
ed80e8ba91 |
Shared channel UI for channel admins (#35448)
* Shared channel UI for channel admins * Fix lint * Use errors.is instead of using string comparison * Fix configuration check * Handle error when sharing an already shared channel * Remove unneeded disabled prop * Add missing tests * Frontend tweaks * Fix lint * Fix lint and test * Address coderabbit review * Fix removing unconfirmed remotes * Better handle errors while saving state * Remove unneeded state * Fix selector not being stable between different renders * Fix i18n and improve one type * Update webapp/channels/src/components/channel_settings_modal/share_channel_with_workspaces/share_channel_with_workspaces.scss Co-authored-by: Matthew Birtch <mattbirtch@gmail.com> * Update webapp/channels/src/components/channel_settings_modal/share_channel_with_workspaces/add_workspace_dropdown.tsx Co-authored-by: Matthew Birtch <mattbirtch@gmail.com> * Update webapp/channels/src/components/channel_settings_modal/share_channel_with_workspaces/share_channel_with_workspaces.scss Co-authored-by: Matthew Birtch <mattbirtch@gmail.com> * Update webapp/channels/src/components/channel_settings_modal/share_channel_with_workspaces/workspace_list.tsx Co-authored-by: Matthew Birtch <mattbirtch@gmail.com> * Update webapp/channels/src/components/channel_settings_modal/share_channel_with_workspaces/share_channel_with_workspaces.scss Co-authored-by: Matthew Birtch <mattbirtch@gmail.com> * Update webapp/channels/src/components/channel_settings_modal/share_channel_with_workspaces/share_channel_with_workspaces.scss Co-authored-by: Matthew Birtch <mattbirtch@gmail.com> * Apply suggestions from code review Co-authored-by: Matthew Birtch <mattbirtch@gmail.com> * Deal with settings option permissions * Add message when no remotes are available * Add dividers * Add disabled tooltip * Fix tests * Fix lint * Touch update at on share/unshare * Fix tests * Fix lint * Add missing await * Add e2e tests * Fix playwright prettier * Update server.prepare to have connected workspaces enabled by default * Revert changes on server.prepare and try with changes on server.generate * Fix shared channel configuration E2E tests (#35786) * Update webapp/channels/src/components/channel_settings_modal/share_channel_with_workspaces/share_channel_with_workspaces.scss Co-authored-by: Matthew Birtch <mattbirtch@gmail.com> * Update initial enabled state to properly handle saves * Update role name in e2e tests --------- Co-authored-by: Matthew Birtch <mattbirtch@gmail.com> Co-authored-by: Mattermost Build <build@mattermost.com> Co-authored-by: yasser khan <attitude3cena.yf@gmail.com> Co-authored-by: Doug Lauder <wiggin77@warpmail.net> |
||
|
|
161f0713a4 |
MM-66612: Add health flag to fast-fail when ES is offline (#35843)
* Add health flag to fast-fail when ES is offline
When Elasticsearch goes offline, the watcher takes up to 3 health
check cycles (~180 s) to detect the outage and stop the engine.
During that window every search query blocks for 30 s before
falling back to the database, and indexing goroutines pile up
unboundedly — causing server-wide slowness, posting failures,
and duplicate posts from client retries (MM-66612).
Introduce a `healthy` atomic flag on each ES/OpenSearch engine.
The watcher sets it to false on the *first* health-check failure
and back to true on success. `Broker.GetActiveEngines()` now
requires both `IsActive()` and `IsHealthy()`, so all search and
indexing operations skip the unhealthy engine immediately. The
existing 3-failure stop/restart cycle is unchanged and continues
to handle full recovery.
* Fix other tests
* Fix unrelated flaky test
* Use atomic.int32 everywhere
* Revert "Fix unrelated flaky test"
This reverts commit
|
||
|
|
a2a896a5de |
MM-67433: Elasticsearch health monitor (#35747)
* Add HealthCheck to search engine interface The watcher (next commit) needs a way to probe whether ES/OS is still reachable. Each backend does a cluster health request with a 5 s timeout, snapshotting the client under the read lock so the network call doesn't block Stop()/Start(). * Add search engine background watcher The old fire-and-forget ps.Go() calls for ES startup had no retry, no health monitoring, and no backoff. If Start() failed at boot the engine stayed down until the next config change or server restart. Replace with a single watcher goroutine that: - retries Start() with exponential backoff (15 s–5 m) - runs periodic health checks once active - stops the engine after N consecutive failures - reacts immediately to config/license changes - shuts down cleanly via context cancellation * Add tests for search engine watcher Covers retry-then-health transition, exponential backoff capping, disable/enable park-unpark via notify, intermittent vs threshold health failures, and edge cases (Start ok but not active, rapid config changes, failure counter reset). * Address review comments * make i18n-extract * Make Stop() a no-op if already stopped Both Elasticsearch and OpenSearch Stop() methods returned an error when the engine was already stopped. This forced every caller to handle or ignore a non-error condition. Returning nil instead is more idiomatic and simplifies all call sites. * Simplify config listener branches Merge the connectionChanged and startingES/stoppingES branches into one. Since Stop() is now a no-op when already stopped, we can always call Stop() then notify the watcher regardless of which condition triggered the change. * Restore license listener conditional order Keep the original order (license-added first, license-removed second) to reduce diff noise. The conditions are mutually exclusive so the order has no semantic effect. * Log consecutive failures in retry phase Track and log consecutiveFailures when Start() fails or returns nil but the engine is not active, so operators can see how many attempts have been made alongside the backoff duration. * Clarify health check timer reset placement Add a comment explaining why the timer reset lives outside the if/else: both success and below-threshold failure share the same health interval, while the critical-failure path continues before reaching this point. * Remove nested select in disabled-engine path Replace the nested select block with timer.Stop() + continue. The main loop's select already handles ctx.Done() and notify, so stopping the timer is enough to park until one of those fires. * Add context.Context to Start() Accept a context in SearchEngineInterface.Start() and propagate it to all network calls (checkMaxVersion, fetchServerInfo, index template creation). This lets the watcher's cancellable context flow through to the HTTP client, so a stuck Start() call returns promptly on shutdown instead of blocking until TCP timeout. * Simplify startSearchEngineWatcher comment Focus on the actual reason (long-lived, owns its lifecycle) rather than the shutdown-blocking detail, which is less relevant now that Start() accepts a cancellable context. * Make the comment on the goroutine more accurate * Fix log * Revert the branches merge This was causing a race condition in the TestElasticsearchAggregation test. * Own Start/Stop by the engine watcher The config listener simply notifies now, so it's easier to follow the logic of the calls. * Refactor the engine watcher into its own type Simplify the code by: 1. Moving the whole watcher into its own type, so that the PlatformService contains an instance of it, instead of all the locks and channels 2. Splitting the main loop into functions. John Carmack may not like this, but it's way easier to read and follow. * make i18n-extract * Rename notify > reevaluate * Park the watcher if there is no license * Call reevaluate from within requestRestart * Use RequestTimeoutSeconds instead of hardcoded 5s * Use atomic.Int32 in the tests as well * Add defensive code ta watcher exit Stop the engine if it was still running when the watcher exits, having a safety net for scenarios like race conditions between Start and a cancelled context. --------- Co-authored-by: Mattermost Build <build@mattermost.com> |
||
|
|
7dccd6eba6 |
MM-68199: Fix shared channel membership sync error for local users (#35938)
When a channel is shared between two instances, the sender was including membership changes for users that originated from the target remote. On the receiver those users are local (RemoteID=""), causing the RemoteID check to fail with "remoteID mismatch", creating log noise. Fix both sides: sender now skips users belonging to the target remote, and receiver silently skips local users instead of logging an error. |
||
|
|
540ccc599b |
MM-68179: Run sendLoop workers on all HA nodes (#35909)
* MM-68179: Run sendLoop workers on all HA nodes
In HA clusters, sendLoop worker goroutines only ran on the leader node.
When an API request to send a channel invite landed on a non-leader node,
SendMsg enqueued the task to a local in-memory channel but no goroutine
consumed it, silently losing the message. Fix by starting sendLoop workers
in Start() on all nodes, independent of the leader-only ping lifecycle.
- Separate sendLoop lifecycle (Start/Shutdown) from ping lifecycle
(pingStart/pingStop on leader change)
- Rename resume/pause to pingStart/pingStop for clarity
- Change Active() to mean "service started" via atomic.Bool
- Remove SetActive (no longer needed; tests use Start())
* address review comment
* Added idempotency guard to Start()
* Start() and Shutdown(): CompareAndSwap instead of Load/Store — eliminates races where concurrent calls could both proceed. Only the winner of the CAS executes; the loser returns nil
immediately.
Ping test: replaced time.Sleep with assert.Never/assert.Eventually — no more brittle fixed sleeps. Uses assert.Never to verify no pings fire on non-leader, and assert.Eventually to
verify pings stop after losing leadership (snapshot-then-compare pattern).
* make unit tests parallel capable
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
|
||
|
|
e694e86d63 |
MM-68204: Use multi-level logging for shared channel and remote cluster service errors (#35949)
* Use multi-level logging for shared channel and remote cluster service errors Service-specific log levels (LvlRemoteClusterServiceError, LvlSharedChannelServiceError) were hidden from the main log by default, requiring explicit configuration to see them. Switch all call sites to use LogM with multi-level combos so each log line is attributed to both the standard level (error/warn) and the service-specific level. This surfaces errors in the main log while preserving the ability to isolate them into dedicated files. Each instance was reviewed and either kept as error (DB failures, security issues, config errors, exhausted retries on critical data) or downgraded to warn (transient network failures, remote-side reported errors with retry logic, non-critical data like profile images, reactions, acknowledgements, and status). |
||
|
|
3888a69479 |
MM-68158: Fix shared channel remote display and notify UI on invite completion (#35908)
* MM-68158: Fix shared channel remote display and add WebSocket notification Fix getSharedChannelRemotes API handler passing ChannelId instead of RemoteId to GetRemoteCluster, which always failed the lookup. Add RemoteId to SharedChannelRemoteStatus model and store query. Add shared_channel_remote_updated WebSocket event published from the onInvite callback so the UI refreshes its cached remote names when the async invite completes, instead of showing the generic "Shared with trusted organizations" fallback. * Improved unit tests per review comments |
||
|
|
f7f2d944e8 | upgrade golangci-lint (#35845) | ||
|
|
48f2fd0873 |
Merge the Integrated Boards MVP feature branch (#35796)
* Add CreatedBy and UpdatedBy to the properties fields and values (#34485) * Add CreatedBy and UpdatedBy to the properties fields and values * Fix types --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Adds ObjectType to the property fields table (#34908) Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Update ObjectType migration setting an empty value and marking the column as not null (#34915) Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Adds uniqueness mechanisms to the property fields (#35058) * Adds uniqueness mechanisms to the property fields After adding ObjectType, this commit ensures that both the PSAv1 and PSAv2 schemas are supported, and enforces property uniqueness through both database indexes and a logical check when creating new property fields. * Adds uniqueness check to property updates Updates are covered on this commit and we refactor as well the SQL code to use the squirrel builder and work better with the conditional addition of the `existingID` piece of the query. * Add translations to error messages * Fixing retrylayer mocks * Remove retrylayer duplication * Address review comments * Fix comment to avoid linter issues * Address PR comments * Update server/channels/db/migrations/postgres/000157_add_object_type_to_property_fields.down.sql Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com> * Update server/channels/db/migrations/postgres/000157_add_object_type_to_property_fields.up.sql Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com> * Update server/channels/db/migrations/postgres/000157_add_object_type_to_property_fields.up.sql Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com> * Update field validation to check only for valid target types * Update migrations to avoid concurrent index creation within a transaction * Update migrations to make all index ops concurrent * Update tests to use valid PSAv2 property fields * Adds a helper for valid PSAv2 TargetTypes --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com> * Fix property tests (#35388) Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Adds Integrated Boards feature flag (#35378) Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Adds Integrated Boards MVP API changes (#34822) This PR includes the necessary changes for channels and posts endpoints and adds a set of generic endpoints to retrieve and manage property fields and values following the new Property System approach. Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> Co-authored-by: Mattermost Build <build@mattermost.com> * Property System Architecture permissions for v2 (#35113) * Adds uniqueness mechanisms to the property fields After adding ObjectType, this commit ensures that both the PSAv1 and PSAv2 schemas are supported, and enforces property uniqueness through both database indexes and a logical check when creating new property fields. * Adds uniqueness check to property updates Updates are covered on this commit and we refactor as well the SQL code to use the squirrel builder and work better with the conditional addition of the `existingID` piece of the query. * Add translations to error messages * Add the permissions to the migrations, model and update the store calls * Adds the property field and property group app layer * Adds authorization helpers for property fields and values * Make sure that users cannot lock themselves out of property fields * Migrate permissions from a JSON column to three normalized columns * Remove the audit comment * Use target level constants in authorization * Log authorization membership failures * Rename admin to sysadmin * Fix i18n sorting --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Add Views store and app layer (#35361) * Add Views store and app layer for Integrated Boards Implements the View entity (model, SQL store, service, app) as described in the Integrated Boards tech spec. Views are channel-scoped board configurations with typed props (board, kanban subviews) and soft-delete. - public/model: View, ViewBoardProps, Subview, ViewPatch types with PreSave/PreUpdate/IsValid/Patch/Clone/Auditable - Migration 158: Views table with jsonb Props column and indexes - SqlViewStore: CRUD with nil-safe Props marshaling (AppendBinaryFlag) - ViewService: CreateView seeds default kanban subview and links the boards property field; caches boardPropertyFieldID at startup - App layer: CreateView/GetView/GetViewsForChannel/UpdateView/DeleteView with channel-membership permission checks and WebSocket events (view_created, view_updated, view_deleted) - doSetupBoardsPropertyField: registers the Boards property group and board field in NewServer() before ViewService construction - GetFieldByName now returns store.ErrNotFound instead of raw sql.ErrNoRows * Move permission checks out of App layer for views - Remove HasPermissionToChannel calls from all App view methods - Drop userID params from GetView, GetViewsForChannel, UpdateView, DeleteView - Fix doSetupBoardsPropertyField to include required TargetType for PSAv2 field * Make View service generic and enforce board validation in model - Remove board-specific auto-setup from service and server startup - Enforce that board views require Props, at least one subview, and at least one linked property in IsValid() - Move default subview seeding out of app layer; callers must provide valid props - Call PreSave on subviews during PreUpdate to assign IDs to new subviews - Update all tests to reflect the new validation requirements * Restore migrations files to match base branch * Distinguish ErrNotFound from other errors in view store Get * Use CONCURRENTLY and nontransactional for index operations in views migration * Split views index creation into separate nontransactional migrations * Update migrations.list * Update i18n translations for views * Fix makeView helper to include required Props for board view validation * Rename ctx parameter from c to rctx in OAuthProvider mock * Remove views service layer, call store directly from app * Return 500 for unexpected DB errors in GetView, 404 only for not-found * Harden View model: deep-copy Props, validate linked property IDs - Add ViewBoardProps.Clone() to deep-copy LinkedProperties and Subviews - Use it in View.Clone() and View.Patch() to prevent shared-slice aliasing - Iterate over LinkedProperties in View.IsValid() and reject invalid IDs with a dedicated i18n key - Register ViewStore in storetest AssertExpectations so mock expectations are enforced - Add tests covering all new behaviours * Restore autotranslation worker_stopped i18n translation * Fix view store test IDs and improve error handling in app layer - Use model.NewId() for linked property IDs in testUpdateView to fix validation failure (IsValid rejects non-UUID strings) - Fix import grouping in app/view.go (stdlib imports in one block) - Return 404 instead of 500 when Update/Delete store calls return ErrNotFound (e.g. concurrent deletion TOCTOU race) * Add View store mock to retrylayer test genStore helper The View store was added to the store interface but the genStore() helper in retrylayer_test.go was not updated, causing TestRetry to panic. Also removes the duplicate Recap mock registration. * Refactor view deletion and websocket event handling; update SQL store methods to use query builder * revert property field store * Remove useless migrations * Add cursor-based pagination to View store GetForChannel - Add ViewQueryCursor and ViewQueryOpts types with validation - Return (views, cursor, error) for caller-driven pagination - PerPage clamping: <=0 defaults to 20, >200 clamps to 200 - Support IncludeDeleted filter - Add comprehensive store tests for pagination, cursor edge cases, PerPage clamping, and invalid input rejection - Add app layer test for empty channelID → 400 - Update interface, retrylayer, timerlayer, and mock signatures Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Refactor test loops in ViewStore tests for improved readability * change pagination to limit/offset * Add upper-bound limits on View Subviews and LinkedProperties Defense-in-depth validation: cap Subviews at 50 and LinkedProperties at 500 to prevent abuse below the 300KB payload limit. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * MM-67388, MM-66528, MM-67750: Add View REST API endpoints, websocket events, and sort order (#35442) * Add Views store and app layer for Integrated Boards Implements the View entity (model, SQL store, service, app) as described in the Integrated Boards tech spec. Views are channel-scoped board configurations with typed props (board, kanban subviews) and soft-delete. - public/model: View, ViewBoardProps, Subview, ViewPatch types with PreSave/PreUpdate/IsValid/Patch/Clone/Auditable - Migration 158: Views table with jsonb Props column and indexes - SqlViewStore: CRUD with nil-safe Props marshaling (AppendBinaryFlag) - ViewService: CreateView seeds default kanban subview and links the boards property field; caches boardPropertyFieldID at startup - App layer: CreateView/GetView/GetViewsForChannel/UpdateView/DeleteView with channel-membership permission checks and WebSocket events (view_created, view_updated, view_deleted) - doSetupBoardsPropertyField: registers the Boards property group and board field in NewServer() before ViewService construction - GetFieldByName now returns store.ErrNotFound instead of raw sql.ErrNoRows * Move permission checks out of App layer for views - Remove HasPermissionToChannel calls from all App view methods - Drop userID params from GetView, GetViewsForChannel, UpdateView, DeleteView - Fix doSetupBoardsPropertyField to include required TargetType for PSAv2 field * Make View service generic and enforce board validation in model - Remove board-specific auto-setup from service and server startup - Enforce that board views require Props, at least one subview, and at least one linked property in IsValid() - Move default subview seeding out of app layer; callers must provide valid props - Call PreSave on subviews during PreUpdate to assign IDs to new subviews - Update all tests to reflect the new validation requirements * Restore migrations files to match base branch * Distinguish ErrNotFound from other errors in view store Get * Use CONCURRENTLY and nontransactional for index operations in views migration * Split views index creation into separate nontransactional migrations * Update migrations.list * Update i18n translations for views * Fix makeView helper to include required Props for board view validation * Rename ctx parameter from c to rctx in OAuthProvider mock * Remove views service layer, call store directly from app * Return 500 for unexpected DB errors in GetView, 404 only for not-found * Harden View model: deep-copy Props, validate linked property IDs - Add ViewBoardProps.Clone() to deep-copy LinkedProperties and Subviews - Use it in View.Clone() and View.Patch() to prevent shared-slice aliasing - Iterate over LinkedProperties in View.IsValid() and reject invalid IDs with a dedicated i18n key - Register ViewStore in storetest AssertExpectations so mock expectations are enforced - Add tests covering all new behaviours * Restore autotranslation worker_stopped i18n translation * Fix view store test IDs and improve error handling in app layer - Use model.NewId() for linked property IDs in testUpdateView to fix validation failure (IsValid rejects non-UUID strings) - Fix import grouping in app/view.go (stdlib imports in one block) - Return 404 instead of 500 when Update/Delete store calls return ErrNotFound (e.g. concurrent deletion TOCTOU race) * Add View store mock to retrylayer test genStore helper The View store was added to the store interface but the genStore() helper in retrylayer_test.go was not updated, causing TestRetry to panic. Also removes the duplicate Recap mock registration. * Refactor view deletion and websocket event handling; update SQL store methods to use query builder * revert property field store * Add View API endpoints with OpenAPI spec, client methods, and i18n Implement REST API for channel views (board-type) behind the IntegratedBoards feature flag. Adds CRUD endpoints under /api/v4/channels/{channel_id}/views with permission checks matching the channel bookmark pattern. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Remove useless migrations * Add cursor-based pagination to View store GetForChannel - Add ViewQueryCursor and ViewQueryOpts types with validation - Return (views, cursor, error) for caller-driven pagination - PerPage clamping: <=0 defaults to 20, >200 clamps to 200 - Support IncludeDeleted filter - Add comprehensive store tests for pagination, cursor edge cases, PerPage clamping, and invalid input rejection - Add app layer test for empty channelID → 400 - Update interface, retrylayer, timerlayer, and mock signatures Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add cursor-based pagination to View API for channel views * Enhance cursor handling in getViewsForChannel and update tests for pagination * Refactor test loops in ViewStore tests for improved readability * Refactor loop in TestGetViewsForChannel for improved readability * change pagination to limit/offset * switch to limit/offset pagination * Add upper-bound limits on View Subviews and LinkedProperties Defense-in-depth validation: cap Subviews at 50 and LinkedProperties at 500 to prevent abuse below the 300KB payload limit. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add view sort order API endpoint Add POST /api/v4/channels/{channel_id}/views/{view_id}/sort_order endpoint following the channel bookmarks reorder pattern. Includes store, app, and API layers with full test coverage at each layer. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add connectionId to view WebSocket events and sort_order API spec Thread connectionId from request header through all view handlers (create, update, delete, sort_order) to WebSocket events, matching the channel bookmarks pattern. Add sort_order endpoint to OpenAPI spec. Update minimum server version to 11.6. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Remove duplicate View/ViewPatch definitions from definitions.yaml The merge from integrated-boards-mvp introduced duplicate View and ViewPatch schema definitions that were already defined earlier in the file with more detail (including ViewBoardProps ref and enums). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Update minimum server version to 11.6 in views API spec Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add missing translations for view sort order error messages Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Merge integrated-boards-mvp into ibmvp_api-views; remove spec files Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix flaky TestViewStore timestamp test on CI Add sleep before UpdateSortOrder to ensure timestamps differ, preventing same-millisecond comparisons on fast CI machines. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * remove duplicate views.yaml imclude * Use c.boolString() for include_deleted query param in GetViewsForChannel Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix views.yaml sort order schema: use integer type and require body Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Refactor view sort order tests to use named IDs instead of array indices Extract idA/idB/idC from views slice and add BEFORE/AFTER comments to make stateful subtest ordering easier to follow. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Return 404 instead of 403 for view operations on deleted channels Deleted channels should appear non-existent to callers rather than revealing their existence via a 403. Detailed error text explains the context for debugging. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * add missing channel deleteat checks * Use c.Params.Page instead of manual page query param parsing in getViewsForChannel c.Params already validates and defaults page/per_page, so the manual parsing was redundant. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add support for total count in views retrieval * Add tests for handling deleted views in GetViewsForChannel and GetView * Short-circuit negative newIndex in UpdateSortOrder before opening transaction Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add per-channel limit on views to bound UpdateSortOrder cost Without a cap, unbounded view creation makes sort-order updates increasingly expensive (CASE WHEN per view, row locks). Adds MaxViewsPerChannel=50 constant and enforces it in the app layer before saving. Includes API and app layer tests. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Remove include_deleted support from views API Soft-deleted views are structural metadata with low risk, but no other similar endpoint (e.g. channel bookmarks) exposes deleted records without an admin gate. Rather than adding an admin-only permission check for consistency, remove the feature entirely since there is no current use case. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Update view permissions to require `create_post` instead of channel management permissions * Remove obsolete view management error messages for direct and group messages --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * feat(migrations): add user tracking and object type to property fields - Introduced user tracking columns (CreatedBy, UpdatedBy) to PropertyFields and PropertyValues. - Added ObjectType column to PropertyFields with associated unique indexes for legacy and typed properties. - Created new migration scripts for adding and dropping these features, including necessary indexes for data integrity. - Established views for managing property fields with new attributes. This update enhances the schema to support better tracking and categorization of property fields. * Add Property System Architecture v2 API endpoints (#35583) * Adds uniqueness mechanisms to the property fields After adding ObjectType, this commit ensures that both the PSAv1 and PSAv2 schemas are supported, and enforces property uniqueness through both database indexes and a logical check when creating new property fields. * Adds uniqueness check to property updates Updates are covered on this commit and we refactor as well the SQL code to use the squirrel builder and work better with the conditional addition of the `existingID` piece of the query. * Add translations to error messages * Add the permissions to the migrations, model and update the store calls * Adds the property field and property group app layer * Adds authorization helpers for property fields and values * Make sure that users cannot lock themselves out of property fields * Migrate permissions from a JSON column to three normalized columns * Remove the audit comment * Use target level constants in authorization * Log authorization membership failures * Rename admin to sysadmin * Adds the Property System Architecture v2 API endpoints * Adds permission checks to the create field endpoint * Add target access checks to value endpoints * Add default branches for object_type and target_type and extra guards for cursor client4 methods * Fix vet API mismatch * Fix error checks * Fix linter * Add merge semantics for property patch logic and API endpoint * Fix i18n * Fix duplicated patch elements and early return on bad cursor * Update docs to use enums * Fix i18n sorting * Update app layer to return model.AppError * Adds a limit to the number of property values that can be patched in the same request * Require target_type filter when searching property fields * Add objectType validation as part of field.IsValid() * Fix linter * Fix test with bad objecttpye * Fix test grouping --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * MM-67968: Flatten view model — remove icon, subviews, typed board props (#35726) * feat(views): flatten view model by removing icon, subview, and board props Simplifies the View data model as part of MM-67968: removes Icon, Subview, and ViewBoardProps types; renames ViewTypeBoard to ViewTypeKanban; replaces typed Props with StringInterface (map[string]any); adds migration 000167 to drop the Icon column from the Views table. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * feat(api): update views OpenAPI spec to reflect flattened model Removes ViewBoardProps, Subview, and icon from the View and ViewPatch schemas. Changes type enum from board to kanban. Replaces typed props with a free-form StringInterface object. Aligns with MM-67968. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * refactor(views): simplify store by dropping dbView and marshalViewProps StringInterface already implements driver.Valuer and sql.Scanner, so the manual JSON marshal/unmarshal and the dbView intermediate struct were redundant. model.View now scans directly from the database. Also removes the dead ViewMaxLinkedProperties constant and wraps the Commit() error in UpdateSortOrder. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(api): allow arbitrary JSON in view props OpenAPI schema The props field was restricted to string values via additionalProperties: { type: string }, conflicting with the Go model's StringInterface (map[string]any). Changed to additionalProperties: true in View, ViewPatch, and inline POST schemas. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * Adds basic implementation of the generic redux store for PSAv2 (#35512) * Adds basic implementation of the generic redux store for PSAv2 * Add created_by and updated_by to the test fixtures * Make target_id, target_type and object_type mandatory * Wrap getPropertyFieldsByIds and getPropertyValuesForTargetByFieldIds with createSelector * Address PR comments --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Adds websocket messages for the PSAv2 API events (#35696) * Adds uniqueness mechanisms to the property fields After adding ObjectType, this commit ensures that both the PSAv1 and PSAv2 schemas are supported, and enforces property uniqueness through both database indexes and a logical check when creating new property fields. * Adds uniqueness check to property updates Updates are covered on this commit and we refactor as well the SQL code to use the squirrel builder and work better with the conditional addition of the `existingID` piece of the query. * Add translations to error messages * Add the permissions to the migrations, model and update the store calls * Adds the property field and property group app layer * Adds authorization helpers for property fields and values * Make sure that users cannot lock themselves out of property fields * Migrate permissions from a JSON column to three normalized columns * Remove the audit comment * Use target level constants in authorization * Log authorization membership failures * Rename admin to sysadmin * Adds the Property System Architecture v2 API endpoints * Adds permission checks to the create field endpoint * Add target access checks to value endpoints * Add default branches for object_type and target_type and extra guards for cursor client4 methods * Fix vet API mismatch * Fix error checks * Fix linter * Add merge semantics for property patch logic and API endpoint * Fix i18n * Fix duplicated patch elements and early return on bad cursor * Update docs to use enums * Fix i18n sorting * Update app layer to return model.AppError * Adds a limit to the number of property values that can be patched in the same request * Adds websocket messages for the PSAv2 API events * Add IsPSAv2 helper to the property field for clarity * Add guard against nil returns on field deletion * Add docs to the websocket endpoints --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * migrations: consolidate views migrations and reorder after master - Merged 000165 (create Views) with 000167 (drop Icon) since Icon was never needed - Renumbered branch migrations 159-166 → 160-167 so master's 000159 (deduplicate_policy_names) runs first - Regenerated migrations.list Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Add API endpoint to retrieve posts for a specific view (#35604) Automatic Merge * Apply fixes after merge * Return a more specific error from getting multiple fields * Prevent getting broadcast params on field deletion if not needed * Remove duplicated migration code * Update property conflict code to always use master * Adds nil guard when iterating on property fields * Check that permission level is valid before getting rejected by the database * Validate correctness on TargetID for PSAv2 fields * Avoid PSAv1 using permissions or protected * Fix test data after validation change * Fix flaky search test * Adds more posts for filter use cases to properly test exclusions --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com> Co-authored-by: Julien Tant <julien@craftyx.fr> Co-authored-by: Mattermost Build <build@mattermost.com> Co-authored-by: Julien Tant <785518+JulienTant@users.noreply.github.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
|
fb11968f87 |
[MM-65701] Fix for docextractor archive handling (#34983)
* do not attempt to decompress 7zip files in docextractor * use SevenZip.Match to detect a 7z file instead of our own method * merge conflicts --------- Co-authored-by: Mattermost Build <build@mattermost.com> |
||
|
|
f0b2a36dbc |
MM-67616: Refactor shared channel membership sync to use ChannelMemberHistory (#35619)
* Refactor shared channel membership sync to use ChannelMemberHistory (MM-67616) Replace the trigger-time membership sync mechanism with a cursor-based approach using ChannelMemberHistory, aligning membership sync with the established pattern used by posts and reactions. Previously, membership changes were built into SyncMsg at trigger time and sent via a separate TopicChannelMembership code path. This meant removals were lost if a remote was offline, since ChannelMembers hard-deletes rows. Now, membership changes are fetched from ChannelMemberHistory at sync time using the LastMembersSyncAt cursor, detecting both joins and leaves reliably. The data flows through the normal syncForRemote pipeline alongside posts, reactions, and other sync data. Key changes: - Add GetMembershipChanges store method for ChannelMemberHistory - Add fetchMembershipsForSync and sendMembershipSyncData to sync pipeline - Replace HandleMembershipChange with NotifyMembershipChanged (trigger-only) - Remove conflict detection (idempotent add/remove resolves naturally) - Remove per-user membership tracking (GetUserChanges, UpdateUserLastMembershipSyncAt) - Add MembershipErrors to SyncResponse - Keep TopicChannelMembership receiver for one release cycle (backward compat) |
||
|
|
ad03248cd3 |
MM-67872 Fixing detection issue in image proxy (#35669)
* Refactoring image proxy * Improving resilience of peeking mechanism * Moving should304 check * Replacing functions with text encoding library |
||
|
|
742e0be950 |
Validate RefreshedToken differs from original invite token (#34864)
* Validate that RefreshedToken differs from original invite token in remote cluster confirmation * Add unit test for MM-67098 --------- Co-authored-by: JG Heithcock <jgheithcock@gmail.com> |
||
|
|
3057ae7e83 |
MM-67646 slack import improvements (#35490)
* improves logging during slack import * add imported users with no password and force reset flow * use i18n key ids during test |
||
|
|
b6e5264731 |
[MM-67739] Rename SlackAttachment to MessageAttachment across the codebase (#35445)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Mattermost Build <build@mattermost.com> |
||
|
|
2ada8d7659 |
MM-67540 - Allow searching public channel messages without channel membership (#35298)
* UpdateByQuery methods for channel_type; rewrite reindexChannelPosts log pre-fetch error in channel Update and upgrade reindexChannelPosts to error level * add backfill orchestration, config listener, webapp toggle changes fix misleading backfill complete log when SaveOrUpdate fails * add integration & unit tests for public channel search and backfill * fix nil pointer dereference on UpdateByQuery response and log partial failures * add tests for compliance mode override and P channel post leakage * update system console snapshots * add instructions to error message * improve compliance-mode test * getAllChannels doesn't filter by O/S, need to do ourselves * in search, load channel info for channels we're not a member of * blank commit -- something is wrong with github, maybe this will help * improve channelType passing; error msg; simplify settings naming * debug logging for timing backfill and channel change; TO BE REVERTED * fix getMissingChannelsFromFiles in search as well * blank commit |
||
|
|
033867a344 |
MM-67522 Add tests for syncing user statuses (#35269)
* MM-67522 Add tests for syncing user statuses * Clean up newly added tests * Fix style * Use SyncResponse.StatusErrors when statuses fail to sync |
||
|
|
8738f8c4b3 | MM-67099 - Membership Sync fix (#35230) | ||
|
|
c6d00615dd |
Force membership sync on reconnect (#35299)
Co-authored-by: Mattermost Build <build@mattermost.com> |
||
|
|
b947f1c38a | Add fileSize limit to extractors (#35200) | ||
|
|
b5a816a657 |
Add audits for accessing posts without membership (#31266)
* Add audits for accessing posts without membership * Fix tests * Use correct audit level * Address feedback * Add missing checks all over the app * Fix lint * Fix test * Fix tests * Fix enterprise test * Add missing test and docs * Fix merge * Fix lint * Add audit logs on the web socket hook for permalink posts * Fix lint * Fix merge conflicts * Handle all events with "non_channel_member_access" parameter * Fix lint and tests * Fix merge * Fix tests |
||
|
|
338cf4a5a0 |
Add docs re Shared Channels APIs calls and data flow (#34682)
* Add docs re Shared Channels APIs calls and data flow * add diagrams |
||
|
|
4ba7f7e16e |
MM-66202: Migrate to aws-sdk-go-v2 (#34496)
* updated aws-sdk dependency to aws-sdk-go-v2 * simplify error handling in case of timeout errors --------- Co-authored-by: Mattermost Build <build@mattermost.com> |
||
|
|
b2df9be70b | Fix errcheck linter errors in helpers (#31578) | ||
|
|
892a7c9c69 | Use golangci-lints's build-in modernize linter (#34341) | ||
|
|
1386024013 | Fix MM-65152 (#34199) | ||
|
|
c21ef29f02 |
Flag post API (#33765)
* Added enable/disable setting and feature flag * added rest of notifgication settings * Added backend for content flagging setting and populated notification values from server side defaults * WIP user selector * Added common reviewers UI * Added additonal reviewers section * WIP * WIP * Team table base * Added search in teams * Added search in teams * Added additional settings section * WIP * Inbtegrated reviewers settings * WIP * WIP * Added server side validation * cleanup * cleanup * [skip ci] * Some refactoring * type fixes * lint fix * test: add content flagging settings test file * test: add comprehensive unit tests for content flagging settings * enhanced tests * test: add test file for content flagging additional settings * test: add comprehensive unit tests for ContentFlaggingAdditionalSettingsSection * Added additoonal settings test * test: add empty test file for team reviewers section * test: add comprehensive unit tests for TeamReviewersSection component * test: update tests to handle async data fetching in team reviewers section * test: add empty test file for content reviewers component * feat: add comprehensive unit tests for ContentFlaggingContentReviewers component * Added ContentFlaggingContentReviewersContentFlaggingContentReviewers test * test: add notification settings test file for content flagging * test: add comprehensive unit tests for content flagging notification settings * Added ContentFlaggingNotificationSettingsSection tests * test: add user profile pill test file * test: add comprehensive unit tests for UserProfilePill component * refactor: Replace enzyme shallow with renderWithContext in user_profile_pill tests * Added UserProfilePill tests * test: add empty test file for content reviewers team option * test: add comprehensive unit tests for TeamOptionComponent * Added TeamOptionComponent tests * test: add empty test file for reason_option component * test: add comprehensive unit tests for ReasonOption component * Added ReasonOption tests * cleanup * Fixed i18n error * fixed e2e test lijnt issues * Updated test cases * Added snaoshot * Updated snaoshot * lint fix * WIP * lint fix * Added post flagging properties setup * review fixes * updated snapshot * CI * Added base APIs * Fetched team status data on load and team switch * WIP * Review fixes * wip * WIP * Removed an test, updated comment * CI * Added tests * Added tests * Lint fix * Added API specs * Fixed types * CI fixes * API tests * lint fixes * Set env variable so API routes are regiustered * Test update * term renaming and disabling API tests on MySQL * typo * Updated store type definition * Minor tweaks * Added tests * Removed error in app startup when content flaghging setup fails * Updated sync condition: * Flag message modal basE * added post preview * displaying options * Adde comment input * Updated tests and docs * finction rename * WIP * Updated tests * refactor * lint fix * MOved to data migration * lint fix * CI * added new migration mocks * Used setup for tests * some comment * Removed unnecesseery nil check * Form validation * WIP tests * WIP tests * WIP tests * fix: mock content flagging config selector with correct reasons format Co-authored-by: aider (anthropic/claude-sonnet-4-20250514) <aider@aider.chat> * fix: add mock for getContentFlaggingConfig in flag post modal test Co-authored-by: aider (anthropic/claude-sonnet-4-20250514) <aider@aider.chat> * Updated error code order in API docs * removed empty files * Added tests * lint fixes * minor tweak * lint fix * type fix * fixed test * nit * test enhancements * API WIP * API WIP * creating values * creating content flagging channel and properties * Able to save properties * Added another property field * WIP * WIP * Added validations * Added data validations and hidden post if confifgured to * lint fixes * Added API spec * Added some tests * Added tests for getContentReviewBot * test: add comprehensive tests for getContentReviewChannels function * Added more app layer tests * Added TestCanFlagPost * test: Add comprehensive tests for FlagPost function * Added all app layer tests * Removed a file that was reamoved downstream * test: add content flagging test file * test: add comprehensive tests for FlagContentRequest.IsValid method * Added model tests * test: add comprehensive tests for SqlPropertyValueStore.CreateMany * test: add comprehensive tests for flagPost() API function * Added API tests * linter fix * WIP * sent post flagging confirmation message * fixed i18n nissues * fixed i18n nissues * CI * Updated test * fix: reset contentFlaggingGroupId for test isolation in content flagging tests * removed cached group ID * removed debug log * review fixes * Used correct ot name * CI * Updated mobile text * Handled JSON error * fixerdf i18n * CI * Integrate flag post api (#33798) * WIP * WIP * Added API call * test: add test for Client4.flagPost API call in FlagPostModal * fix: remove userEvent.setup() from flag post modal test * test: wrap submit button click in act for proper state updates * Updated tests * lint fix * CI * Updated to allow special characters in comments * Handled empty comment * Used finally * CI * Fixed test * Spillage card integration (#33832) * Created getContentFlaggingFields API * created getPostPropertyValues API * WIP * Created useContentFlaggingFields hook * WIP * WIP * Added option to retain data for reviewers * Displayed deleted post's preview * DIsplayed all properties * Adding field name i18n * WIP - managing i18n able texts * Finished displaying all fields * Manual cleanup * lint fixes * team role filter logic fix * Fixed tests * created new API to fetch flagged posts * lint fix * Added new client methods * test: add comprehensive tests for content flagging APIs * Added new API tests * fixed openapi spec * Fixed DataSpillageReport tests * Fixed PostMarkdown test * Fixed PostPreviewPropertyRenderer test * Added metadata to card renderer * test fixes * Added no comment placeholder * Fixed test * refactor: improve test mocking for data spillage report component * test mock updates * Updated reducer * not resetting mocks * WIP * review fixes * CI * Fixed * fixes * Content flagging actions implementation (#33852) * Added view detail button * Created RemoveFlaggedMessageConfirmationModal modal * Added key and remove flag request modal * IMplemented delete flagged post * Handled edge cases of deleting flagged post * keep message * UI integration * Added WS event for post report update and handled deleted files of flagged post * Added error handling in keep/remove forms * i18n fixes * Updated OpenAPI specs * fixed types * fixed types * refactoring * Fixed tests * review fixes * Added new property translations * Improved test * fixed test * CI * fixes * CI * fixed a test * CI --------- Co-authored-by: aider (anthropic/claude-sonnet-4-20250514) <aider@aider.chat> |
||
|
|
18eb1347db |
[MM-64900] Migrate to use request.CTX instead of context.Context (#33541)
* Migrate GetRoleByName * Migrate users GetUsers * Migrate Post and Thread store * Migrate channel store * Fix TestConvertGroupMessageToChannel * Fix TestGetMemberCountsByGroup * Fix TestPostStoreLastPostTimeCache |
||
|
|
d78d59babe |
Standardize request.CTX parameter naming to rctx (#33499)
* Standardize request.CTX parameter naming to rctx - Migrate 886 request.CTX parameters across 147 files to use consistent 'rctx' naming - Updated function signatures from 'c', 'ctx', and 'cancelContext' to 'rctx' - Updated function bodies to reference the new parameter names - Preserved underscore parameters unchanged as they are unused - Fixed method receiver context issue in store.go 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * Use request.CTX interface in batch worker * Manual fixes * Fix parameter naming * Add linter check --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
8cace74692 |
MM-64486: Remove telemetry (#33606)
* MM-64486: Remove telemetry Remove telemetry from Mattermost. We're no longer relying on Rudder upstream, and no longer making use of this information. * recover mock for SystemStore.Get * Fix TestClearPushNotificationSync by adding missing SystemStore mock The test was failing because the SystemStore mock was missing the Get() method that's required by the ServerId() function. Added the missing mock to return a StringMap with SystemServerId. * fix mocking issue * Remove now-unused telemetry and constants * Remove "Disable telemetry events" debug setting * Remove empty functions * Remove most "Telemetry tracking removed" comments * Remove remains of DataPrefetch telemetry * Remove now-unused prop from InviteMembersButton * Remove trackDotMenuEvent * Remove some more leftover comments * Remove lingering logic related to trackingLocation * Remove now-unused argument from useCopyText * Remove lingering telemetry references from PreparingWorkspace * fixup Remove trackDotMenuEvent * Remove lingering telemetry references from signup page and password check * Update snapshots and fix test broken by my changes * Fix unintended behavior change in thread list filtering Remove handleSetFilter wrapper that was accidentally modified during telemetry removal. The function was calling clear() when switching to unread filter, which was not the original behavior. Use setFilter directly instead, restoring the original functionality. * Remove unused useOpenDowngradeModal hook The useOpenDowngradeModal hook was not being used anywhere in the codebase. * Remove unused expandableLink from useExpandOverageUsersCheck The expandableLink return value was not being used by any components. * Re-add missing TeamLinkClicked performance telemetry The mark(Mark.TeamLinkClicked) call was accidentally removed from the handleSwitch function. This telemetry is needed for Looker-based performance tracking. * drop LogSettings.VerboseDiagnostics --------- Co-authored-by: Harrison Healey <harrisonmhealey@gmail.com> Co-authored-by: Mattermost Build <build@mattermost.com> |
||
|
|
553f99612e |
MM-60441: Re-index public channels when a user joins a team (#33400)
* Index all public channels when a user joins a team * Precompute team members for indexChannelsForTeam * Refactor RequestContextWithMaster to store package This way, we can import it from both the sqlstore and the searchlayer packages. The alternative for this is duplicating the code in those two packages, but that will *not* work: The context package expects custom types for the keys stored in it, so that different packages never clash with each other when trying to register a new key. See the docs for the WithValue function: https://pkg.go.dev/context#WithValue If we try to duplicate the storeContextKey type in both the sqlstore and searchlayer packages, although they *look* the same, they are not, and HasMaster will fail to get the value of the storeContextKey(useMaster) key if it's from the other package. * Use master in call to GetTeamMembersForChannel In GetTeamMembersForChannel, use the DB from the newly passed context, which will be the receiving context everywhere except in the call done from indexChannelsForTeam, to avoid the read after write issue when saving a team member. * Fix GetPublicChannelsForTeam paging We were using the page and perPage arguments as is in the call to GetPublicChannelsForTeam, but that function expects and offset and a limit as understood by SQL. Although perPage and limit are interchangeable, offset is not equal to page, but to page * perPage. * Add a synchronous bulk indexer for Opensearch * Implement Opensearch's SyncBulkIndexChannels * Add a synchronous bulk indexer for Elasticsearch * Implement Elasticsearch's SynkBulkIndexChannels * Test SyncBulkIndexChannels * make mocks * Bulk index channels on indexChannelsForTeam * Handle error from SyncBulkIndexChannels * Fix style * Revert indexChannelWithTeamMembers refactor * Remove defensive code on sync bulk processor * Revert "Add a synchronous bulk indexer for Opensearch" This reverts commit |