* Add a new ClusterReliableFallbackLength metric
This metric tracks the length of the cluster messages that are meant to
be sent via UDP but that result in a UDP datagram larger than the
maximum length allowed.
* make mocks
* Use 8 exponential buckets: from 32KiB to 4MiB
* Add rank property field type and migrate classification field to use it
Introduces a new 'rank' property field type that behaves identically to
'select' across validation, access-control masking, options handling, and
rendering. The classification markings admin panel now creates its
template, system, and channel fields as 'rank' instead of 'select', with
a paired DB migration to flip any existing classification rows.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* Enforce rank validation on ranked property field options
For PropertyFieldTypeRank fields, every option must carry a non-negative,
unique rank. Adds the Rank field to CustomProfileAttributesSelectOption
and strips stray Rank values from options on non-rank field types so
they cannot drift into persisted attrs.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* Materialize rank options in AttributeView and signal field updates to access control
Extend the AttributeView materialized view so rank-typed property values
are exposed as {"name", "rank"} objects, enabling downstream ABAC SQL and
CEL machinery to compare a user's rank against a named option without
baking rank integers into policy expressions.
Add OnPropertyFieldOptionsChanged on the access control service interface
and call it after every property field update so the service can drop any
per-field metadata it caches (such as the rank-by-name map) and invalidate
compiled-policy cache entries that reference the field.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Show rank-and-lower options/values for shared_only rank fields
shared_only masking on a rank field previously required an exact option
match, the same as select/multiselect. For rank fields the intended
semantics are clearance-style: a caller sees every option and every
target value at or below their own rank.
filterSharedOnlyFieldOptions and filterSharedOnlyValue now branch rank
fields to filterSharedOnlyRankFieldOptions / filterSharedOnlyRankValue,
which compare against the caller's rank instead of intersecting option
IDs. Select, multiselect, and scalar (text/date/user) masking are
unchanged. A caller with no value of their own has no rank and sees
nothing; the source plugin still sees everything.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Render rank property field type like select in webapp
Treat the new 'rank' custom profile attribute type as a single-select
everywhere it is editable, and make it fully usable from the system
console:
- user_settings/general: render rank fields as a single-select dropdown
(resolving option IDs to names) instead of a free-text input.
- system_user_detail: resolve rank option names and render the value
input as a native single-select.
- user_properties_type_menu: add a selectable "Rank" field type
(reusing the Select icon) so existing rank fields display correctly
and admins can create new rank fields from the UI.
- en.json: add the "Rank" type label string.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Make rank cache invalidation cluster-aware and cover field deletion
OnPropertyFieldOptionsChanged previously dropped the access control
service's per-field rank cache only on the node handling the property
field update, so peer nodes kept serving stale name->rank maps until
restart. It was also wired into the update path only, leaving deleted
rank fields cached indefinitely.
Call OnPropertyFieldOptionsChanged from DeletePropertyField so a removed
rank field's cached options are dropped too. The access control service
now broadcasts the invalidation cluster-wide (see companion enterprise
change).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Backfill option ranks when migrating classification fields to rank
The select->rank conversion previously only flipped the field type. On
upgrades the existing classification options carry no rank (master's
select option struct has no rank field, so the UI-sent rank was dropped
on save), which left an invalid rank field: the matview projected null
ranks, shared_only masking hid every value, and the validation hook
rejected any later edit.
Backfill a rank onto each option from its 1-based array position. The
classification UI keeps levels in severity order and rewrites the full
options array on every save, so position is the authoritative ordering;
1-based matches both the UI's `opt.rank ?? (i+1)` fallback and the
presets' ranks, so a configured preset is still recognized after upgrade.
The flip and backfill share one migration (one transaction), so the field
is never observable in the invalid (rank, null-rank) state, and the down
migration strips the rank key to restore the prior select shape.
Only the three known classification fields with a non-empty options array
are touched; the updates match at most three rows and add no meaningful
lock footprint.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Add System Console UI for ranked property fields
Implements the admin UI for the ranked property field type across three
System Console surfaces:
- User Attributes: the "Ranked" attribute type, numbered value chips in
ascending rank order, a per-chip popover (rename / change rank / remove),
and an "Edit ranking" modal with drag-reorder, arrow steppers, numeric
rank inputs, auto-assigned next rank, and inline duplicate rejection.
- Membership Policy editor: ranked comparison operators (is exactly,
is not, is at least, is greater than, is at most, is less than) shown for
ranked attributes in place of the standard set, with CEL build/parse and
default-operator wiring.
- User detail: a ranked-value picker rendering options highest-rank-first
with numbered badges and a checkmark on the assigned value.
Adds Playwright e2e coverage for all three surfaces (plus page-object
helpers) and webapp unit tests for the operator wiring and rank utilities.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Fix enterprise CI: regenerate access-control mocks and add rank i18n key
The enterprise CI lane fails on two pre-existing backend gaps on this
branch:
- go vet (check-style): OnPropertyFieldOptionsChanged was added to
PolicyAdministrationPointInterface but the generated einterfaces mocks
were never regenerated, so AccessControlServiceInterface /
PolicyAdministrationPointInterface mocks no longer satisfy the
interface (access_control_test.go, access_control_masking_test.go).
Add the missing method to both mocks.
- Check i18n: enterprise access_control/administration.go references the
app.pap.rehydrate_rank.app_error key, which was never added to
server/i18n/en.json. Add it so `make i18n-extract` produces a clean
diff.
Verified: both affected test packages compile, `make i18n-extract`
yields no diff, and `make i18n-check` passes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Retrigger enterprise CI
Pick up enterprise fix 15914c11 (require.NoError in rank cel_utils tests)
in the combined Enterprise CI/tests lane, which pins the enterprise SHA at
mattermost-side dispatch time.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Retrigger enterprise CI
Pick up enterprise fix 48db14db (golangci-lint findings in rank
access-control code) in the combined Enterprise CI/tests lane.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Clamp shared_only rank values to the caller's rank instead of hiding
For a shared_only rank field, filterSharedOnlyRankValue previously hid a
target's value entirely when the target outranked the caller, returning
nil. That answered "is the target at or below me?" but not the question
the field exists for: "what can we talk about, and at what level?" A
higher-ranked target simply disappeared.
Now the value is clamped to the highest rank the caller shares with the
target — the target's own value when it is at or below the caller's rank,
otherwise the option at the caller's own rank. The caller always learns
their shared ceiling and never sees a rank above their own. Ranks are
unique per field, so the clamp target is unambiguous.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Renumber rank migrations to 194-197 (after property_groups 193)
000193_add_property_groups_schema_version was already on master, so
our rank migrations should follow it: 194-197.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Fix e2e classification helper field names after rename migration
Migration 000196 renamed channel_classification → classification and
system_classification → classification. Update the e2e helpers to
create and clean up fields with the new canonical names so the frontend
(which looks for 'classification') can find them.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Remove unused i18n key and fix e2e prettier formatting
app.pap.save_policy.advanced_expression_blocked was added to en.json
but never used as an AppError key in Go code — remove it so i18n-check
passes. Also fix prettier formatting in ranked_operators.spec.ts.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Fix 000194 down migration: drop AttributeView before enum rebuild
TestUpAndDownMigrations/Should_be_reversible_for_postgres failed because
the down migration rebuilds the property_field_type enum (Postgres can't
drop an enum value in place), which requires ALTER COLUMN on
PropertyFields.Type. The AttributeView materialized view reads that
column, so Postgres rejects the alter: "cannot alter type of a column
used by a view or rule".
The up path is unaffected because it uses ADD VALUE (no column rewrite);
only the down rebuild trips the dependency. Mirror the canonical
drop-view / alter-column / recreate-view pattern: drop AttributeView,
rebuild the enum, then recreate the no-rank view (the same definition
000197's down restores, which 000177's down later replaces).
Verified end-to-end against PostgreSQL 16: full up-then-down sequence
now passes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Add unit tests for rank validation and field-change signalling
Fills two gaps flagged by PR test analysis:
- access_control_attribute_validation_test.go: rank option validation
(valid ranks persist; missing/negative/duplicate rank rejected; zero
rank allowed; non-rank fields strip stray rank values).
- property_field_test.go: UpdatePropertyFields/DeletePropertyField
signal the access control service via OnPropertyFieldOptionsChanged
for each affected field, and stay nil-safe when no AC service exists.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Add api4 tests for rank-field permission branching
Fills the API gap flagged by PR test analysis: rank fields participate
in the isOptionsOnly permission branch alongside select/multiselect.
- properties_test.go: an options-only PATCH on a rank field uses the
narrower manage-options permission (member succeeds), while a
structural PATCH (name change) requires the full edit-field
permission (member forbidden, admin succeeds).
- custom_profile_attributes_test.go: an options-only PATCH on a rank
CPA field routes through the options path and round-trips ranks.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Add mmctl unit tests for rank attribute resolution
Fills the mmctl gap flagged by PR test analysis. Both resolution
directions for rank-typed attributes were untested:
- user_attributes_test.go (TestResolveDisplayValue): a stored rank
option ID resolves to its option name for display; unknown IDs and
option-less fields fall back to the raw value.
- user_attributes_value_test.go (TestResolveOptionNamesToIDs): setting
a rank value by option name resolves to the option ID; already-an-ID
and unknown names pass through.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Refine ranked attribute editing UI
Edit ranking modal:
- Derive ranks from row position so reordering or removing a value always
keeps them contiguous (1..N) with no gaps or duplicates
- Drop the arrow steppers in favor of drag-only reordering; render the drag
clone through a body portal so it isn't offset by the modal dialog transform
- Show values as read-only chips (lowest-first) with Lowest/Highest labels
- "Add value" toggles into an inline row (fake handle + borderless field);
Enter commits the value instead of closing the modal
- Title shows the field name plus "Ranked attribute"
Inline rank values:
- Rank badge fills the left of the chip; add an inline remove (X) that reuses
react-select's CrossIcon so it matches select fields
Also give the rank attribute type its own SortAscendingIcon and trim a couple
of stale comments.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Fix SCSS property order lint errors
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Fix e2e rank modal tests to match drag-and-drop UX
The ranked schema modal uses position-based ranks (drag to reorder) — there
are no editable numeric rank inputs per row and no per-row name inputs. The
two failing tests were written against an earlier design that had those inputs:
- "rejects a duplicate rank inline" expected .ranked-schema-modal__rank-input
and .ranked-schema-modal__error (neither exists). Replaced with a test that
covers the actual duplicate-label guard on the add-value inline input.
- "adds a value via the Edit ranking modal" expected .ranked-schema-modal__name-input
(doesn't exist) and that save is disabled while the add input is open (it
isn't — save is disabled only when rows is empty). Updated to use the real
add-value flow: click "Add value", fill .ranked-schema-modal__add-input, blur
to commit.
Also removes the three broken page-object helpers (rankedModalRankInputs,
rankedModalError, rankedModalNameInputs) whose CSS selectors never existed.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Enforce positive ranks and backfill them on select-to-rank conversion
Rank values must be positive integers (>= 1), matching the webapp's
isValidRank helper and rankForIndex which both require rank >= 1. The
server was accepting rank = 0 (rank < 0 check), allowing invalid data
that would cause filtering bugs: a caller with a rank-0 option would
see nothing in a shared-only field since no option would satisfy
rank <= 0. Change the guard to rank <= 0 and update the error message
and tests accordingly.
When converting a field from select/multiselect to rank via the type
dropdown, the existing options had no ranks. The server's option
validation then rejected the save (or the UI showed stale local state
appearing valid). Auto-assign contiguous 1-based ranks in array order
at the point of type change so the conversion is immediately valid and
the user does not need to open the rank-ordering modal just to commit.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Show an inline error when a ranked option is renamed to a duplicate
Renaming a ranked option to a name already used by another option was
silently ignored: the draft reverted with no feedback, leaving the user
unsure why their edit didn't stick. The add-value flow already warns on
duplicates; the rename path now matches it.
The chip popover surfaces "Values must be unique." beneath the label
input while the typed name collides, and the rename stays blocked so a
duplicate is never committed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Scroll the ranked schema list so the modal footer stays in view
A ranked attribute with many values (e.g. 30) grew the edit modal taller
than the viewport, pushing the Save/Cancel footer off the bottom of the
screen where it couldn't be reached.
Cap the value list at 50vh and scroll it internally, keeping the header
and footer anchored. The list is the droppable's own scroll container, so
react-beautiful-dnd auto-scrolls it while a row is being dragged, and a
small right padding keeps the scrollbar clear of the remove buttons.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Add a shared allow-list for option-bearing property field types
The "is this a select/multiselect/rank field?" check was open-coded as a
three-way type-negation chain in the model, both PSAv2 patch handlers, and
the access-control shared-options filter. Each copy had to be kept in sync
by hand as field types were added.
Introduce an optionFieldTypes allow-list in the model with a
PropertyFieldType.SupportsOptions() helper (mirroring the webapp's
supportsOptions) and route every call site through it, so adding a future
option-bearing type is a one-line change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Guard the rank options filter before its store lookup
filterSharedOnlyRankFieldOptions built the option-rank map and ran the
caller-rank store lookup before checking whether the field had a usable
options array at all. A field with no options (or a malformed attrs blob)
has nothing to filter, yet still paid for a database query.
Move the cheap nil/shape guards to the top so an optionless field returns
immediately, and reuse the extracted options slice in the filter loop
instead of pulling it out of attrs a second time.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Memoize the ranked attribute editors' event handlers
The rank schema modal and the inline rank-values cell recreated every
handler (drag-end, move, remove, confirm, rename, add) on each render.
These are passed down to the modal footer, the drag-and-drop context, and
per-chip popovers, so the fresh identities defeated memoization downstream.
Wrap them in useCallback with explicit dependencies. No behavior change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Add tests for the classification select-to-rank migration
The 000195 conversion is non-trivial: it flips specific (Name, ObjectType)
fields in the access_control group from select to rank, then backfills a
1-based rank onto each option from its array position, guarding against
empty or absent options arrays.
Cover the behavior end to end against a real Postgres schema: the type
flip with position-derived ranks in order, the empty- and absent-options
guards (type flips without fabricating options), the name/object-type/group
mismatches that must stay select, and the down round-trip that strips the
ranks and reverts the type.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Gate the rank property field type behind a feature flag
Add a PropertyFieldRank feature flag (off by default) that gates the
"rank" custom profile attribute type.
The enforcement is a single app-layer gate, rankPropertyFieldGate, shared
by both CreatePropertyField (blocks creating a rank field) and
UpdatePropertyFields (blocks converting an existing field to rank). When
the flag is off it returns app.property_field.rank_disabled.app_error.
The admin console CPA type menu hides the rank option unless the flag is
on, read via useGetFeatureFlagValue.
Existing app/api4 tests that exercise rank fields now enable the flag in
their setup so they continue to pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Scope the rank feature-flag gate to user-object fields
The PropertyFieldRank gate rejected any rank-typed field create/update
when the flag was off. But classification markings now uses the rank
type internally (template/system/channel object types) and ships GA
behind the separate ClassificationMarkings flag, which is on by default
while PropertyFieldRank is off by default. The migration also converts
existing classification fields to rank unconditionally. Together this
broke the classification admin panel in the default configuration: both
creating new classification fields and editing existing (migrated) ones
returned app.property_field.rank_disabled.app_error.
Scope the gate to ObjectType == user, which is the only origin of the
user-facing rank CPA type (createCPAField forces ObjectType=user). Rank
fields on other object types are exempt, so classification keeps working
regardless of the flag while the user-facing CPA rank type stays gated.
Add regression cases proving a non-user (classification-style) rank
field is creatable and convertible-to with the flag off, and enable
PropertyFieldRank in the e2e default config so the user-facing rank
specs pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Update webapp/channels/src/components/admin_console/system_properties/rank_badge.scss
Co-authored-by: abhijit-singh <abhijitsingh0702@gmail.com>
* Enable the rank feature flag in the e2e CI server env
The new rank e2e tests failed in playwright-full because the CI docker
server gets feature flags from MM_FEATUREFLAGS_* env vars (config patches
don't stick without a SplitKey), and PropertyFieldRank was only set in the
local default_config.ts. Add the matching env var so the flag is on in CI.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Fix stale migration-number references in classification rank down migrations
The 000195 and 000196 down migrations carried comment references to their
development-era numbers (000191/000192). Correct them to the current numbering
(000195 reverses itself; 000196's down restores distinct names before 000195's
down matches on them). Addresses review feedback from @mgdelacroix.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Use PropertyFieldType.SupportsOptions() for option-field type checks
Replace the repeated `Select || Multiselect || Rank` comparisons with the
existing SupportsOptions() helper across the access-control masking, validation,
and option-filtering paths, and in mmctl value resolution. Behavior is unchanged
(SupportsOptions covers exactly those three types). Addresses review feedback
from @mgdelacroix.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* webapp: use supportsOptions() helper for option-field type checks
Replace the repeated `type === 'select' || 'multiselect' || 'rank'` checks with
the existing supportsOptions() helper from @mattermost/types/properties, matching
the server-side SupportsOptions() usage. Addresses review feedback from @mgdelacroix.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Include rank in extractOptionIDsFromValue error message
The function handles select, multiselect and rank, but the error message only
listed select and multiselect. Addresses review feedback from @mgdelacroix.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Validate option rank in CustomProfileAttributesSelectOption.IsValid
When an option carries a rank, enforce that it is a positive integer at the
model layer, mirroring the field-level option validation. Rank stays optional
so select/multiselect options (which carry none) remain valid. Addresses review
feedback from @mgdelacroix.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Invalidate access-control cache before broadcasting field update
In UpdatePropertyFields, move the OnPropertyFieldOptionsChanged notification
above the websocket broadcast so a client reacting to the update event never
re-reads stale cached field metadata. This matches the ordering already used in
DeletePropertyField. Addresses review feedback from @mgdelacroix.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Assert stored options in options-only patch permission tests
The select/multiselect/rank options-only update tests only checked that the
request succeeded; also assert the returned field's options (id, name, and rank
for the rank case) so the tests actually verify the change was persisted.
Addresses review feedback from @mgdelacroix.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Add FeatureFlagPropertyFieldRank to webapp ClientConfig type
The server-side PropertyFieldRank feature flag and user_properties_type_menu.test.tsx
reference FeatureFlagPropertyFieldRank, but it was never added to the webapp
ClientConfig FeatureFlags type, leaving tsc -b red. Add the missing field.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Tighten the ranked option popover spacing
Drop MUI's default 8px MuiList top/bottom padding on the per-option popover and
reduce the label input's bottom padding to 4px, so the popover reads tighter
without jamming the label input against the top edge. Addresses design feedback
from @abhijit-singh.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Add header and footer dividers to the Edit ranking modal
Enable GenericModal's bodyDivider and footerDivider so the ranked schema modal
shows a divider under the header and above the Save/Cancel footer, which also
delineates the scroll area when the value list is long. Addresses design
feedback from @abhijit-singh.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Match ranked 'Add values…' affordance to select/multiselect
The ranked values cell now mirrors the select/multiselect CreatableSelect
cell for consistency across field types:
- Tab commits a pending value (keeping focus in the input for the next
one) when it's non-empty and not a duplicate, matching the select cell;
a blank/duplicate input lets Tab move focus away normally.
- The empty-state placeholder uses the shared 'Add values… (required)'
text and matches react-select's placeholder color (full-opacity
neutral50) and 10px content inset, so size, color, and left alignment
line up with the select cell.
- The placeholder is hidden once values exist, mirroring react-select.
- The chips well gets the same hover/focus background tint and text
cursor as the select control.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Auto-size the ranked 'Add values…' input like react-select
The bare <input> scrolled its overflow in a fixed-width box, while the
select/multiselect cell grows its input and wraps to a new row as you
type. Replicate react-select's sizer technique: wrap the input in an
inline-grid whose hidden ::after mirrors the live text (content:
attr(data-value), white-space: pre, font: inherit) and sizes the grid
column the input fills. The input now grows with its content and, as a
flex child of the values well, wraps to a new row when it no longer fits.
Both the input and the sizer use 'font: inherit' (matching react-select)
so the measured and rendered text line up and the font matches the select
cell. Since the auto-sized input no longer spans the whole cell, a
mousedown handler on the well forwards focus to the input, mirroring
react-select's clickable control.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Fix consistent-type-imports lint errors in Playwright e2e specs
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Enable PropertyFieldRank feature flag in rank e2e test setup
The Rank type option in the attribute type menu is gated behind the
PropertyFieldRank feature flag. initSetup resets config, so the flag
must be explicitly re-enabled before the test exercises the UI selector.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Remove broken feature-flag patchConfig from rank e2e setup
setupTest called patchConfig({FeatureFlags: {PropertyFieldRank: 'true'}})
with the string 'true'. PropertyFieldRank is a Go bool, so the server's
patchConfig handler (which does json.Decode(&cfg) into *model.Config before
any filtering) failed to unmarshal the string into a bool and returned 400
"Invalid or missing config in request body." This hard-failed setupTest and
all four specs (CI run 27387310039: 4/4 failed at spec line 36).
The call was both broken and unnecessary:
- Without a SplitKey the config store marks FeatureFlags read-only
(PlatformService.SetupFeatureFlags -> SetReadOnlyFF(!splitConfigured)), so
no patchConfig can toggle a flag. The flag is enabled at the server level:
the MM_FEATUREFLAGS_PROPERTYFIELDRANK env var in CI (added to
e2e-tests/.ci/server.generate.sh) and the server config locally.
- With the env var in place, all four specs passed at commit 8fca57bf3b; the
later commit that added this line is what regressed them.
Remove the call and document where the flag actually comes from.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Locate ranked add-value input by class, not placeholder
The "creates a ranked attribute" spec timed out at addRankValuesToLast
waiting for getByPlaceholder('Add value…') on the inline add-value input.
Commit 3c12f30008 ("Match ranked 'Add values…' affordance to
select/multiselect") changed that input in two ways the page object never
caught up with:
- the placeholder text became 'Add values… (required)' (plural, suffixed),
so 'Add value…' no longer matches; and
- the placeholder now renders only in the empty state
(placeholder={showPlaceholder ? ... : undefined}), so it disappears after
the first value is added — a placeholder lookup can never add 3 values.
Locate the input by its stable class .user-property-rank-values__add-input
instead, which is present regardless of options count or placeholder text.
This is the real failure the reverted patchConfig change had misdiagnosed as
a feature-flag issue. The spec passed at 8fca57bf3b because that predates
3c12f30008 (placeholder was then 'Add value…' and always shown).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Empty commit to retrigger CI after enterprise branch update
Co-authored-by: Maria A Nunez <maria.nunez@mattermost.com>
* Fix migration test after renumbering 000194 to 000198
During conflict resolution, migration 000194_add_type_id_index_to_access_control_policies
was renumbered to 000198 to maintain sequential ordering after the new rank migrations.
This commit updates the corresponding test file to match:
- Renamed migration_000194_test.go to migration_000198_test.go
- Updated test function name from TestMigration000194 to TestMigration000198
- Updated migration file references to point to 000198 instead of 000194
Co-authored-by: Maria A Nunez <maria.nunez@mattermost.com>
* Renumber migrations: restore 000194 from master, move rank migrations to 000196-000199
The 000194_add_type_id_index_to_access_control_policies migration came from master and should keep its original number. Our rank migrations have been renumbered:
- 000194 -> 000196: add_rank_to_property_field_type
- 000195 -> 000197: convert_classification_fields_to_rank
- 000196 -> 000198: rename_classification_linked_fields
- 000197 -> 000199: add_rank_to_attribute_view
This leaves 000195 available for the threadmemberships_cleanup_v2 migration from master.
Co-authored-by: Maria A Nunez <maria.nunez@mattermost.com>
* Rename migration test files to match renumbered migrations
After renumbering rank migrations from 000194-000197 to 000196-000199, the test file migration_000195_test.go (which tests the classification->rank conversion) needed to be renamed to migration_000197_test.go to match the new migration number. Updated the test function name and migration file references accordingly.
Co-authored-by: Maria A Nunez <maria.nunez@mattermost.com>
* Filter policy field autocomplete to user-type fields only
Add ObjectType: user filter to GetAccessControlFieldsAutocomplete so
the attribute picker in the Membership Policies editor no longer
surfaces template, system, and channel classification-marking fields
as ghost entries alongside real user CPAs.
MM-69366
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: abhijit-singh <abhijitsingh0702@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Maria A Nunez <maria.nunez@mattermost.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
* MM-68846 - add canonical CEL AST masking walker and model resolver interface
* add canonical masking methods to PAP einterface and update mock
* Migrate app-layer masking to canonical CEL AST walker
* Remove Visual AST masking dead code and obsolete i18n key
* Update simulation masking tests to mock MaskExpressionForCaller
* reject persisted tokens and enforce merge shape match and add corresponding tests
* MM-68900 - abac masking add e2e back
* add missing config values and split the tests
* implement coderabbit feedback
* adjust timeout and disable button logic
* enhance masking logic and tests for access control policies
* refactor masking tests and database setup for improved field deletion handling
* Enhance error handling in TestMergeStoredPolicyExpressions to verify error ID and status code
* Refactor error handling in access control policy methods for clarity
* Refactor masking logic and improve clarity in access control methods
* Allow deny-all 'false' policies by dropping the redundant sentinel check in rejectMaskedTokens; add tests
* Add masking-related error messages and update test descriptions with tags
* Add error handling for uninitialized Policy Administration Point in access control
* MM-69063 - Add team ABAC model and constants foundation
* Add team ABAC store EXISTS, channel Type retrofit, policy count split, and index migration
* Add team ABAC app layer: access gate, hydrators, assign/unassign, cleanup, and GetTeamMembersToRemove store
* Enforce team membership ABAC on join and hide policy governed teams from non-qualifying users in the directory
* Add team_ids to access policy assign/unassign, expose per-team policy GET, and support abac_match_only for not_in_team user listing
* Add team ABAC client methods, websocket handler, per-team System Console policy UI, and hide policy-governed teams from non-qualifying users
* Make team ABAC mode-aware: advisory on public teams, strict on private, and surface governed private teams to qualifying users in directory listings
* Flag-gate team ABAC mutation/read APIs and fix policy-save error handling, member-removal limit, team-id validation, export, and audit cleanup
* coderabbit feedback; Broadcast team policy enforcement updates on policy create/update and activation, not only on delete
* Update team access control policy schema to allow nullable policies and enhance test cases with channel counts
* Enhance access control policy tests to include team policy search alongside channel policy search
* Add team membership access control feature flag to docker-compose generation
* Implement team access control policy checks and refactor related components
* Audit-log team ABAC policy removal on team archive and delete
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
* allow workflow_dispatch trigger for Server CI (for plugins CI)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* [MM-68402] MBE Phase 2: declare four generic plugin hooks (#36291)
* new hooks-only phase 2
* remove ChannelWillBeMoved
* remove RecapWillBeProcessed and MessageWillBeRewrittenByAI
Drop the AI/recap hooks from the new-hook surface; AI-LLM paths
remain uncovered in tech preview and are documented as residuals.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [MM-68403] MBE Phase 3: ChannelGuards primitive (storage + cache + plugin API) (#36365)
* phase 3
* phase 3: register ChannelGuard mock in test setup helper
NewChannels' startup-time call to reloadGuardCache invokes
s.ChannelGuard().GetAll(); without an expectation on the mock store,
every test that sets up the server with GetMockStoreForSetupFunctions
panics during init.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* phase 3: register ChannelGuard mock in retrylayer test
retrylayer.New walks every store getter to wrap it; without the mock
expectation on ChannelGuard, TestRetry panics during layer construction.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* use rctx properly in the store methods
* phase 3: match rctx arg in testlib ChannelGuard mock
GetAll now takes request.CTX, so the testify expectation must include
mock.Anything; otherwise the call panics under the mocked store.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* phase 3: set api.ctx in TestChannelGuardLowercaseNormalization
The test constructs PluginAPI directly without a ctx, which used to
work when App.RegisterChannelGuard built its own EmptyContext. Now
that the App methods take rctx from the caller, the nil ctx panics
inside RequestContextWithMaster.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [MM-68404] MBE Phase 4: App-layer plugin hook wiring (#36407)
* phase 4
* Fix nil rctx in TestChannelGuardLowercaseNormalization
The PluginAPI struct literal was missing ctx: rctx after a refactor
moved the rctx declaration below the struct construction, leaving
api.ctx as nil. This caused a nil pointer dereference in reloadGuardCache
when RegisterChannelGuard called store.RequestContextWithMaster(nil).
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* Remove ChannelWillBeMoved hook call from MoveChannel (phase 4)
The hook and its ID were removed from mbe-phase-2 but the call site in
MoveChannel and its i18n string were not cleaned up during the rebase.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* remove channel will be moved test
* Remove RecapWillBeProcessed and MessageWillBeRewrittenByAI hook calls (phase 4)
The hooks and their IDs were removed from mbe-phase-2 but the call sites
in ProcessRecapChannel and RewriteMessage, their i18n strings, and their
tests were not cleaned up during the rebase.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Revert channel_id plumbing on rewrite endpoint (phase 4)
The channel_id field on RewriteRequest was added in phase 4 to feed the
synthetic post passed to MessageWillBeRewrittenByAI. With that hook
removed from mbe-phase-2, channel_id has no consumer; revert the field,
the api4 validation, the app.RewriteMessage parameter, and the
corresponding webapp client + hook plumbing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [MM-68555] MBE Phase 5: Channel-guard enforcement + two-phase dispatch (#36473)
* phase 5
* Bake plugin counter-file paths into source instead of env vars
t.Setenv panics when an ancestor test calls t.Parallel, so the two
channel-guard tests broke under ENABLE_FULLY_PARALLEL_TESTS in CI.
Build each plugin source per-subtest with its temp file path embedded
as a Go literal — same pattern as TestPluginUploadsAPI.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Remove guarded helpers and tests for dropped hooks (phase 5)
The runGuardedRecapWillBeProcessed and runGuardedMessageWillBeRewrittenByAI
helpers were never wired (their app-layer call sites were already removed
in the phase-4 cleanup), and the corresponding sub-tests across panic /
allow / reject / partial plugins reference hooks that no longer exist.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [MM-68405] MBE Phase 6: fire MessagesWillBeConsumed on the edit path (#36475)
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* rebase onto master
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Add SAML connectivity status to support packet diagnostics
Co-authored-by: Ben Schumacher <hanzei@users.noreply.github.com>
* Fix SAML diagnostics tests for config validation
Co-authored-by: Ben Schumacher <hanzei@users.noreply.github.com>
* Add enterprise SAML diagnostics hook for support packet
Co-authored-by: Ben Schumacher <hanzei@users.noreply.github.com>
* Cleanup
* Fix SAML support packet tests to use enterprise mock interface
Tests were expecting the platform layer to perform HTTP metadata URL
checks directly, but that logic belongs in the enterprise SAML
diagnostic implementation. Updated tests to install a mock enterprise
interface (matching the existing pattern in the override test) instead
of relying on bare HTTP calls that only work without the enterprise
interface registered.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Simplify SamlDiagnosticInterface to return error instead of (string, string)
The status return was always either StatusOk or StatusFail, which maps
directly to nil/non-nil error. Removing the redundant status string
makes the interface idiomatic Go and lets the call site derive status
from error presence.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* lint fix
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ben Schumacher <hanzei@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* rebased all prev commits into one (see commit desc)
add UsePreferredUsername support to gitlab; tests
resort en.json
update an out of date comment
webapp i18n
simplify username logic
new arguments needed in tests
debug statements -- revert
* merge conflicts
* fix i18n
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
* Add the ability to patch channel autotranslations
* Fix lint
* Update docs
* Fix CI
* Fix CI
* Fix mmctl test
* Check whether the channel is translated for the user when checking user enabled
* Fix wrong uses of patch acrros e2e and frontend
* Fix test
* Fix wording
* Fix tests and column name
* Move group constrained test so they don't mess with the basic entities
* Fix patch sending too much information
* Add endpoint to update channel member autotranslations
* Add several improvements and remove unneeded functions
* Add user id to audit record
* Ensure autotranslation is defined
* Update texts
* Fix merge
* Add new column for channel member autotranslations (#35111)
* Minor renamings
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Ben Cooke <benkcooke@gmail.com>
* remove newsletter signup and replace with terms/privacy agreement
* removed subscribeToSecurityNewsletter, made checkbox required
* update signup test to remove newsletter and ensure the terms checkbox is required
* update unit test and e2e test to reflect changes
* fix e2e test
* Removed susbcribe-newsletter endpoint in server
* Update signup.test.tsx
* remove unused css
* remove unused css
* fixed broken tests
* fixed linter issues
* Remove redundant IntlProvider and comments
* Remove usage of test IDs from Signup tests
* Remove usage of fireEvent
* Remove usage of mountWithIntl from Signup tests
* update e2e tests
* fix playwright test
* Fix Lint in signup.ts
---------
Co-authored-by: maria.nunez <maria.nunez@mattermost.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Harrison Healey <harrisonmhealey@gmail.com>
Co-authored-by: yasserfaraazkhan <attitude3cena.yf@gmail.com>
* Add Entra ID token authentication and Intune MAM config exposure
* Add Intune MAM toggle to Mobile Security admin console
* Add IntuneSettings with the AuthService to use and its own TenantID andClientID for the Entra App registration
Include Admin console changes
switch from /oauth/entra to /oauth/intune endpoint
* openAPI documentation
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: yasser khan <attitude3cena.yf@gmail.com>
* Initial Implementation of Push Proxy Authentication
* Include Config Listener for Leader plus delete startup function as job scheduler runs on initialization
* Remove push proxy auth from local imports
* Add push proxy auth to external imports
* Add push proxy auth error messages
* Update error codes
* Fix enterprise dep definition
* make i18n-extract
* Mock System store Get
* m
* m
* m
* m
* Update serverID header
* Add install type env var to docker
* Update Push Proxy config with new options
Global, US, Germany and Japan. Previous configurations will keep working
* use model.SafeDereference
* Delete token when new push proxy URL is empty
* ServerID header only if auth token is available
---------
Co-authored-by: Daniel Schalla <daniel@mattermost.com>
Co-authored-by: Nick Misasi <nick.misasi@mattermost.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
* Standardize request.CTX parameter naming to rctx
- Migrate 886 request.CTX parameters across 147 files to use consistent 'rctx' naming
- Updated function signatures from 'c', 'ctx', and 'cancelContext' to 'rctx'
- Updated function bodies to reference the new parameter names
- Preserved underscore parameters unchanged as they are unused
- Fixed method receiver context issue in store.go
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Use request.CTX interface in batch worker
* Manual fixes
* Fix parameter naming
* Add linter check
---------
Co-authored-by: Claude <noreply@anthropic.com>
* [MM-63717] LDAP Wizard skeleton (#31029)
* add ldap_wizard component to render its admin components
* i18n
* test adjustment
* keys and props fixes
* title fix
* fix placeholders
* fix value initialization
* linting
* remove all ...props (except custom component); any->unknown
* fix i18n (temp, will be changed in later PR)
* better return; simplify function checking/calling
* [MM-64259] Sections sidebar and navigation (#31059)
* initial sections list sidebar
* sidebar highlighting and scroll on click
* some tidying up
* add custom section titles for section sidebar
* i18n
* updating border on sections
* scss style lint
* color -> border-color
* simplify activeSectionKey initialization; remove trailing newline
* add useSectionNavigation; clean up ldap_wizard and scss; PR comments
* extract section of code into renderSidebar()
---------
Co-authored-by: Asaad Mahmood <asaadmahmood@users.noreply.github.com>
* [MM-64296] Add test connection for connection settings panel (#31190)
* button -> ldap test connect api
* fix console error by sanitizing value in text component
* return detailed error as error; adjust button -> primary, flushLeft
* middle of redesigning how we do hover text, first button
* add hover text to bools and file uploads
* i18n
* add LdapSettings as api type; add new endpoint to api yaml
* allow testing without first enabling LDAP and saving config
* i18n id changes
* improve TestLdapConnection to current standards
* PR comments
* safeDereference; cleaner returns
* remove hover markdown; formatting and typing simplification
* use button for "More Info"; i18n
* finish renaming help_text_hover -> help_text_more_info
* fix error output
* only send bindpassword if it has been changed
* fix: don't send blank bindPassword when it is still *****
* merge conflict
* [MM-64480] Refactor Admin Definition (#31280)
* move ldap definition to its own file for simplicity & context
* refactor admin_definition to eliminate circular dependencies
* merge conflicts
* before: buggy userHasReadPermissinOnSomeResources; after: fix incorrect snapshot
* merge conflict: new bindPasssword definition was left behind; fixed.
* merge conflict
* [MM-63765] LDAP Wizard: User filter expandable section (#31286)
* add "more info" hover to user filter help texts; make wider
* add expandable_setting type and component
* use Dislosure show/hide pattern for accessibility
* fix tooltip scss selectors
* fix hover -> more_info; make sure translation files are correct
* use join('\n\n') instead of the eslint disable line
* Revert "use join('\n\n') instead of the eslint disable line"
This reverts commit 274667e875b34703f14fee0706cd28b0125cefc9.
* [MM-64482] LDAP Wizard - Test User filters (#31312)
* initial cut at UI and backend for test filters
* api definitions; mocks
* clean up to current standards
* [MM-64512] - Test user filters UI (#31355)
* result_count -> total_count
* json cannot marshal error, returning error as string as god intended
* render errors with icon, hover text, and better feedback texts
* gather the settings that may be in expandable sections
* remove success, use error == "" to indicate success
* [MM-64536] LDAP Wizard: Test user attributes (#31373)
* LdapFilterTestResult -> LdapDiagnosticResult; FilterName -> TestName
* implement test_attributes endpoint and limited frontend (first step)
* adding EntriesWithValue
* [MM-64550] LDAP Wizard: Test user attributes UI (#31374)
* [MM-64551] LDAP Wizard: Test group attributes (#31375)
* remove Test LDAP button (not needed); reused helptext for other btn
* implement test_group_attributes endpoint; button/client-side paths
* [MM-64552] LDAP Wizard: Test group attributes UI (#31376)
* implement Test Group Attributes button
* simplify helper functions (improves useCallback dependencies)
* show the default filter that was used on the backend in the tooltip
* show the icon when there's an error (e.g. required filter/attribute)
* fix infinite rerendering
* fix error after failed save; fix navigation unlocked after save
* empty
* Adjust message feedback given we don't test the schema anymore
* improve css; don't use inline styles
* removed unneccesary pointer indirection
* improved i18n strings and logic
* combining filters/attributes/group attributes endpoints
improve types
* improve help text for User Filter (it's tricky)
* AvailableAttrs -> AvailableAttributes
* fix for e2e tests (renamed title)
* more e2e fixes
* skip broken e2e test
---------
Co-authored-by: Asaad Mahmood <asaadmahmood@users.noreply.github.com>
* Upgrade Go to 1.24.3
Updates the following files:
- server/.go-version: 1.23.9 → 1.24.3
- server/build/Dockerfile.buildenv: golang:1.23.9-bullseye → golang:1.24.3-bullseye
- server/go.mod: go 1.23.0 → go 1.24.3, toolchain go1.23.9 → go1.24.3
- server/public/go.mod: go 1.23.0 → go 1.24.3, toolchain go1.23.9 → go1.24.3
Also fixes non-constant format string errors introduced by Go 1.24.3's stricter format string checking:
- Added response() helper function in slashcommands/util.go for simple string responses
- Removed unused responsef() function from slashcommands/util.go
- Replaced responsef() with response() for translated strings that don't need formatting
- Fixed fmt.Errorf and fmt.Fprintf calls to use proper format verbs instead of string concatenation
- Updated marketplace buildURL to handle format strings conditionally
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Update generated mocks for Go 1.24.3
Regenerated mocks using mockery v2.53.4 to ensure compatibility with Go 1.24.3.
This addresses mock generation failures that occurred with the Go upgrade.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Update to bookworm and fix non-existent sha
Signed-off-by: Stavros Foteinopoulos <stafot@gmail.com>
* fix non-constant format string
---------
Signed-off-by: Stavros Foteinopoulos <stafot@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Stavros Foteinopoulos <stafot@gmail.com>
We've recently spent some effort improving websocket reconnection logic. With this commit, I've augmented the websocket reconnect metric to include a disconnect reason. This will help us measure the impact of these changes in production.
A recent change to the enterprise cluster code introduced a change to the enterprise API interface. GetClusterInfos() can now return an error. This commit introduces code to handle that error.
* feat: Add certificate upload option for audit logging settings
* Commit current changes
* Additions
* MM-62944 Fix fileupload settings not being clickable
* Support for uploading a cert for experimental audit logging cert. Pre cloud implementation in the backend
* Forgot to add new hook
* Add support for setting custom audit log certifcates in Cloud
* Permissions
* I18n
* Change order
* Linter fixes
* Linter fixes, add openapi spec
* additions for openapi
* More openapi fixes because it won't run locally
* Undo, cursor went rogue
* newline fix
* Align types properly
* Fix i18n
* Fix i18n AGAIN
* Fix error
* Update api/v4/source/audit_logging.yaml
---------
Co-authored-by: Harrison Healey <harrisonmhealey@gmail.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
We add a new config setting to allow the admin to set a fixed
list of userIDs to track for all client side webapp metrics.
This gives the admin to get a deeper look at how the application
is behaving for a single user.
A new section in the system console is also added for the user
to edit this setting from the UI.
https://mattermost.atlassian.net/browse/MM-61888
```release-note
A new config setting MetricsSettings.ClientSideUserIds is added
where you can set the user ids you want to track for client side webapp
metrics.
```
* fix lint errors
```release-note
NONE
```
* fixing tests
```release-note
NONE
```
We do a cluster request to get the active and dead queues
from other nodes in the cluster to sync any missing
information.
We check the dead queue in the other nodes to see
if there's been any message loss or not. Accordingly,
we send just the active queue or both active and dead queues.
There's still an edge case that is left out where
a client could have potentially connected and reconnected
to multiple nodes leaving multiple active queues
in multiple nodes. We don't handle this scenario
because then potentially we need to create
a slice of sendQueueSize * number_of_nodes. And then
this can happen again, leading to an infinite increase
in sendQueueSize.
We leave this edge-case to Redis, acknowledging
a limitation in our architecture.
In this PR, when there's no message loss, we just
take the active queue from the last node it connected
to.
And if there's message loss where the client's
seqNum is within the last node's dead queue, we also
handle that.
But if there's severe message loss where the client's
seqNum falls within the dead queue of another node, then
we just send the data from that node to reconstruct the
data as much as possible. It could be possible to set
a new connection ID in this case, but this involves
more data transfer always from all nodes and recomputing
the state in the requestor node.
https://mattermost.atlassian.net/browse/MM-61904
```release-note
NONE
```
Co-authored-by: Mattermost Build <build@mattermost.com>
* MM-61865: Add mobile client content load network metrics
* added new common label
* renaming from MobileClientContentLoad to MobileClientNetworkRequests
* content_load_group => network_request_group
* refactor more NetworkRequest-* changes
* replace contentLoadGroup to networkRequestGroup
* new metrics elapsedTime
* Refactor urlCount to totalRequests
* add averageSpeed metric
* replace contentLoadGroup with networkRequestGroup
* use h.Labels vs commonLabels for network_request_group
* add agent
* add effective latency metrics
* add total parallel requests & total sequential
* mocks generated by mockery
* did a bit of cleanup and sorting
* formatting
* updated the AcceptedNetworkRequestGroups
* cleanup and sorting
* [MM-59089] Add a compliance export constant (#27919)
* add a useful constant
* i18n
* another constant
* another i18n
* [MM-60422] Add GetChannelsWithActivityDuring (#28301)
* modify GetUsersInChannelDuring to accept a slice of channelIds
* add GetChannelsWithActivityDuring
* add compliance export progress message; remove unused custom status
* linting
* tests running too fast
* add batch size config settings
* add store tests
* linting
* empty commit
* i18n changes
* fix i18n ordering
* MM-60570 - Server-side changes consolidating the export CLI with server/ent code (#28640)
* add an i18n field; add the CLI's export directory
* int64 -> int
* Add UntilUpdateAt for MessageExport and AnalyticsPostCount
to merge
* remove now-unused i18n strings
* add TranslationsPreInitFromBuffer to allow CLI to use i18n
* use GetBuilder to simplify; rename TranslationsPreInitFromFileBytes
* [MM-59089] Improve compliance export timings (#1733 - Enterprise repo)
* MM-60422 - Performance and logic fixes for Compliance Exports (#1757 - Enterprise repo)
* MM-60570 - Enterprise-side changes consolidating the export CLI with server/ent code (#1769 - Enterprise repo)
* merge conflicts; missed file from ent branch
* MM-61038 - Add an option to sqlstore.New (#28702)
remove useless comment
add test
add an option to sqlstore.New
* MM-60976: Remove RunExport command from Mattermost binary (#28805)
* remove RunExport command from mattermost binary
* remove the code it was calling
* fix i18n
* remove test (was only testing license, not functionality)
* empty commit
* fix flaky GetChannelsWithActivityDuring test
* MM-60063: Dedicated Export Filestore fix, redo of #1772 (enterprise) (#28803)
* redo filestore fix#1772 (enterprise repo) on top of MM-59966 feature
* add new e2e tests for export filestore
* golint
* ok, note to self: shadowing bad, actually (when there's a defer)
* empty commit
* MM-61137 - Message export: Support 7.8.11 era dbs (#28824)
* support 7.8.11 era dbs by wrapping the store using only what we need
* fix flaky GetChannelsWithActivityDuring test
* add a comment
* only need to define the MEFileInfoStore (the one that'll be overridden)
* blank commit
* MM-60974 - Message Export: Add performance metrics (#28836)
* support 7.8.11 era dbs by wrapping the store using only what we need
* fix flaky GetChannelsWithActivityDuring test
* add a comment
* only need to define the MEFileInfoStore (the one that'll be overridden)
* performance metrics
* cleanup unneeded named returns
* blank commit
* MM-60975 - Message export: Add startTime and endTime to export folder name (#28840)
* support 7.8.11 era dbs by wrapping the store using only what we need
* fix flaky GetChannelsWithActivityDuring test
* add a comment
* only need to define the MEFileInfoStore (the one that'll be overridden)
* performance metrics
* output startTime and endTime in export folder
* empty commit
* merge conflict
* MM-60978 - Message export: Improve xml fields; fix delete semantics (#28873)
* support 7.8.11 era dbs by wrapping the store using only what we need
* fix flaky GetChannelsWithActivityDuring test
* add a comment
* only need to define the MEFileInfoStore (the one that'll be overridden)
* performance metrics
* output startTime and endTime in export folder
* empty commit
* add xml fields, omit when empty, tests
* fix delete semantics; test (and test for update semantics)
* clarify comments
* simplify edited post detection, now there's no edge case.
* add some spacing to help fast running tests
* merge conflicts/updates needed for new deleted post semantics
* linting; fixing tests from upstream merge
* use SafeDereference
* linting
* stronger typing; better wrapped errors; better formatting
* blank commit
* goimports formatting
* fix merge mistake
* minor fixes due to changes in master
* MM-61755 - Simplifying and Support reporting to the db from the CLI (#29281)
* finally clean up JobData struct and stringMap; prep for CLI using db
* and now simplify using StringMapToJobDataWithZeroValues
* remove unused fn
* create JobDataExported; clean up errors
* MM-60176 - Message Export: Global relay cleanup (#29168)
* move global relay logic into global_relay_export
* blank commit
* blank commit
* improve errors
* MM-60693 - Refactor CSV to use same codepath as Actiance (#29191)
* move global relay logic into global_relay_export
* blank commit
* refactor (and simplify) ExportParams into shared
* blank commit
* remove unused fn
* csv now uses pre-calculated joins/leaves like actiance
* improve errors
* remove nil post check; remove ignoredPosts metric
* remove unneeded copy
* MM-61696 - Refactor GlobalRelay to use same codepath as Actiance (#29225)
* move global relay logic into global_relay_export
* blank commit
* refactor (and simplify) ExportParams into shared
* blank commit
* remove unused fn
* csv now uses pre-calculated joins/leaves like actiance
* remove newly unneeded function and its test. goodbye.
* refactor GetPostAttachments for csv + global relay to share
* refactor global_relay_export and fix tests (no changes to output)
* improve errors
* remove nil post check; remove ignoredPosts metric
* remove unneeded copy
* remove unneeded nil check
* PR comments
* MM-61715 - Generalize e2e to all export types 🤖 (#29369)
* move global relay logic into global_relay_export
* blank commit
* refactor (and simplify) ExportParams into shared
* blank commit
* remove unused fn
* csv now uses pre-calculated joins/leaves like actiance
* remove newly unneeded function and its test. goodbye.
* refactor GetPostAttachments for csv + global relay to share
* refactor global_relay_export and fix tests (no changes to output)
* improve errors
* remove nil post check; remove ignoredPosts metric
* remove unneeded copy
* remove unneeded nil check
* PR comments
* refactor isDeletedMsg for all export types
* fix start and endtime, nasty csv createAt bug; bring closer to Actiance
* align unit tests with new logic (e.g. starttime / endtime)
* refactor a TimestampConvert fn for code + tests
* bug: pass templates to global relay (hurray for e2e tests, otherwise...)
* add global relay zip to allowed list (only for tests)
* test helpers
* new templates for e2e tests
* e2e tests... phew.
* linting
* merge conflicts
* unexport PostToRow; add test helper marker
* cleanup, shortening, thanks to PR comments
* MM-61972 - Generalize export data path - Actiance (#29399)
* extract and generalize the export data generation functions
* finish moving test (bc of previous extraction)
* lift a function from common -> shared (to break an import cycle)
* actiance now takes general export data, processes it into actiance data
* bring tests in line with correct sorting rules (upadateAt, messageId)
* fixups, PR comments
* turn strings.Repeat into a more descriptive const
amended: one letter fix; bad rebase
* MM-62009 - e2e clock heisenbug (#29434)
* consolidate assertions; output debuggable diffs (keeping for future)
* refactor test output generator to generators file
* waitUntilZeroPosts + pass through until to job = fix all clock issues
* simplify messages to model.NewId(); remove unneeded waitUntilZeroPosts
* model.NewId() -> storetest.NewTestID()
* MM-61980 - Generalize export data path - CSV (#29482)
* simple refactoring
* increase sleep times for (very) rare test failures
* add extra information to the generic export for CSV
* adj Actiance to handle new generic export (no difference in its output)
* no longer need mergePosts (yay), move getJoinLeavePosts for everyone
* adjust tests for new csv semantics (detailed in summary)
* and need to add the new exported data to the export_data_tests
* rearrange csv writing to happen after data export (more logical)
* linting
* remove debug statements
* figured out what was wrong with global relay e2e test 3; solid now
* PR comments
* MM-61718 - Generalize export data path - Global Relay (#29508)
* move global relay over to using the generalized export data
* performance pass -- not much can be done
* Update server/enterprise/message_export/global_relay_export/global_relay_export.go
Co-authored-by: Claudio Costa <cstcld91@gmail.com>
---------
Co-authored-by: Claudio Costa <cstcld91@gmail.com>
* MM-62058 - Align CSV with Actiance (#29551)
* refactoring actiance files and var names for clarity
* bug found in exported attachments (we used to miss some start/ends)
* changes needed for actiance due to new generic exports
* bringing CSV up to actiance standards
* fixing global relay b/c of new semantics (adding a note on an edge case)
* aligning e2e tests, adding comments to clarify what is expected/tested
* necessary changes; 1 more test for added functionality (ignoreDeleted)
* comment style
* MM-62059 - Align Global Relay with Actiance/CSV; many fixes (#29665)
* core logic changes to general export_data and the specific export paths
* unit tests and e2e tests, covering all new edge cases and all logic
* linting
* better var naming, const value, and cleaning up functions calls
* MM-62436 - Temporarily skip cypress tests that require download link (#29772)
---------
Co-authored-by: Claudio Costa <cstcld91@gmail.com>
We create a custom histogram metric that logs the userID
when the observed value is greater or equal to the last bucket value.
This allows us to start tracking the slowest users of a system
while at the same time not polluting the Prometheus metrics
by storing a userID for every observation.
https://mattermost.atlassian.net/browse/MM-61887
```release-note
NONE
```
Page load is one of the metrics that we track and present
to MLT. However, in its current form, it is not very
actionable because it also contains the network latency.
We split the whole metric into these parts:
startTime
|
responseStart = TTFB
|
responseEnd = TTLB
|
domInteractive = Start of processing phase
|
loadEventEnd = Load complete
This gives us better visibility into exactly
which phase in the load process is slow.
I have experimented with other metrics like
- domContentLoadedEventStart
- domContentLoadedEventEnd
- domComplete
and observed that they do not have sufficient
gaps in the timespan to have any relevance.
Additionally, I have moved TTFB from being a
web vitals metric to being tracked from the performance
metrics to remain consistent with the other navigation
metrics measured.
Lastly, I took this chance to improve some of the
validation errors that we threw to include more
context into the input that was passed and why
does it fail.
This also meant that I had to change the tests
to check for error strings rather than direct
errors which is a bad thing, but I don't think
it's worth the effort trying to have named error
variables for all of them.
https://mattermost.atlassian.net/browse/MM-61886
```release-note
NONE
```
* Add metrics for mobile versions snapshots
* Add notifications disabled and fix lint
* Address feedback
* Verify all references to JobTypeActiveUsers
* Fix typos
* Improve platform values
* Add test and MySQL support
Jesse Hallam