mirror of
https://github.com/mattermost/mattermost.git
synced 2026-08-29 02:28:16 +08:00
* E2E/Playwright: Add testcontainers to playwright-lib (#37570) * add testcontainers to playwright-lib * include retry mechanism for transient failures * - Introduced a new command `testcontainers:up` in package.json to run Playwright tests with Testcontainers. - Created a standalone Playwright configuration file `playwright.testcontainers-up.config.ts` for managing Testcontainers. - Added a no-op test `testcontainers_up.spec.ts` to ensure the Testcontainers stack is up during the test run. - Implemented a global setup script `testcontainers_up_global_setup.ts` to start and stop the Testcontainers stack. - Updated dependencies in package.json, including adding `chalk` for logging. * fix package-lock * fix tsc and restore waiting for all migrations to complete --------- Co-authored-by: Mattermost Build <build@mattermost.com> * fix(e2e): remove obsolete CRT intro modal dismiss from uiClickSidebarItem The CRT tutorial modal was removed in MM-66470; dismissing #genericModalLabel races and flakes MM-T4261_3. Co-authored-by: sabril <saturninoabril@users.noreply.github.com> * fix(e2e): sync Playwright navigations to live testcontainers baseURL After restartMattermostContainer remaps the host port, relative page.goto must follow testConfig.baseURL rather than the worker-start config value. Co-authored-by: sabril <saturninoabril@users.noreply.github.com> * fix(e2e): restore eslint-disable for axe empty fixture pattern Co-authored-by: sabril <saturninoabril@users.noreply.github.com> * fix(e2e): ignore webhook assets in eslint; prettier README Co-authored-by: sabril <saturninoabril@users.noreply.github.com> * ci(e2e): fully adopt test-system-io and lighten Playwright services Port #37413: drop the dual v1/v2 dispatch path, promote the test-system-io templates to the canonical names, and remove legacy calculate-results / AWS artifact upload plumbing. For Playwright testcontainers on this release, leave optional sidecars (openldap, keycloak, elasticsearch, opensearch, minio, azurite, webhook) off by default and remove the related service-specific specs that are not part of the older suite. Co-authored-by: sabril <saturninoabril@users.noreply.github.com> * style(e2e): prettier stack.ts after optional webhook change Co-authored-by: sabril <saturninoabril@users.noreply.github.com> * test(e2e): align testcontainers feature flags with release docker-compose Mirror MM_FEATUREFLAGS_* from e2e-tests/.ci/server.generate.sh for each release, dropping flags that do not exist in that release's FeatureFlags struct (or were never enabled in the compose stack). Co-authored-by: sabril <saturninoabril@users.noreply.github.com> * e2e(playwright): omit unset Testcontainers env keys; align release docs When optional webhook/sidecars are off, .env.testcontainers no longer writes PW_*=undefined. Docs match empty DEFAULT_TESTCONTAINERS_SERVICES. Co-authored-by: sabril <saturninoabril@users.noreply.github.com> * e2e(playwright): prettier-format lib/README.md table Fixes CI npm run check (prettier) failures from the services table edit. Co-authored-by: sabril <saturninoabril@users.noreply.github.com> --------- Co-authored-by: Mattermost Build <build@mattermost.com> Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: sabril <saturninoabril@users.noreply.github.com>
This commit is contained in:
sabril
@@ -1,2 +0,0 @@
|
||||
node_modules/
|
||||
.env
|
||||
@@ -1,50 +0,0 @@
|
||||
name: Calculate Cypress Results
|
||||
description: Calculate Cypress test results with optional merge of retest results
|
||||
author: Mattermost
|
||||
|
||||
inputs:
|
||||
original-results-path:
|
||||
description: Path to the original Cypress results directory (e.g., e2e-tests/cypress/results)
|
||||
required: true
|
||||
retest-results-path:
|
||||
description: Path to the retest Cypress results directory (optional - if not provided, only calculates from original)
|
||||
required: false
|
||||
write-merged:
|
||||
description: Whether to write merged results back to the original directory (default true)
|
||||
required: false
|
||||
default: "true"
|
||||
|
||||
outputs:
|
||||
# Merge outputs
|
||||
merged:
|
||||
description: Whether merge was performed (true/false)
|
||||
|
||||
# Calculation outputs (same as calculate-cypress-test-results)
|
||||
passed:
|
||||
description: Number of passed tests
|
||||
failed:
|
||||
description: Number of failed tests
|
||||
pending:
|
||||
description: Number of pending/skipped tests
|
||||
total_specs:
|
||||
description: Total number of spec files
|
||||
commit_status_message:
|
||||
description: Message for commit status (e.g., "X failed, Y passed (Z spec files)")
|
||||
failed_specs:
|
||||
description: Comma-separated list of failed spec files (for retest)
|
||||
failed_specs_count:
|
||||
description: Number of failed spec files
|
||||
failed_tests:
|
||||
description: Markdown table rows of failed tests (for GitHub summary)
|
||||
total:
|
||||
description: Total number of tests (passed + failed)
|
||||
pass_rate:
|
||||
description: Pass rate percentage (e.g., "100.00")
|
||||
color:
|
||||
description: Color for webhook based on pass rate (green=100%, yellow=99%+, orange=98%+, red=<98%)
|
||||
test_duration:
|
||||
description: Wall-clock test duration (earliest start to latest end across all specs, formatted as "Xm Ys")
|
||||
|
||||
runs:
|
||||
using: node24
|
||||
main: dist/index.js
|
||||
File diff suppressed because one or more lines are too long
@@ -1,15 +0,0 @@
|
||||
/** @type {import('ts-jest').JestConfigWithTsJest} */
|
||||
module.exports = {
|
||||
preset: "ts-jest",
|
||||
testEnvironment: "node",
|
||||
testMatch: ["**/*.test.ts"],
|
||||
moduleFileExtensions: ["ts", "js"],
|
||||
transform: {
|
||||
"^.+\\.ts$": [
|
||||
"ts-jest",
|
||||
{
|
||||
useESM: false,
|
||||
},
|
||||
],
|
||||
},
|
||||
};
|
||||
-9136
File diff suppressed because it is too large
Load Diff
@@ -1,27 +0,0 @@
|
||||
{
|
||||
"name": "calculate-cypress-results",
|
||||
"private": true,
|
||||
"version": "0.1.0",
|
||||
"main": "dist/index.js",
|
||||
"scripts": {
|
||||
"build": "tsup",
|
||||
"prettier": "npx prettier --write \"src/**/*.ts\"",
|
||||
"local-action": "local-action . src/main.ts .env",
|
||||
"test": "jest --verbose",
|
||||
"test:watch": "jest --watch --verbose",
|
||||
"test:silent": "jest --silent",
|
||||
"tsc": "tsc -b"
|
||||
},
|
||||
"dependencies": {
|
||||
"@actions/core": "3.0.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@github/local-action": "7.0.0",
|
||||
"@types/jest": "30.0.0",
|
||||
"@types/node": "25.2.0",
|
||||
"jest": "30.2.0",
|
||||
"ts-jest": "29.4.6",
|
||||
"tsup": "8.5.1",
|
||||
"typescript": "5.9.3"
|
||||
}
|
||||
}
|
||||
@@ -1,3 +0,0 @@
|
||||
import { run } from "./main";
|
||||
|
||||
run();
|
||||
@@ -1,101 +0,0 @@
|
||||
import * as core from "@actions/core";
|
||||
import {
|
||||
loadSpecFiles,
|
||||
mergeResults,
|
||||
writeMergedResults,
|
||||
calculateResultsFromSpecs,
|
||||
} from "./merge";
|
||||
|
||||
export async function run(): Promise<void> {
|
||||
const originalPath = core.getInput("original-results-path", {
|
||||
required: true,
|
||||
});
|
||||
const retestPath = core.getInput("retest-results-path"); // Optional
|
||||
const shouldWriteMerged = core.getInput("write-merged") !== "false"; // Default true
|
||||
|
||||
core.info(`Original results: ${originalPath}`);
|
||||
core.info(`Retest results: ${retestPath || "(not provided)"}`);
|
||||
|
||||
let merged = false;
|
||||
let specs;
|
||||
|
||||
if (retestPath) {
|
||||
// Check if retest path has results
|
||||
const retestSpecs = await loadSpecFiles(retestPath);
|
||||
|
||||
if (retestSpecs.length > 0) {
|
||||
core.info(`Found ${retestSpecs.length} retest spec files`);
|
||||
|
||||
// Merge results
|
||||
core.info("Merging results...");
|
||||
const mergeResult = await mergeResults(originalPath, retestPath);
|
||||
specs = mergeResult.specs;
|
||||
merged = true;
|
||||
|
||||
core.info(`Retested specs: ${mergeResult.retestFiles.join(", ")}`);
|
||||
core.info(`Total merged specs: ${specs.length}`);
|
||||
|
||||
// Write merged results back to original directory
|
||||
if (shouldWriteMerged) {
|
||||
core.info("Writing merged results to original directory...");
|
||||
const writeResult = await writeMergedResults(
|
||||
originalPath,
|
||||
retestPath,
|
||||
);
|
||||
core.info(`Updated files: ${writeResult.updatedFiles.length}`);
|
||||
core.info(
|
||||
`Removed duplicates: ${writeResult.removedFiles.length}`,
|
||||
);
|
||||
}
|
||||
} else {
|
||||
core.warning(
|
||||
`No retest results found at ${retestPath}, using original only`,
|
||||
);
|
||||
specs = await loadSpecFiles(originalPath);
|
||||
}
|
||||
} else {
|
||||
core.info("No retest path provided, using original results only");
|
||||
specs = await loadSpecFiles(originalPath);
|
||||
}
|
||||
|
||||
core.info(`Calculating results from ${specs.length} spec files...`);
|
||||
|
||||
// Handle case where no results found
|
||||
if (specs.length === 0) {
|
||||
core.setFailed("No Cypress test results found");
|
||||
return;
|
||||
}
|
||||
|
||||
// Calculate all outputs from final results
|
||||
const calc = calculateResultsFromSpecs(specs);
|
||||
|
||||
// Log results
|
||||
core.startGroup("Final Results");
|
||||
core.info(`Passed: ${calc.passed}`);
|
||||
core.info(`Failed: ${calc.failed}`);
|
||||
core.info(`Pending: ${calc.pending}`);
|
||||
core.info(`Total: ${calc.total}`);
|
||||
core.info(`Pass Rate: ${calc.passRate}%`);
|
||||
core.info(`Color: ${calc.color}`);
|
||||
core.info(`Spec Files: ${calc.totalSpecs}`);
|
||||
core.info(`Failed Specs Count: ${calc.failedSpecsCount}`);
|
||||
core.info(`Commit Status Message: ${calc.commitStatusMessage}`);
|
||||
core.info(`Failed Specs: ${calc.failedSpecs || "none"}`);
|
||||
core.info(`Test Duration: ${calc.testDuration}`);
|
||||
core.endGroup();
|
||||
|
||||
// Set all outputs
|
||||
core.setOutput("merged", merged.toString());
|
||||
core.setOutput("passed", calc.passed);
|
||||
core.setOutput("failed", calc.failed);
|
||||
core.setOutput("pending", calc.pending);
|
||||
core.setOutput("total_specs", calc.totalSpecs);
|
||||
core.setOutput("commit_status_message", calc.commitStatusMessage);
|
||||
core.setOutput("failed_specs", calc.failedSpecs);
|
||||
core.setOutput("failed_specs_count", calc.failedSpecsCount);
|
||||
core.setOutput("failed_tests", calc.failedTests);
|
||||
core.setOutput("total", calc.total);
|
||||
core.setOutput("pass_rate", calc.passRate);
|
||||
core.setOutput("color", calc.color);
|
||||
core.setOutput("test_duration", calc.testDuration);
|
||||
}
|
||||
@@ -1,271 +0,0 @@
|
||||
import { calculateResultsFromSpecs } from "./merge";
|
||||
import type { ParsedSpecFile, MochawesomeResult } from "./types";
|
||||
|
||||
/**
|
||||
* Helper to create a mochawesome result for testing
|
||||
*/
|
||||
function createMochawesomeResult(
|
||||
specFile: string,
|
||||
tests: { title: string; state: "passed" | "failed" | "pending" }[],
|
||||
): MochawesomeResult {
|
||||
return {
|
||||
stats: {
|
||||
suites: 1,
|
||||
tests: tests.length,
|
||||
passes: tests.filter((t) => t.state === "passed").length,
|
||||
pending: tests.filter((t) => t.state === "pending").length,
|
||||
failures: tests.filter((t) => t.state === "failed").length,
|
||||
start: new Date().toISOString(),
|
||||
end: new Date().toISOString(),
|
||||
duration: 1000,
|
||||
testsRegistered: tests.length,
|
||||
passPercent: 0,
|
||||
pendingPercent: 0,
|
||||
other: 0,
|
||||
hasOther: false,
|
||||
skipped: 0,
|
||||
hasSkipped: false,
|
||||
},
|
||||
results: [
|
||||
{
|
||||
uuid: "uuid-1",
|
||||
title: specFile,
|
||||
fullFile: `/app/e2e-tests/cypress/tests/integration/${specFile}`,
|
||||
file: `tests/integration/${specFile}`,
|
||||
beforeHooks: [],
|
||||
afterHooks: [],
|
||||
tests: tests.map((t, i) => ({
|
||||
title: t.title,
|
||||
fullTitle: `${specFile} > ${t.title}`,
|
||||
timedOut: null,
|
||||
duration: 500,
|
||||
state: t.state,
|
||||
speed: "fast",
|
||||
pass: t.state === "passed",
|
||||
fail: t.state === "failed",
|
||||
pending: t.state === "pending",
|
||||
context: null,
|
||||
code: "",
|
||||
err: t.state === "failed" ? { message: "Test failed" } : {},
|
||||
uuid: `test-uuid-${i}`,
|
||||
parentUUID: "uuid-1",
|
||||
isHook: false,
|
||||
skipped: false,
|
||||
})),
|
||||
suites: [],
|
||||
passes: tests
|
||||
.filter((t) => t.state === "passed")
|
||||
.map((_, i) => `test-uuid-${i}`),
|
||||
failures: tests
|
||||
.filter((t) => t.state === "failed")
|
||||
.map((_, i) => `test-uuid-${i}`),
|
||||
pending: tests
|
||||
.filter((t) => t.state === "pending")
|
||||
.map((_, i) => `test-uuid-${i}`),
|
||||
skipped: [],
|
||||
duration: 1000,
|
||||
root: true,
|
||||
rootEmpty: false,
|
||||
_timeout: 60000,
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
function createParsedSpecFile(
|
||||
specFile: string,
|
||||
tests: { title: string; state: "passed" | "failed" | "pending" }[],
|
||||
): ParsedSpecFile {
|
||||
return {
|
||||
filePath: `/path/to/${specFile}.json`,
|
||||
specPath: `tests/integration/${specFile}`,
|
||||
result: createMochawesomeResult(specFile, tests),
|
||||
};
|
||||
}
|
||||
|
||||
describe("calculateResultsFromSpecs", () => {
|
||||
it("should calculate all outputs correctly for passing results", () => {
|
||||
const specs: ParsedSpecFile[] = [
|
||||
createParsedSpecFile("login.spec.ts", [
|
||||
{
|
||||
title: "should login with valid credentials",
|
||||
state: "passed",
|
||||
},
|
||||
]),
|
||||
createParsedSpecFile("messaging.spec.ts", [
|
||||
{ title: "should send a message", state: "passed" },
|
||||
]),
|
||||
];
|
||||
|
||||
const calc = calculateResultsFromSpecs(specs);
|
||||
|
||||
expect(calc.passed).toBe(2);
|
||||
expect(calc.failed).toBe(0);
|
||||
expect(calc.pending).toBe(0);
|
||||
expect(calc.total).toBe(2);
|
||||
expect(calc.passRate).toBe("100.00");
|
||||
expect(calc.color).toBe("#43A047"); // green
|
||||
expect(calc.totalSpecs).toBe(2);
|
||||
expect(calc.failedSpecs).toBe("");
|
||||
expect(calc.failedSpecsCount).toBe(0);
|
||||
expect(calc.commitStatusMessage).toBe("100% passed (2), 2 specs");
|
||||
});
|
||||
|
||||
it("should calculate all outputs correctly for results with failures", () => {
|
||||
const specs: ParsedSpecFile[] = [
|
||||
createParsedSpecFile("login.spec.ts", [
|
||||
{
|
||||
title: "should login with valid credentials",
|
||||
state: "passed",
|
||||
},
|
||||
]),
|
||||
createParsedSpecFile("channels.spec.ts", [
|
||||
{ title: "should create a channel", state: "failed" },
|
||||
]),
|
||||
];
|
||||
|
||||
const calc = calculateResultsFromSpecs(specs);
|
||||
|
||||
expect(calc.passed).toBe(1);
|
||||
expect(calc.failed).toBe(1);
|
||||
expect(calc.pending).toBe(0);
|
||||
expect(calc.total).toBe(2);
|
||||
expect(calc.passRate).toBe("50.00");
|
||||
expect(calc.color).toBe("#F44336"); // red
|
||||
expect(calc.totalSpecs).toBe(2);
|
||||
expect(calc.failedSpecs).toBe("tests/integration/channels.spec.ts");
|
||||
expect(calc.failedSpecsCount).toBe(1);
|
||||
expect(calc.commitStatusMessage).toBe(
|
||||
"50.0% passed (1/2), 1 failed, 2 specs",
|
||||
);
|
||||
expect(calc.failedTests).toContain("should create a channel");
|
||||
});
|
||||
|
||||
it("should handle pending tests correctly", () => {
|
||||
const specs: ParsedSpecFile[] = [
|
||||
createParsedSpecFile("login.spec.ts", [
|
||||
{ title: "should login", state: "passed" },
|
||||
{ title: "should logout", state: "pending" },
|
||||
]),
|
||||
];
|
||||
|
||||
const calc = calculateResultsFromSpecs(specs);
|
||||
|
||||
expect(calc.passed).toBe(1);
|
||||
expect(calc.failed).toBe(0);
|
||||
expect(calc.pending).toBe(1);
|
||||
expect(calc.total).toBe(1); // Total excludes pending
|
||||
expect(calc.passRate).toBe("100.00");
|
||||
});
|
||||
|
||||
it("should limit failed tests to 10 entries", () => {
|
||||
const specs: ParsedSpecFile[] = [
|
||||
createParsedSpecFile("big-test.spec.ts", [
|
||||
{ title: "test 1", state: "failed" },
|
||||
{ title: "test 2", state: "failed" },
|
||||
{ title: "test 3", state: "failed" },
|
||||
{ title: "test 4", state: "failed" },
|
||||
{ title: "test 5", state: "failed" },
|
||||
{ title: "test 6", state: "failed" },
|
||||
{ title: "test 7", state: "failed" },
|
||||
{ title: "test 8", state: "failed" },
|
||||
{ title: "test 9", state: "failed" },
|
||||
{ title: "test 10", state: "failed" },
|
||||
{ title: "test 11", state: "failed" },
|
||||
{ title: "test 12", state: "failed" },
|
||||
]),
|
||||
];
|
||||
|
||||
const calc = calculateResultsFromSpecs(specs);
|
||||
|
||||
expect(calc.failed).toBe(12);
|
||||
expect(calc.failedTests).toContain("...and 2 more failed tests");
|
||||
});
|
||||
});
|
||||
|
||||
describe("merge simulation", () => {
|
||||
it("should produce correct results when merging original with retest", () => {
|
||||
// Simulate original: 2 passed, 1 failed
|
||||
const originalSpecs: ParsedSpecFile[] = [
|
||||
createParsedSpecFile("login.spec.ts", [
|
||||
{ title: "should login", state: "passed" },
|
||||
]),
|
||||
createParsedSpecFile("messaging.spec.ts", [
|
||||
{ title: "should send message", state: "passed" },
|
||||
]),
|
||||
createParsedSpecFile("channels.spec.ts", [
|
||||
{ title: "should create channel", state: "failed" },
|
||||
]),
|
||||
];
|
||||
|
||||
// Verify original has failure
|
||||
const originalCalc = calculateResultsFromSpecs(originalSpecs);
|
||||
expect(originalCalc.passed).toBe(2);
|
||||
expect(originalCalc.failed).toBe(1);
|
||||
expect(originalCalc.passRate).toBe("66.67");
|
||||
|
||||
// Simulate retest: channels.spec.ts now passes
|
||||
const retestSpec = createParsedSpecFile("channels.spec.ts", [
|
||||
{ title: "should create channel", state: "passed" },
|
||||
]);
|
||||
|
||||
// Simulate merge: replace original channels.spec.ts with retest
|
||||
const specMap = new Map<string, ParsedSpecFile>();
|
||||
for (const spec of originalSpecs) {
|
||||
specMap.set(spec.specPath, spec);
|
||||
}
|
||||
specMap.set(retestSpec.specPath, retestSpec);
|
||||
|
||||
const mergedSpecs = Array.from(specMap.values());
|
||||
|
||||
// Calculate final results
|
||||
const finalCalc = calculateResultsFromSpecs(mergedSpecs);
|
||||
|
||||
expect(finalCalc.passed).toBe(3);
|
||||
expect(finalCalc.failed).toBe(0);
|
||||
expect(finalCalc.pending).toBe(0);
|
||||
expect(finalCalc.total).toBe(3);
|
||||
expect(finalCalc.passRate).toBe("100.00");
|
||||
expect(finalCalc.color).toBe("#43A047"); // green
|
||||
expect(finalCalc.totalSpecs).toBe(3);
|
||||
expect(finalCalc.failedSpecs).toBe("");
|
||||
expect(finalCalc.failedSpecsCount).toBe(0);
|
||||
expect(finalCalc.commitStatusMessage).toBe("100% passed (3), 3 specs");
|
||||
});
|
||||
|
||||
it("should handle case where retest still fails", () => {
|
||||
// Original: 1 passed, 1 failed
|
||||
const originalSpecs: ParsedSpecFile[] = [
|
||||
createParsedSpecFile("login.spec.ts", [
|
||||
{ title: "should login", state: "passed" },
|
||||
]),
|
||||
createParsedSpecFile("channels.spec.ts", [
|
||||
{ title: "should create channel", state: "failed" },
|
||||
]),
|
||||
];
|
||||
|
||||
// Retest: channels.spec.ts still fails
|
||||
const retestSpec = createParsedSpecFile("channels.spec.ts", [
|
||||
{ title: "should create channel", state: "failed" },
|
||||
]);
|
||||
|
||||
// Merge
|
||||
const specMap = new Map<string, ParsedSpecFile>();
|
||||
for (const spec of originalSpecs) {
|
||||
specMap.set(spec.specPath, spec);
|
||||
}
|
||||
specMap.set(retestSpec.specPath, retestSpec);
|
||||
|
||||
const mergedSpecs = Array.from(specMap.values());
|
||||
const finalCalc = calculateResultsFromSpecs(mergedSpecs);
|
||||
|
||||
expect(finalCalc.passed).toBe(1);
|
||||
expect(finalCalc.failed).toBe(1);
|
||||
expect(finalCalc.passRate).toBe("50.00");
|
||||
expect(finalCalc.color).toBe("#F44336"); // red
|
||||
expect(finalCalc.failedSpecs).toBe(
|
||||
"tests/integration/channels.spec.ts",
|
||||
);
|
||||
expect(finalCalc.failedSpecsCount).toBe(1);
|
||||
});
|
||||
});
|
||||
@@ -1,358 +0,0 @@
|
||||
import * as fs from "fs/promises";
|
||||
import * as path from "path";
|
||||
import type {
|
||||
MochawesomeResult,
|
||||
ParsedSpecFile,
|
||||
CalculationResult,
|
||||
FailedTest,
|
||||
TestItem,
|
||||
SuiteItem,
|
||||
ResultItem,
|
||||
} from "./types";
|
||||
|
||||
/**
|
||||
* Find all JSON files in a directory recursively
|
||||
*/
|
||||
async function findJsonFiles(dir: string): Promise<string[]> {
|
||||
const files: string[] = [];
|
||||
|
||||
try {
|
||||
const entries = await fs.readdir(dir, { withFileTypes: true });
|
||||
|
||||
for (const entry of entries) {
|
||||
const fullPath = path.join(dir, entry.name);
|
||||
if (entry.isDirectory()) {
|
||||
const subFiles = await findJsonFiles(fullPath);
|
||||
files.push(...subFiles);
|
||||
} else if (entry.isFile() && entry.name.endsWith(".json")) {
|
||||
files.push(fullPath);
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
// Directory doesn't exist or not accessible
|
||||
}
|
||||
|
||||
return files;
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse a mochawesome JSON file
|
||||
*/
|
||||
async function parseSpecFile(filePath: string): Promise<ParsedSpecFile | null> {
|
||||
try {
|
||||
const content = await fs.readFile(filePath, "utf8");
|
||||
const result: MochawesomeResult = JSON.parse(content);
|
||||
|
||||
// Extract spec path from results[0].file
|
||||
const specPath = result.results?.[0]?.file;
|
||||
if (!specPath) {
|
||||
return null;
|
||||
}
|
||||
|
||||
return {
|
||||
filePath,
|
||||
specPath,
|
||||
result,
|
||||
};
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract all tests from a result recursively
|
||||
*/
|
||||
function getAllTests(result: MochawesomeResult): TestItem[] {
|
||||
const tests: TestItem[] = [];
|
||||
|
||||
function extractFromSuite(suite: SuiteItem | ResultItem) {
|
||||
tests.push(...(suite.tests || []));
|
||||
for (const nestedSuite of suite.suites || []) {
|
||||
extractFromSuite(nestedSuite);
|
||||
}
|
||||
}
|
||||
|
||||
for (const resultItem of result.results || []) {
|
||||
extractFromSuite(resultItem);
|
||||
}
|
||||
|
||||
return tests;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get color based on pass rate
|
||||
*/
|
||||
function getColor(passRate: number): string {
|
||||
if (passRate === 100) {
|
||||
return "#43A047"; // green
|
||||
} else if (passRate >= 99) {
|
||||
return "#FFEB3B"; // yellow
|
||||
} else if (passRate >= 98) {
|
||||
return "#FF9800"; // orange
|
||||
} else {
|
||||
return "#F44336"; // red
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Calculate results from parsed spec files
|
||||
*/
|
||||
/**
|
||||
* Format milliseconds as "Xm Ys"
|
||||
*/
|
||||
function formatDuration(ms: number): string {
|
||||
const totalSeconds = Math.round(ms / 1000);
|
||||
const minutes = Math.floor(totalSeconds / 60);
|
||||
const seconds = totalSeconds % 60;
|
||||
return `${minutes}m ${seconds}s`;
|
||||
}
|
||||
|
||||
export function calculateResultsFromSpecs(
|
||||
specs: ParsedSpecFile[],
|
||||
): CalculationResult {
|
||||
let passed = 0;
|
||||
let failed = 0;
|
||||
let pending = 0;
|
||||
const failedSpecsSet = new Set<string>();
|
||||
const failedTestsList: FailedTest[] = [];
|
||||
|
||||
for (const spec of specs) {
|
||||
const tests = getAllTests(spec.result);
|
||||
|
||||
for (const test of tests) {
|
||||
if (test.state === "passed") {
|
||||
passed++;
|
||||
} else if (test.state === "failed") {
|
||||
failed++;
|
||||
failedSpecsSet.add(spec.specPath);
|
||||
failedTestsList.push({
|
||||
title: test.title,
|
||||
file: spec.specPath,
|
||||
});
|
||||
} else if (test.state === "pending") {
|
||||
pending++;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Compute test duration from earliest start to latest end across all specs
|
||||
let earliestStart: number | null = null;
|
||||
let latestEnd: number | null = null;
|
||||
for (const spec of specs) {
|
||||
const { start, end } = spec.result.stats;
|
||||
if (start) {
|
||||
const startMs = new Date(start).getTime();
|
||||
if (earliestStart === null || startMs < earliestStart) {
|
||||
earliestStart = startMs;
|
||||
}
|
||||
}
|
||||
if (end) {
|
||||
const endMs = new Date(end).getTime();
|
||||
if (latestEnd === null || endMs > latestEnd) {
|
||||
latestEnd = endMs;
|
||||
}
|
||||
}
|
||||
}
|
||||
const testDurationMs =
|
||||
earliestStart !== null && latestEnd !== null
|
||||
? latestEnd - earliestStart
|
||||
: 0;
|
||||
const testDuration = formatDuration(testDurationMs);
|
||||
|
||||
const totalSpecs = specs.length;
|
||||
const failedSpecs = Array.from(failedSpecsSet).join(",");
|
||||
const failedSpecsCount = failedSpecsSet.size;
|
||||
|
||||
// Build failed tests markdown table (limit to 10)
|
||||
let failedTests = "";
|
||||
const uniqueFailedTests = failedTestsList.filter(
|
||||
(test, index, self) =>
|
||||
index ===
|
||||
self.findIndex(
|
||||
(t) => t.title === test.title && t.file === test.file,
|
||||
),
|
||||
);
|
||||
|
||||
if (uniqueFailedTests.length > 0) {
|
||||
const limitedTests = uniqueFailedTests.slice(0, 10);
|
||||
failedTests = limitedTests
|
||||
.map((t) => {
|
||||
const escapedTitle = t.title
|
||||
.replace(/`/g, "\\`")
|
||||
.replace(/\|/g, "\\|");
|
||||
return `| ${escapedTitle} | ${t.file} |`;
|
||||
})
|
||||
.join("\n");
|
||||
|
||||
if (uniqueFailedTests.length > 10) {
|
||||
const remaining = uniqueFailedTests.length - 10;
|
||||
failedTests += `\n| _...and ${remaining} more failed tests_ | |`;
|
||||
}
|
||||
} else if (failed > 0) {
|
||||
failedTests = "| Unable to parse failed tests | - |";
|
||||
}
|
||||
|
||||
// Calculate totals and pass rate
|
||||
// Pass rate = passed / (passed + failed), excluding pending
|
||||
const total = passed + failed;
|
||||
const passRate = total > 0 ? ((passed * 100) / total).toFixed(2) : "0.00";
|
||||
const color = getColor(parseFloat(passRate));
|
||||
|
||||
// Build commit status message
|
||||
const rate = total > 0 ? (passed * 100) / total : 0;
|
||||
const rateStr = rate === 100 ? "100%" : `${rate.toFixed(1)}%`;
|
||||
const specSuffix = totalSpecs > 0 ? `, ${totalSpecs} specs` : "";
|
||||
const commitStatusMessage =
|
||||
rate === 100
|
||||
? `${rateStr} passed (${passed})${specSuffix}`
|
||||
: `${rateStr} passed (${passed}/${total}), ${failed} failed${specSuffix}`;
|
||||
|
||||
return {
|
||||
passed,
|
||||
failed,
|
||||
pending,
|
||||
totalSpecs,
|
||||
commitStatusMessage,
|
||||
failedSpecs,
|
||||
failedSpecsCount,
|
||||
failedTests,
|
||||
total,
|
||||
passRate,
|
||||
color,
|
||||
testDuration,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Load all spec files from a mochawesome results directory
|
||||
*/
|
||||
export async function loadSpecFiles(
|
||||
resultsPath: string,
|
||||
): Promise<ParsedSpecFile[]> {
|
||||
// Mochawesome results are at: results/mochawesome-report/json/tests/
|
||||
const mochawesomeDir = path.join(
|
||||
resultsPath,
|
||||
"mochawesome-report",
|
||||
"json",
|
||||
"tests",
|
||||
);
|
||||
|
||||
const jsonFiles = await findJsonFiles(mochawesomeDir);
|
||||
const specs: ParsedSpecFile[] = [];
|
||||
|
||||
for (const file of jsonFiles) {
|
||||
const parsed = await parseSpecFile(file);
|
||||
if (parsed) {
|
||||
specs.push(parsed);
|
||||
}
|
||||
}
|
||||
|
||||
return specs;
|
||||
}
|
||||
|
||||
/**
|
||||
* Merge original and retest results
|
||||
* - For each spec in retest, replace the matching spec in original
|
||||
* - Keep original specs that are not in retest
|
||||
*/
|
||||
export async function mergeResults(
|
||||
originalPath: string,
|
||||
retestPath: string,
|
||||
): Promise<{
|
||||
specs: ParsedSpecFile[];
|
||||
retestFiles: string[];
|
||||
mergedCount: number;
|
||||
}> {
|
||||
const originalSpecs = await loadSpecFiles(originalPath);
|
||||
const retestSpecs = await loadSpecFiles(retestPath);
|
||||
|
||||
// Build a map of original specs by spec path
|
||||
const specMap = new Map<string, ParsedSpecFile>();
|
||||
for (const spec of originalSpecs) {
|
||||
specMap.set(spec.specPath, spec);
|
||||
}
|
||||
|
||||
// Replace with retest results
|
||||
const retestFiles: string[] = [];
|
||||
for (const retestSpec of retestSpecs) {
|
||||
specMap.set(retestSpec.specPath, retestSpec);
|
||||
retestFiles.push(retestSpec.specPath);
|
||||
}
|
||||
|
||||
return {
|
||||
specs: Array.from(specMap.values()),
|
||||
retestFiles,
|
||||
mergedCount: retestSpecs.length,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Write merged results back to the original directory
|
||||
* This updates the original JSON files with retest results
|
||||
*/
|
||||
export async function writeMergedResults(
|
||||
originalPath: string,
|
||||
retestPath: string,
|
||||
): Promise<{ updatedFiles: string[]; removedFiles: string[] }> {
|
||||
const mochawesomeDir = path.join(
|
||||
originalPath,
|
||||
"mochawesome-report",
|
||||
"json",
|
||||
"tests",
|
||||
);
|
||||
const retestMochawesomeDir = path.join(
|
||||
retestPath,
|
||||
"mochawesome-report",
|
||||
"json",
|
||||
"tests",
|
||||
);
|
||||
|
||||
const originalJsonFiles = await findJsonFiles(mochawesomeDir);
|
||||
const retestJsonFiles = await findJsonFiles(retestMochawesomeDir);
|
||||
|
||||
const updatedFiles: string[] = [];
|
||||
const removedFiles: string[] = [];
|
||||
|
||||
// For each retest file, find and replace the original
|
||||
for (const retestFile of retestJsonFiles) {
|
||||
const retestSpec = await parseSpecFile(retestFile);
|
||||
if (!retestSpec) continue;
|
||||
|
||||
const specPath = retestSpec.specPath;
|
||||
|
||||
// Find all original files with matching spec path
|
||||
// Prefer nested path (under integration/), remove flat duplicates
|
||||
let nestedFile: string | null = null;
|
||||
const flatFiles: string[] = [];
|
||||
|
||||
for (const origFile of originalJsonFiles) {
|
||||
const origSpec = await parseSpecFile(origFile);
|
||||
if (origSpec && origSpec.specPath === specPath) {
|
||||
if (origFile.includes("/integration/")) {
|
||||
nestedFile = origFile;
|
||||
} else {
|
||||
flatFiles.push(origFile);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Update the nested file (proper location) or first flat file if no nested
|
||||
const retestContent = await fs.readFile(retestFile, "utf8");
|
||||
|
||||
if (nestedFile) {
|
||||
await fs.writeFile(nestedFile, retestContent);
|
||||
updatedFiles.push(nestedFile);
|
||||
|
||||
// Remove flat duplicates
|
||||
for (const flatFile of flatFiles) {
|
||||
await fs.unlink(flatFile);
|
||||
removedFiles.push(flatFile);
|
||||
}
|
||||
} else if (flatFiles.length > 0) {
|
||||
await fs.writeFile(flatFiles[0], retestContent);
|
||||
updatedFiles.push(flatFiles[0]);
|
||||
}
|
||||
}
|
||||
|
||||
return { updatedFiles, removedFiles };
|
||||
}
|
||||
@@ -1,139 +0,0 @@
|
||||
/**
|
||||
* Mochawesome result structure for a single spec file
|
||||
*/
|
||||
export interface MochawesomeResult {
|
||||
stats: MochawesomeStats;
|
||||
results: ResultItem[];
|
||||
}
|
||||
|
||||
export interface MochawesomeStats {
|
||||
suites: number;
|
||||
tests: number;
|
||||
passes: number;
|
||||
pending: number;
|
||||
failures: number;
|
||||
start: string;
|
||||
end: string;
|
||||
duration: number;
|
||||
testsRegistered: number;
|
||||
passPercent: number;
|
||||
pendingPercent: number;
|
||||
other: number;
|
||||
hasOther: boolean;
|
||||
skipped: number;
|
||||
hasSkipped: boolean;
|
||||
}
|
||||
|
||||
export interface ResultItem {
|
||||
uuid: string;
|
||||
title: string;
|
||||
fullFile: string;
|
||||
file: string;
|
||||
beforeHooks: Hook[];
|
||||
afterHooks: Hook[];
|
||||
tests: TestItem[];
|
||||
suites: SuiteItem[];
|
||||
passes: string[];
|
||||
failures: string[];
|
||||
pending: string[];
|
||||
skipped: string[];
|
||||
duration: number;
|
||||
root: boolean;
|
||||
rootEmpty: boolean;
|
||||
_timeout: number;
|
||||
}
|
||||
|
||||
export interface SuiteItem {
|
||||
uuid: string;
|
||||
title: string;
|
||||
fullFile: string;
|
||||
file: string;
|
||||
beforeHooks: Hook[];
|
||||
afterHooks: Hook[];
|
||||
tests: TestItem[];
|
||||
suites: SuiteItem[];
|
||||
passes: string[];
|
||||
failures: string[];
|
||||
pending: string[];
|
||||
skipped: string[];
|
||||
duration: number;
|
||||
root: boolean;
|
||||
rootEmpty: boolean;
|
||||
_timeout: number;
|
||||
}
|
||||
|
||||
export interface TestItem {
|
||||
title: string;
|
||||
fullTitle: string;
|
||||
timedOut: boolean | null;
|
||||
duration: number;
|
||||
state: "passed" | "failed" | "pending";
|
||||
speed: string | null;
|
||||
pass: boolean;
|
||||
fail: boolean;
|
||||
pending: boolean;
|
||||
context: string | null;
|
||||
code: string;
|
||||
err: TestError;
|
||||
uuid: string;
|
||||
parentUUID: string;
|
||||
isHook: boolean;
|
||||
skipped: boolean;
|
||||
}
|
||||
|
||||
export interface TestError {
|
||||
message?: string;
|
||||
estack?: string;
|
||||
diff?: string | null;
|
||||
}
|
||||
|
||||
export interface Hook {
|
||||
title: string;
|
||||
fullTitle: string;
|
||||
timedOut: boolean | null;
|
||||
duration: number;
|
||||
state: string | null;
|
||||
speed: string | null;
|
||||
pass: boolean;
|
||||
fail: boolean;
|
||||
pending: boolean;
|
||||
context: string | null;
|
||||
code: string;
|
||||
err: TestError;
|
||||
uuid: string;
|
||||
parentUUID: string;
|
||||
isHook: boolean;
|
||||
skipped: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* Parsed spec file with its path and results
|
||||
*/
|
||||
export interface ParsedSpecFile {
|
||||
filePath: string;
|
||||
specPath: string;
|
||||
result: MochawesomeResult;
|
||||
}
|
||||
|
||||
/**
|
||||
* Calculation result outputs
|
||||
*/
|
||||
export interface CalculationResult {
|
||||
passed: number;
|
||||
failed: number;
|
||||
pending: number;
|
||||
totalSpecs: number;
|
||||
commitStatusMessage: string;
|
||||
failedSpecs: string;
|
||||
failedSpecsCount: number;
|
||||
failedTests: string;
|
||||
total: number;
|
||||
passRate: string;
|
||||
color: string;
|
||||
testDuration: string;
|
||||
}
|
||||
|
||||
export interface FailedTest {
|
||||
title: string;
|
||||
file: string;
|
||||
}
|
||||
@@ -1,17 +0,0 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "ES2022",
|
||||
"module": "CommonJS",
|
||||
"moduleResolution": "Node",
|
||||
"strict": true,
|
||||
"esModuleInterop": true,
|
||||
"skipLibCheck": true,
|
||||
"forceConsistentCasingInFileNames": true,
|
||||
"outDir": "./dist",
|
||||
"rootDir": "./src",
|
||||
"declaration": true,
|
||||
"isolatedModules": true
|
||||
},
|
||||
"include": ["src/**/*"],
|
||||
"exclude": ["node_modules", "dist", "**/*.test.ts"]
|
||||
}
|
||||
@@ -1 +0,0 @@
|
||||
{"root":["./src/index.ts","./src/main.ts","./src/merge.ts","./src/types.ts"],"version":"5.9.3"}
|
||||
@@ -1,13 +0,0 @@
|
||||
import { defineConfig } from "tsup";
|
||||
|
||||
export default defineConfig({
|
||||
entry: ["src/index.ts"],
|
||||
format: ["cjs"],
|
||||
target: "node24",
|
||||
clean: true,
|
||||
minify: false,
|
||||
sourcemap: false,
|
||||
splitting: false,
|
||||
bundle: true,
|
||||
noExternal: [/.*/],
|
||||
});
|
||||
@@ -1,2 +0,0 @@
|
||||
node_modules/
|
||||
.env
|
||||
@@ -1,53 +0,0 @@
|
||||
name: Calculate Playwright Results
|
||||
description: Calculate Playwright test results with optional merge of retest results
|
||||
author: Mattermost
|
||||
|
||||
inputs:
|
||||
original-results-path:
|
||||
description: Path to the original Playwright results.json file
|
||||
required: true
|
||||
retest-results-path:
|
||||
description: Path to the retest Playwright results.json file (optional - if not provided, only calculates from original)
|
||||
required: false
|
||||
output-path:
|
||||
description: Path to write the merged results.json file (defaults to original-results-path)
|
||||
required: false
|
||||
|
||||
outputs:
|
||||
# Merge outputs
|
||||
merged:
|
||||
description: Whether merge was performed (true/false)
|
||||
|
||||
# Calculation outputs (same as calculate-playwright-test-results)
|
||||
passed:
|
||||
description: Number of passed tests (not including flaky)
|
||||
failed:
|
||||
description: Number of failed tests
|
||||
flaky:
|
||||
description: Number of flaky tests (failed initially but passed on retry)
|
||||
skipped:
|
||||
description: Number of skipped tests
|
||||
total_specs:
|
||||
description: Total number of spec files
|
||||
commit_status_message:
|
||||
description: Message for commit status (e.g., "X failed, Y passed (Z spec files)")
|
||||
failed_specs:
|
||||
description: Comma-separated list of failed spec files (for retest)
|
||||
failed_specs_count:
|
||||
description: Number of failed spec files
|
||||
failed_tests:
|
||||
description: Markdown table rows of failed tests (for GitHub summary)
|
||||
total:
|
||||
description: Total number of tests (passed + flaky + failed)
|
||||
pass_rate:
|
||||
description: Pass rate percentage (e.g., "100.00")
|
||||
passing:
|
||||
description: Number of passing tests (passed + flaky)
|
||||
color:
|
||||
description: Color for webhook based on pass rate (green=100%, yellow=99%+, orange=98%+, red=<98%)
|
||||
test_duration:
|
||||
description: Test execution duration from stats (formatted as "Xm Ys")
|
||||
|
||||
runs:
|
||||
using: node24
|
||||
main: dist/index.js
|
||||
File diff suppressed because one or more lines are too long
@@ -1,6 +0,0 @@
|
||||
module.exports = {
|
||||
preset: "ts-jest",
|
||||
testEnvironment: "node",
|
||||
testMatch: ["**/*.test.ts"],
|
||||
moduleFileExtensions: ["ts", "js"],
|
||||
};
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,27 +0,0 @@
|
||||
{
|
||||
"name": "calculate-playwright-results",
|
||||
"private": true,
|
||||
"version": "0.1.0",
|
||||
"main": "dist/index.js",
|
||||
"scripts": {
|
||||
"build": "tsup",
|
||||
"prettier": "npx prettier --write \"src/**/*.ts\"",
|
||||
"local-action": "local-action . src/main.ts .env",
|
||||
"test": "jest --verbose",
|
||||
"test:watch": "jest --watch --verbose",
|
||||
"test:silent": "jest --silent",
|
||||
"tsc": "tsc -b"
|
||||
},
|
||||
"dependencies": {
|
||||
"@actions/core": "3.0.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@github/local-action": "7.0.0",
|
||||
"@types/jest": "30.0.0",
|
||||
"@types/node": "25.2.0",
|
||||
"jest": "30.2.0",
|
||||
"ts-jest": "29.4.6",
|
||||
"tsup": "8.5.1",
|
||||
"typescript": "5.9.3"
|
||||
}
|
||||
}
|
||||
@@ -1,3 +0,0 @@
|
||||
import { run } from "./main";
|
||||
|
||||
run();
|
||||
@@ -1,123 +0,0 @@
|
||||
import * as core from "@actions/core";
|
||||
import * as fs from "fs/promises";
|
||||
import type { PlaywrightResults } from "./types";
|
||||
import { mergeResults, calculateResults } from "./merge";
|
||||
|
||||
export async function run(): Promise<void> {
|
||||
const originalPath = core.getInput("original-results-path", {
|
||||
required: true,
|
||||
});
|
||||
const retestPath = core.getInput("retest-results-path"); // Optional
|
||||
const outputPath = core.getInput("output-path") || originalPath;
|
||||
|
||||
core.info(`Original results: ${originalPath}`);
|
||||
core.info(`Retest results: ${retestPath || "(not provided)"}`);
|
||||
core.info(`Output path: ${outputPath}`);
|
||||
|
||||
// Check if original file exists
|
||||
const originalExists = await fs
|
||||
.access(originalPath)
|
||||
.then(() => true)
|
||||
.catch(() => false);
|
||||
|
||||
if (!originalExists) {
|
||||
core.setFailed(`Original results not found at ${originalPath}`);
|
||||
return;
|
||||
}
|
||||
|
||||
// Read original file
|
||||
core.info("Reading original results...");
|
||||
const originalContent = await fs.readFile(originalPath, "utf8");
|
||||
const original: PlaywrightResults = JSON.parse(originalContent);
|
||||
|
||||
core.info(
|
||||
`Original: ${original.suites.length} suites, stats: ${JSON.stringify(original.stats)}`,
|
||||
);
|
||||
|
||||
// Check if retest path is provided and exists
|
||||
let finalResults: PlaywrightResults;
|
||||
let merged = false;
|
||||
|
||||
if (retestPath) {
|
||||
const retestExists = await fs
|
||||
.access(retestPath)
|
||||
.then(() => true)
|
||||
.catch(() => false);
|
||||
|
||||
if (retestExists) {
|
||||
// Read retest file and merge
|
||||
core.info("Reading retest results...");
|
||||
const retestContent = await fs.readFile(retestPath, "utf8");
|
||||
const retest: PlaywrightResults = JSON.parse(retestContent);
|
||||
|
||||
core.info(
|
||||
`Retest: ${retest.suites.length} suites, stats: ${JSON.stringify(retest.stats)}`,
|
||||
);
|
||||
|
||||
// Merge results
|
||||
core.info("Merging results at suite level...");
|
||||
const mergeResult = mergeResults(original, retest);
|
||||
finalResults = mergeResult.merged;
|
||||
merged = true;
|
||||
|
||||
core.info(`Retested specs: ${mergeResult.retestFiles.join(", ")}`);
|
||||
core.info(
|
||||
`Kept ${original.suites.length - mergeResult.retestFiles.length} original suites`,
|
||||
);
|
||||
core.info(`Added ${retest.suites.length} retest suites`);
|
||||
core.info(`Total merged suites: ${mergeResult.totalSuites}`);
|
||||
|
||||
// Write merged results
|
||||
core.info(`Writing merged results to ${outputPath}...`);
|
||||
await fs.writeFile(
|
||||
outputPath,
|
||||
JSON.stringify(finalResults, null, 2),
|
||||
);
|
||||
} else {
|
||||
core.warning(
|
||||
`Retest results not found at ${retestPath}, using original only`,
|
||||
);
|
||||
finalResults = original;
|
||||
}
|
||||
} else {
|
||||
core.info("No retest path provided, using original results only");
|
||||
finalResults = original;
|
||||
}
|
||||
|
||||
// Calculate all outputs from final results
|
||||
const calc = calculateResults(finalResults);
|
||||
|
||||
// Log results
|
||||
core.startGroup("Final Results");
|
||||
core.info(`Passed: ${calc.passed}`);
|
||||
core.info(`Failed: ${calc.failed}`);
|
||||
core.info(`Flaky: ${calc.flaky}`);
|
||||
core.info(`Skipped: ${calc.skipped}`);
|
||||
core.info(`Passing (passed + flaky): ${calc.passing}`);
|
||||
core.info(`Total: ${calc.total}`);
|
||||
core.info(`Pass Rate: ${calc.passRate}%`);
|
||||
core.info(`Color: ${calc.color}`);
|
||||
core.info(`Spec Files: ${calc.totalSpecs}`);
|
||||
core.info(`Failed Specs Count: ${calc.failedSpecsCount}`);
|
||||
core.info(`Commit Status Message: ${calc.commitStatusMessage}`);
|
||||
core.info(`Failed Specs: ${calc.failedSpecs || "none"}`);
|
||||
core.info(`Test Duration: ${calc.testDuration}`);
|
||||
core.endGroup();
|
||||
|
||||
// Set all outputs
|
||||
core.setOutput("merged", merged.toString());
|
||||
core.setOutput("passed", calc.passed);
|
||||
core.setOutput("failed", calc.failed);
|
||||
core.setOutput("flaky", calc.flaky);
|
||||
core.setOutput("skipped", calc.skipped);
|
||||
core.setOutput("total_specs", calc.totalSpecs);
|
||||
core.setOutput("commit_status_message", calc.commitStatusMessage);
|
||||
core.setOutput("failed_specs", calc.failedSpecs);
|
||||
core.setOutput("failed_specs_count", calc.failedSpecsCount);
|
||||
core.setOutput("failed_tests", calc.failedTests);
|
||||
core.setOutput("total", calc.total);
|
||||
core.setOutput("pass_rate", calc.passRate);
|
||||
core.setOutput("passing", calc.passing);
|
||||
core.setOutput("color", calc.color);
|
||||
core.setOutput("test_duration", calc.testDuration);
|
||||
}
|
||||
@@ -1,509 +0,0 @@
|
||||
import { mergeResults, computeStats, calculateResults } from "./merge";
|
||||
import type { PlaywrightResults, Suite } from "./types";
|
||||
|
||||
describe("mergeResults", () => {
|
||||
const createSuite = (file: string, tests: { status: string }[]): Suite => ({
|
||||
title: file,
|
||||
file,
|
||||
column: 0,
|
||||
line: 0,
|
||||
specs: [
|
||||
{
|
||||
title: "test spec",
|
||||
ok: true,
|
||||
tags: [],
|
||||
tests: tests.map((t) => ({
|
||||
timeout: 60000,
|
||||
annotations: [],
|
||||
expectedStatus: "passed",
|
||||
projectId: "chrome",
|
||||
projectName: "chrome",
|
||||
results: [
|
||||
{
|
||||
workerIndex: 0,
|
||||
parallelIndex: 0,
|
||||
status: t.status,
|
||||
duration: 1000,
|
||||
errors: [],
|
||||
stdout: [],
|
||||
stderr: [],
|
||||
retry: 0,
|
||||
startTime: new Date().toISOString(),
|
||||
annotations: [],
|
||||
},
|
||||
],
|
||||
})),
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
it("should keep original suites not in retest", () => {
|
||||
const original: PlaywrightResults = {
|
||||
config: {},
|
||||
suites: [
|
||||
createSuite("spec1.ts", [{ status: "passed" }]),
|
||||
createSuite("spec2.ts", [{ status: "failed" }]),
|
||||
createSuite("spec3.ts", [{ status: "passed" }]),
|
||||
],
|
||||
stats: {
|
||||
startTime: new Date().toISOString(),
|
||||
duration: 10000,
|
||||
expected: 2,
|
||||
unexpected: 1,
|
||||
skipped: 0,
|
||||
flaky: 0,
|
||||
},
|
||||
};
|
||||
|
||||
const retest: PlaywrightResults = {
|
||||
config: {},
|
||||
suites: [createSuite("spec2.ts", [{ status: "passed" }])],
|
||||
stats: {
|
||||
startTime: new Date().toISOString(),
|
||||
duration: 5000,
|
||||
expected: 1,
|
||||
unexpected: 0,
|
||||
skipped: 0,
|
||||
flaky: 0,
|
||||
},
|
||||
};
|
||||
|
||||
const result = mergeResults(original, retest);
|
||||
|
||||
expect(result.totalSuites).toBe(3);
|
||||
expect(result.retestFiles).toEqual(["spec2.ts"]);
|
||||
expect(result.merged.suites.map((s) => s.file)).toEqual([
|
||||
"spec1.ts",
|
||||
"spec3.ts",
|
||||
"spec2.ts",
|
||||
]);
|
||||
});
|
||||
|
||||
it("should compute correct stats from merged suites", () => {
|
||||
const original: PlaywrightResults = {
|
||||
config: {},
|
||||
suites: [
|
||||
createSuite("spec1.ts", [{ status: "passed" }]),
|
||||
createSuite("spec2.ts", [{ status: "failed" }]),
|
||||
],
|
||||
stats: {
|
||||
startTime: new Date().toISOString(),
|
||||
duration: 10000,
|
||||
expected: 1,
|
||||
unexpected: 1,
|
||||
skipped: 0,
|
||||
flaky: 0,
|
||||
},
|
||||
};
|
||||
|
||||
const retest: PlaywrightResults = {
|
||||
config: {},
|
||||
suites: [createSuite("spec2.ts", [{ status: "passed" }])],
|
||||
stats: {
|
||||
startTime: new Date().toISOString(),
|
||||
duration: 5000,
|
||||
expected: 1,
|
||||
unexpected: 0,
|
||||
skipped: 0,
|
||||
flaky: 0,
|
||||
},
|
||||
};
|
||||
|
||||
const result = mergeResults(original, retest);
|
||||
|
||||
expect(result.stats.expected).toBe(2);
|
||||
expect(result.stats.unexpected).toBe(0);
|
||||
expect(result.stats.duration).toBe(15000);
|
||||
});
|
||||
});
|
||||
|
||||
describe("computeStats", () => {
|
||||
it("should count flaky tests correctly", () => {
|
||||
const suites: Suite[] = [
|
||||
{
|
||||
title: "spec1.ts",
|
||||
file: "spec1.ts",
|
||||
column: 0,
|
||||
line: 0,
|
||||
specs: [
|
||||
{
|
||||
title: "flaky test",
|
||||
ok: true,
|
||||
tags: [],
|
||||
tests: [
|
||||
{
|
||||
timeout: 60000,
|
||||
annotations: [],
|
||||
expectedStatus: "passed",
|
||||
projectId: "chrome",
|
||||
projectName: "chrome",
|
||||
results: [
|
||||
{
|
||||
workerIndex: 0,
|
||||
parallelIndex: 0,
|
||||
status: "failed",
|
||||
duration: 1000,
|
||||
errors: [],
|
||||
stdout: [],
|
||||
stderr: [],
|
||||
retry: 0,
|
||||
startTime: new Date().toISOString(),
|
||||
annotations: [],
|
||||
},
|
||||
{
|
||||
workerIndex: 0,
|
||||
parallelIndex: 0,
|
||||
status: "passed",
|
||||
duration: 1000,
|
||||
errors: [],
|
||||
stdout: [],
|
||||
stderr: [],
|
||||
retry: 1,
|
||||
startTime: new Date().toISOString(),
|
||||
annotations: [],
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
];
|
||||
|
||||
const stats = computeStats(suites);
|
||||
|
||||
expect(stats.expected).toBe(0);
|
||||
expect(stats.flaky).toBe(1);
|
||||
expect(stats.unexpected).toBe(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe("calculateResults", () => {
|
||||
const createSuiteWithSpec = (
|
||||
file: string,
|
||||
specTitle: string,
|
||||
testResults: { status: string; retry: number }[],
|
||||
): Suite => ({
|
||||
title: file,
|
||||
file,
|
||||
column: 0,
|
||||
line: 0,
|
||||
specs: [
|
||||
{
|
||||
title: specTitle,
|
||||
ok: testResults[testResults.length - 1].status === "passed",
|
||||
tags: [],
|
||||
tests: [
|
||||
{
|
||||
timeout: 60000,
|
||||
annotations: [],
|
||||
expectedStatus: "passed",
|
||||
projectId: "chrome",
|
||||
projectName: "chrome",
|
||||
results: testResults.map((r) => ({
|
||||
workerIndex: 0,
|
||||
parallelIndex: 0,
|
||||
status: r.status,
|
||||
duration: 1000,
|
||||
errors:
|
||||
r.status === "failed"
|
||||
? [{ message: "error" }]
|
||||
: [],
|
||||
stdout: [],
|
||||
stderr: [],
|
||||
retry: r.retry,
|
||||
startTime: new Date().toISOString(),
|
||||
annotations: [],
|
||||
})),
|
||||
location: {
|
||||
file,
|
||||
line: 10,
|
||||
column: 5,
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
it("should calculate all outputs correctly for passing results", () => {
|
||||
const results: PlaywrightResults = {
|
||||
config: {},
|
||||
suites: [
|
||||
createSuiteWithSpec("login.spec.ts", "should login", [
|
||||
{ status: "passed", retry: 0 },
|
||||
]),
|
||||
createSuiteWithSpec(
|
||||
"messaging.spec.ts",
|
||||
"should send message",
|
||||
[{ status: "passed", retry: 0 }],
|
||||
),
|
||||
],
|
||||
stats: {
|
||||
startTime: new Date().toISOString(),
|
||||
duration: 5000,
|
||||
expected: 2,
|
||||
unexpected: 0,
|
||||
skipped: 0,
|
||||
flaky: 0,
|
||||
},
|
||||
};
|
||||
|
||||
const calc = calculateResults(results);
|
||||
|
||||
expect(calc.passed).toBe(2);
|
||||
expect(calc.failed).toBe(0);
|
||||
expect(calc.flaky).toBe(0);
|
||||
expect(calc.skipped).toBe(0);
|
||||
expect(calc.total).toBe(2);
|
||||
expect(calc.passing).toBe(2);
|
||||
expect(calc.passRate).toBe("100.00");
|
||||
expect(calc.color).toBe("#43A047"); // green
|
||||
expect(calc.totalSpecs).toBe(2);
|
||||
expect(calc.failedSpecs).toBe("");
|
||||
expect(calc.failedSpecsCount).toBe(0);
|
||||
expect(calc.commitStatusMessage).toBe("100% passed (2), 2 specs");
|
||||
});
|
||||
|
||||
it("should calculate all outputs correctly for results with failures", () => {
|
||||
const results: PlaywrightResults = {
|
||||
config: {},
|
||||
suites: [
|
||||
createSuiteWithSpec("login.spec.ts", "should login", [
|
||||
{ status: "passed", retry: 0 },
|
||||
]),
|
||||
createSuiteWithSpec(
|
||||
"channels.spec.ts",
|
||||
"should create channel",
|
||||
[
|
||||
{ status: "failed", retry: 0 },
|
||||
{ status: "failed", retry: 1 },
|
||||
{ status: "failed", retry: 2 },
|
||||
],
|
||||
),
|
||||
],
|
||||
stats: {
|
||||
startTime: new Date().toISOString(),
|
||||
duration: 10000,
|
||||
expected: 1,
|
||||
unexpected: 1,
|
||||
skipped: 0,
|
||||
flaky: 0,
|
||||
},
|
||||
};
|
||||
|
||||
const calc = calculateResults(results);
|
||||
|
||||
expect(calc.passed).toBe(1);
|
||||
expect(calc.failed).toBe(1);
|
||||
expect(calc.flaky).toBe(0);
|
||||
expect(calc.total).toBe(2);
|
||||
expect(calc.passing).toBe(1);
|
||||
expect(calc.passRate).toBe("50.00");
|
||||
expect(calc.color).toBe("#F44336"); // red
|
||||
expect(calc.totalSpecs).toBe(2);
|
||||
expect(calc.failedSpecs).toBe("channels.spec.ts");
|
||||
expect(calc.failedSpecsCount).toBe(1);
|
||||
expect(calc.commitStatusMessage).toBe(
|
||||
"50.0% passed (1/2), 1 failed, 2 specs",
|
||||
);
|
||||
expect(calc.failedTests).toContain("should create channel");
|
||||
});
|
||||
});
|
||||
|
||||
describe("full integration: original with failure, retest passes", () => {
|
||||
const createSuiteWithSpec = (
|
||||
file: string,
|
||||
specTitle: string,
|
||||
testResults: { status: string; retry: number }[],
|
||||
): Suite => ({
|
||||
title: file,
|
||||
file,
|
||||
column: 0,
|
||||
line: 0,
|
||||
specs: [
|
||||
{
|
||||
title: specTitle,
|
||||
ok: testResults[testResults.length - 1].status === "passed",
|
||||
tags: [],
|
||||
tests: [
|
||||
{
|
||||
timeout: 60000,
|
||||
annotations: [],
|
||||
expectedStatus: "passed",
|
||||
projectId: "chrome",
|
||||
projectName: "chrome",
|
||||
results: testResults.map((r) => ({
|
||||
workerIndex: 0,
|
||||
parallelIndex: 0,
|
||||
status: r.status,
|
||||
duration: 1000,
|
||||
errors:
|
||||
r.status === "failed"
|
||||
? [{ message: "error" }]
|
||||
: [],
|
||||
stdout: [],
|
||||
stderr: [],
|
||||
retry: r.retry,
|
||||
startTime: new Date().toISOString(),
|
||||
annotations: [],
|
||||
})),
|
||||
location: {
|
||||
file,
|
||||
line: 10,
|
||||
column: 5,
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
it("should merge and calculate correctly when failed test passes on retest", () => {
|
||||
// Original: 2 passed, 1 failed (channels.spec.ts)
|
||||
const original: PlaywrightResults = {
|
||||
config: {},
|
||||
suites: [
|
||||
createSuiteWithSpec("login.spec.ts", "should login", [
|
||||
{ status: "passed", retry: 0 },
|
||||
]),
|
||||
createSuiteWithSpec(
|
||||
"messaging.spec.ts",
|
||||
"should send message",
|
||||
[{ status: "passed", retry: 0 }],
|
||||
),
|
||||
createSuiteWithSpec(
|
||||
"channels.spec.ts",
|
||||
"should create channel",
|
||||
[
|
||||
{ status: "failed", retry: 0 },
|
||||
{ status: "failed", retry: 1 },
|
||||
{ status: "failed", retry: 2 },
|
||||
],
|
||||
),
|
||||
],
|
||||
stats: {
|
||||
startTime: new Date().toISOString(),
|
||||
duration: 18000,
|
||||
expected: 2,
|
||||
unexpected: 1,
|
||||
skipped: 0,
|
||||
flaky: 0,
|
||||
},
|
||||
};
|
||||
|
||||
// Retest: channels.spec.ts now passes
|
||||
const retest: PlaywrightResults = {
|
||||
config: {},
|
||||
suites: [
|
||||
createSuiteWithSpec(
|
||||
"channels.spec.ts",
|
||||
"should create channel",
|
||||
[{ status: "passed", retry: 0 }],
|
||||
),
|
||||
],
|
||||
stats: {
|
||||
startTime: new Date().toISOString(),
|
||||
duration: 3000,
|
||||
expected: 1,
|
||||
unexpected: 0,
|
||||
skipped: 0,
|
||||
flaky: 0,
|
||||
},
|
||||
};
|
||||
|
||||
// Step 1: Verify original has failure
|
||||
const originalCalc = calculateResults(original);
|
||||
expect(originalCalc.passed).toBe(2);
|
||||
expect(originalCalc.failed).toBe(1);
|
||||
expect(originalCalc.passRate).toBe("66.67");
|
||||
|
||||
// Step 2: Merge results
|
||||
const mergeResult = mergeResults(original, retest);
|
||||
|
||||
// Step 3: Verify merge structure
|
||||
expect(mergeResult.totalSuites).toBe(3);
|
||||
expect(mergeResult.retestFiles).toEqual(["channels.spec.ts"]);
|
||||
expect(mergeResult.merged.suites.map((s) => s.file)).toEqual([
|
||||
"login.spec.ts",
|
||||
"messaging.spec.ts",
|
||||
"channels.spec.ts",
|
||||
]);
|
||||
|
||||
// Step 4: Calculate final results
|
||||
const finalCalc = calculateResults(mergeResult.merged);
|
||||
|
||||
// Step 5: Verify all outputs
|
||||
expect(finalCalc.passed).toBe(3);
|
||||
expect(finalCalc.failed).toBe(0);
|
||||
expect(finalCalc.flaky).toBe(0);
|
||||
expect(finalCalc.skipped).toBe(0);
|
||||
expect(finalCalc.total).toBe(3);
|
||||
expect(finalCalc.passing).toBe(3);
|
||||
expect(finalCalc.passRate).toBe("100.00");
|
||||
expect(finalCalc.color).toBe("#43A047"); // green
|
||||
expect(finalCalc.totalSpecs).toBe(3);
|
||||
expect(finalCalc.failedSpecs).toBe("");
|
||||
expect(finalCalc.failedSpecsCount).toBe(0);
|
||||
expect(finalCalc.commitStatusMessage).toBe("100% passed (3), 3 specs");
|
||||
expect(finalCalc.failedTests).toBe("");
|
||||
});
|
||||
|
||||
it("should handle case where retest still fails", () => {
|
||||
// Original: 2 passed, 1 failed
|
||||
const original: PlaywrightResults = {
|
||||
config: {},
|
||||
suites: [
|
||||
createSuiteWithSpec("login.spec.ts", "should login", [
|
||||
{ status: "passed", retry: 0 },
|
||||
]),
|
||||
createSuiteWithSpec(
|
||||
"channels.spec.ts",
|
||||
"should create channel",
|
||||
[{ status: "failed", retry: 0 }],
|
||||
),
|
||||
],
|
||||
stats: {
|
||||
startTime: new Date().toISOString(),
|
||||
duration: 10000,
|
||||
expected: 1,
|
||||
unexpected: 1,
|
||||
skipped: 0,
|
||||
flaky: 0,
|
||||
},
|
||||
};
|
||||
|
||||
// Retest: channels.spec.ts still fails
|
||||
const retest: PlaywrightResults = {
|
||||
config: {},
|
||||
suites: [
|
||||
createSuiteWithSpec(
|
||||
"channels.spec.ts",
|
||||
"should create channel",
|
||||
[
|
||||
{ status: "failed", retry: 0 },
|
||||
{ status: "failed", retry: 1 },
|
||||
],
|
||||
),
|
||||
],
|
||||
stats: {
|
||||
startTime: new Date().toISOString(),
|
||||
duration: 5000,
|
||||
expected: 0,
|
||||
unexpected: 1,
|
||||
skipped: 0,
|
||||
flaky: 0,
|
||||
},
|
||||
};
|
||||
|
||||
const mergeResult = mergeResults(original, retest);
|
||||
const finalCalc = calculateResults(mergeResult.merged);
|
||||
|
||||
expect(finalCalc.passed).toBe(1);
|
||||
expect(finalCalc.failed).toBe(1);
|
||||
expect(finalCalc.passRate).toBe("50.00");
|
||||
expect(finalCalc.color).toBe("#F44336"); // red
|
||||
expect(finalCalc.failedSpecs).toBe("channels.spec.ts");
|
||||
expect(finalCalc.failedSpecsCount).toBe(1);
|
||||
});
|
||||
});
|
||||
@@ -1,304 +0,0 @@
|
||||
import type {
|
||||
PlaywrightResults,
|
||||
Suite,
|
||||
Test,
|
||||
Stats,
|
||||
MergeResult,
|
||||
CalculationResult,
|
||||
FailedTest,
|
||||
} from "./types";
|
||||
|
||||
interface TestInfo {
|
||||
title: string;
|
||||
file: string;
|
||||
finalStatus: string;
|
||||
hadFailure: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract all tests from suites recursively with their info
|
||||
*/
|
||||
function getAllTestsWithInfo(suites: Suite[]): TestInfo[] {
|
||||
const tests: TestInfo[] = [];
|
||||
|
||||
function extractFromSuite(suite: Suite) {
|
||||
for (const spec of suite.specs || []) {
|
||||
for (const test of spec.tests || []) {
|
||||
if (!test.results || test.results.length === 0) {
|
||||
continue;
|
||||
}
|
||||
|
||||
const finalResult = test.results[test.results.length - 1];
|
||||
const hadFailure = test.results.some(
|
||||
(r) => r.status === "failed" || r.status === "timedOut",
|
||||
);
|
||||
|
||||
tests.push({
|
||||
title: spec.title || test.projectName,
|
||||
file: test.location?.file || suite.file,
|
||||
finalStatus: finalResult.status,
|
||||
hadFailure,
|
||||
});
|
||||
}
|
||||
}
|
||||
for (const nestedSuite of suite.suites || []) {
|
||||
extractFromSuite(nestedSuite);
|
||||
}
|
||||
}
|
||||
|
||||
for (const suite of suites) {
|
||||
extractFromSuite(suite);
|
||||
}
|
||||
|
||||
return tests;
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract all tests from suites recursively
|
||||
*/
|
||||
function getAllTests(suites: Suite[]): Test[] {
|
||||
const tests: Test[] = [];
|
||||
|
||||
function extractFromSuite(suite: Suite) {
|
||||
for (const spec of suite.specs || []) {
|
||||
tests.push(...spec.tests);
|
||||
}
|
||||
for (const nestedSuite of suite.suites || []) {
|
||||
extractFromSuite(nestedSuite);
|
||||
}
|
||||
}
|
||||
|
||||
for (const suite of suites) {
|
||||
extractFromSuite(suite);
|
||||
}
|
||||
|
||||
return tests;
|
||||
}
|
||||
|
||||
/**
|
||||
* Compute stats from suites
|
||||
*/
|
||||
export function computeStats(
|
||||
suites: Suite[],
|
||||
originalStats?: Stats,
|
||||
retestStats?: Stats,
|
||||
): Stats {
|
||||
const tests = getAllTests(suites);
|
||||
|
||||
let expected = 0;
|
||||
let unexpected = 0;
|
||||
let skipped = 0;
|
||||
let flaky = 0;
|
||||
|
||||
for (const test of tests) {
|
||||
if (!test.results || test.results.length === 0) {
|
||||
continue;
|
||||
}
|
||||
|
||||
const finalResult = test.results[test.results.length - 1];
|
||||
const finalStatus = finalResult.status;
|
||||
|
||||
// Check if any result was a failure
|
||||
const hadFailure = test.results.some(
|
||||
(r) => r.status === "failed" || r.status === "timedOut",
|
||||
);
|
||||
|
||||
if (finalStatus === "skipped") {
|
||||
skipped++;
|
||||
} else if (finalStatus === "failed" || finalStatus === "timedOut") {
|
||||
unexpected++;
|
||||
} else if (finalStatus === "passed") {
|
||||
if (hadFailure) {
|
||||
flaky++;
|
||||
} else {
|
||||
expected++;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Compute duration as sum of both runs
|
||||
const duration =
|
||||
(originalStats?.duration || 0) + (retestStats?.duration || 0);
|
||||
|
||||
return {
|
||||
startTime: originalStats?.startTime || new Date().toISOString(),
|
||||
duration,
|
||||
expected,
|
||||
unexpected,
|
||||
skipped,
|
||||
flaky,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Format milliseconds as "Xm Ys"
|
||||
*/
|
||||
function formatDuration(ms: number): string {
|
||||
const totalSeconds = Math.round(ms / 1000);
|
||||
const minutes = Math.floor(totalSeconds / 60);
|
||||
const seconds = totalSeconds % 60;
|
||||
return `${minutes}m ${seconds}s`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get color based on pass rate
|
||||
*/
|
||||
function getColor(passRate: number): string {
|
||||
if (passRate === 100) {
|
||||
return "#43A047"; // green
|
||||
} else if (passRate >= 99) {
|
||||
return "#FFEB3B"; // yellow
|
||||
} else if (passRate >= 98) {
|
||||
return "#FF9800"; // orange
|
||||
} else {
|
||||
return "#F44336"; // red
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Calculate all outputs from results
|
||||
*/
|
||||
export function calculateResults(
|
||||
results: PlaywrightResults,
|
||||
): CalculationResult {
|
||||
const stats = results.stats || {
|
||||
expected: 0,
|
||||
unexpected: 0,
|
||||
skipped: 0,
|
||||
flaky: 0,
|
||||
startTime: new Date().toISOString(),
|
||||
duration: 0,
|
||||
};
|
||||
|
||||
const passed = stats.expected;
|
||||
const failed = stats.unexpected;
|
||||
const flaky = stats.flaky;
|
||||
const skipped = stats.skipped;
|
||||
|
||||
// Count unique spec files
|
||||
const specFiles = new Set<string>();
|
||||
for (const suite of results.suites) {
|
||||
specFiles.add(suite.file);
|
||||
}
|
||||
const totalSpecs = specFiles.size;
|
||||
|
||||
// Get all tests with info for failed tests extraction
|
||||
const testsInfo = getAllTestsWithInfo(results.suites);
|
||||
|
||||
// Extract failed specs
|
||||
const failedSpecsSet = new Set<string>();
|
||||
const failedTestsList: FailedTest[] = [];
|
||||
|
||||
for (const test of testsInfo) {
|
||||
if (test.finalStatus === "failed" || test.finalStatus === "timedOut") {
|
||||
failedSpecsSet.add(test.file);
|
||||
failedTestsList.push({
|
||||
title: test.title,
|
||||
file: test.file,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
const failedSpecs = Array.from(failedSpecsSet).join(",");
|
||||
const failedSpecsCount = failedSpecsSet.size;
|
||||
|
||||
// Build failed tests markdown table (limit to 10)
|
||||
let failedTests = "";
|
||||
const uniqueFailedTests = failedTestsList.filter(
|
||||
(test, index, self) =>
|
||||
index ===
|
||||
self.findIndex(
|
||||
(t) => t.title === test.title && t.file === test.file,
|
||||
),
|
||||
);
|
||||
|
||||
if (uniqueFailedTests.length > 0) {
|
||||
const limitedTests = uniqueFailedTests.slice(0, 10);
|
||||
failedTests = limitedTests
|
||||
.map((t) => {
|
||||
const escapedTitle = t.title
|
||||
.replace(/`/g, "\\`")
|
||||
.replace(/\|/g, "\\|");
|
||||
return `| ${escapedTitle} | ${t.file} |`;
|
||||
})
|
||||
.join("\n");
|
||||
|
||||
if (uniqueFailedTests.length > 10) {
|
||||
const remaining = uniqueFailedTests.length - 10;
|
||||
failedTests += `\n| _...and ${remaining} more failed tests_ | |`;
|
||||
}
|
||||
} else if (failed > 0) {
|
||||
failedTests = "| Unable to parse failed tests | - |";
|
||||
}
|
||||
|
||||
// Calculate totals and pass rate
|
||||
const passing = passed + flaky;
|
||||
const total = passing + failed;
|
||||
const passRate = total > 0 ? ((passing * 100) / total).toFixed(2) : "0.00";
|
||||
const color = getColor(parseFloat(passRate));
|
||||
|
||||
// Build commit status message
|
||||
const rate = total > 0 ? (passing * 100) / total : 0;
|
||||
const rateStr = rate === 100 ? "100%" : `${rate.toFixed(1)}%`;
|
||||
const specSuffix = totalSpecs > 0 ? `, ${totalSpecs} specs` : "";
|
||||
const commitStatusMessage =
|
||||
rate === 100
|
||||
? `${rateStr} passed (${passing})${specSuffix}`
|
||||
: `${rateStr} passed (${passing}/${total}), ${failed} failed${specSuffix}`;
|
||||
|
||||
const testDuration = formatDuration(stats.duration || 0);
|
||||
|
||||
return {
|
||||
passed,
|
||||
failed,
|
||||
flaky,
|
||||
skipped,
|
||||
totalSpecs,
|
||||
commitStatusMessage,
|
||||
failedSpecs,
|
||||
failedSpecsCount,
|
||||
failedTests,
|
||||
total,
|
||||
passRate,
|
||||
passing,
|
||||
color,
|
||||
testDuration,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Merge original and retest results at suite level
|
||||
* - Keep original suites that are NOT in retest
|
||||
* - Add all retest suites (replacing matching originals)
|
||||
*/
|
||||
export function mergeResults(
|
||||
original: PlaywrightResults,
|
||||
retest: PlaywrightResults,
|
||||
): MergeResult {
|
||||
// Get list of retested spec files
|
||||
const retestFiles = retest.suites.map((s) => s.file);
|
||||
|
||||
// Filter original suites - keep only those NOT in retest
|
||||
const keptOriginalSuites = original.suites.filter(
|
||||
(suite) => !retestFiles.includes(suite.file),
|
||||
);
|
||||
|
||||
// Merge: kept original suites + all retest suites
|
||||
const mergedSuites = [...keptOriginalSuites, ...retest.suites];
|
||||
|
||||
// Compute stats from merged suites
|
||||
const stats = computeStats(mergedSuites, original.stats, retest.stats);
|
||||
|
||||
const merged: PlaywrightResults = {
|
||||
config: original.config,
|
||||
suites: mergedSuites,
|
||||
stats,
|
||||
};
|
||||
|
||||
return {
|
||||
merged,
|
||||
stats,
|
||||
totalSuites: mergedSuites.length,
|
||||
retestFiles,
|
||||
};
|
||||
}
|
||||
@@ -1,89 +0,0 @@
|
||||
export interface PlaywrightResults {
|
||||
config: Record<string, unknown>;
|
||||
suites: Suite[];
|
||||
stats?: Stats;
|
||||
}
|
||||
|
||||
export interface Suite {
|
||||
title: string;
|
||||
file: string;
|
||||
column: number;
|
||||
line: number;
|
||||
specs: Spec[];
|
||||
suites?: Suite[];
|
||||
}
|
||||
|
||||
export interface Spec {
|
||||
title: string;
|
||||
ok: boolean;
|
||||
tags: string[];
|
||||
tests: Test[];
|
||||
}
|
||||
|
||||
export interface Test {
|
||||
timeout: number;
|
||||
annotations: unknown[];
|
||||
expectedStatus: string;
|
||||
projectId: string;
|
||||
projectName: string;
|
||||
results: TestResult[];
|
||||
location?: TestLocation;
|
||||
}
|
||||
|
||||
export interface TestResult {
|
||||
workerIndex: number;
|
||||
parallelIndex: number;
|
||||
status: string;
|
||||
duration: number;
|
||||
errors: unknown[];
|
||||
stdout: unknown[];
|
||||
stderr: unknown[];
|
||||
retry: number;
|
||||
startTime: string;
|
||||
annotations: unknown[];
|
||||
attachments?: unknown[];
|
||||
}
|
||||
|
||||
export interface TestLocation {
|
||||
file: string;
|
||||
line: number;
|
||||
column: number;
|
||||
}
|
||||
|
||||
export interface Stats {
|
||||
startTime: string;
|
||||
duration: number;
|
||||
expected: number;
|
||||
unexpected: number;
|
||||
skipped: number;
|
||||
flaky: number;
|
||||
}
|
||||
|
||||
export interface MergeResult {
|
||||
merged: PlaywrightResults;
|
||||
stats: Stats;
|
||||
totalSuites: number;
|
||||
retestFiles: string[];
|
||||
}
|
||||
|
||||
export interface CalculationResult {
|
||||
passed: number;
|
||||
failed: number;
|
||||
flaky: number;
|
||||
skipped: number;
|
||||
totalSpecs: number;
|
||||
commitStatusMessage: string;
|
||||
failedSpecs: string;
|
||||
failedSpecsCount: number;
|
||||
failedTests: string;
|
||||
total: number;
|
||||
passRate: string;
|
||||
passing: number;
|
||||
color: string;
|
||||
testDuration: string;
|
||||
}
|
||||
|
||||
export interface FailedTest {
|
||||
title: string;
|
||||
file: string;
|
||||
}
|
||||
@@ -1,17 +0,0 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "ES2022",
|
||||
"module": "CommonJS",
|
||||
"moduleResolution": "Node",
|
||||
"strict": true,
|
||||
"esModuleInterop": true,
|
||||
"skipLibCheck": true,
|
||||
"forceConsistentCasingInFileNames": true,
|
||||
"outDir": "dist",
|
||||
"rootDir": "./src",
|
||||
"declaration": true,
|
||||
"isolatedModules": true
|
||||
},
|
||||
"include": ["src/**/*"],
|
||||
"exclude": ["node_modules", "dist", "**/*.test.ts"]
|
||||
}
|
||||
@@ -1 +0,0 @@
|
||||
{"root":["./src/index.ts","./src/main.ts","./src/merge.ts","./src/types.ts"],"version":"5.9.3"}
|
||||
@@ -1,12 +0,0 @@
|
||||
import { defineConfig } from "tsup";
|
||||
|
||||
export default defineConfig({
|
||||
entry: ["src/index.ts"],
|
||||
format: ["cjs"],
|
||||
outDir: "dist",
|
||||
clean: true,
|
||||
noExternal: [/.*/], // Bundle all dependencies
|
||||
minify: false,
|
||||
sourcemap: false,
|
||||
target: "node24",
|
||||
});
|
||||
@@ -16,25 +16,109 @@ All pipelines follow the **smoke-then-full** pattern: smoke tests run first, ful
|
||||
|
||||
```
|
||||
.github/workflows/
|
||||
├── e2e-tests-ci.yml # PR orchestrator
|
||||
├── e2e-tests-on-merge.yml # Merge orchestrator (master/release branches)
|
||||
├── e2e-tests-on-release.yml # Release cut orchestrator
|
||||
├── e2e-tests-cypress.yml # Shared wrapper: cypress smoke -> full
|
||||
├── e2e-tests-playwright.yml # Shared wrapper: playwright smoke -> full
|
||||
├── e2e-tests-cypress-template.yml # Template: actual cypress test execution
|
||||
└── e2e-tests-playwright-template.yml # Template: actual playwright test execution
|
||||
├── e2e-tests-ci.yml # PR orchestrator
|
||||
├── e2e-tests-on-merge.yml # Merge orchestrator (master/release branches)
|
||||
├── e2e-tests-on-release.yml # Release cut orchestrator
|
||||
├── e2e-tests-cypress.yml # Shared wrapper: calls the cypress template
|
||||
├── e2e-tests-playwright.yml # Shared wrapper: calls the playwright template
|
||||
├── e2e-tests-cypress-template.yml # cypress + test-system-io dispatch
|
||||
└── e2e-tests-playwright-template.yml # playwright + test-system-io dispatch
|
||||
```
|
||||
|
||||
### Call hierarchy
|
||||
|
||||
```
|
||||
e2e-tests-ci.yml ─────────────────┐
|
||||
e2e-tests-on-merge.yml ───────────┤──► e2e-tests-cypress.yml ──► e2e-tests-cypress-template.yml
|
||||
e2e-tests-on-release.yml ─────────┘ e2e-tests-playwright.yml ──► e2e-tests-playwright-template.yml
|
||||
e2e-tests-on-merge.yml ───────────┤──► e2e-tests-cypress.yml ────► e2e-tests-cypress-template.yml
|
||||
e2e-tests-on-release.yml ─────────┘ e2e-tests-playwright.yml ─► e2e-tests-playwright-template.yml
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Workflow Architecture
|
||||
|
||||
The template splits into five jobs — `prepare-run`, `prep-deps`, `dispatch-begin`, `workers` (matrix), and `report` — and pushes spec-level execution to [Test System IO](https://github.com/mattermost/mattermost-test-system-io) so workers stay thin and identical.
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────────────────────┐
|
||||
│ Template: e2e-tests-{cypress,playwright}-template.yml │
|
||||
│ │
|
||||
│ ┌───────────────────┐ ┌──────────────────────────────┐ │
|
||||
│ │ prepare-run │ │ prep-deps │ │
|
||||
│ │ (1 runner) │ parallel │ (1 runner) │ │
|
||||
│ │ │ ◄────────────► │ │ │
|
||||
│ │ • build workers │ │ Cypress: │ │
|
||||
│ │ matrix [1..N] │ │ • cypress/node_modules │ │
|
||||
│ │ • compute commit │ │ • ~/.cache/Cypress (binary)│ │
|
||||
│ │ status context │ │ │ │
|
||||
│ │ • emit composite │ │ Playwright: │ │
|
||||
│ │ identity │ │ • webapp/platform/{client, │ │
|
||||
│ │ │ │ types}/{lib,node_mod} │ │
|
||||
│ │ │ │ • playwright/node_modules │ │
|
||||
│ │ │ │ • playwright/lib/dist │ │
|
||||
│ │ │ │ • ~/.cache/ms-playwright │ │
|
||||
│ │ │ │ (chromium only) │ │
|
||||
│ │ │ │ │ │
|
||||
│ │ │ │ → saved to actions/cache │ │
|
||||
│ └─────────┬─────────┘ └───────────────┬──────────────┘ │
|
||||
│ │ │ │
|
||||
│ │ ▼ │
|
||||
│ │ ┌──────────────────────────────┐ │
|
||||
│ │ │ dispatch-begin │ │
|
||||
│ │ │ • register run with │ │
|
||||
│ │ │ Test System IO │ │
|
||||
│ │ │ • runs immediately before │ │
|
||||
│ │ │ workers to minimise the │ │
|
||||
│ │ │ inactivity-timeout window │ │
|
||||
│ │ └───────────────┬──────────────┘ │
|
||||
│ │ │ │
|
||||
│ └────────────────────┬─────────────────────┘ │
|
||||
│ ▼ │
|
||||
│ ┌─────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ workers (matrix, fail-fast: false) │ │
|
||||
│ │ Cypress full: N=40 | Playwright full: N=10 │ │
|
||||
│ │ │ │
|
||||
│ │ each worker, in parallel: │ │
|
||||
│ │ 1. sparse-checkout actions + full checkout-repo │ │
|
||||
│ │ 2. setup-node │ │
|
||||
│ │ 3. restore caches ◄─── actions/cache (from prep-deps) │ │
|
||||
│ │ (fail-on-cache-miss: true) │ │
|
||||
│ │ 4. cloud-init + start-server (docker compose stack) │ │
|
||||
│ │ 5. prepare-cypress | prepare-playwright (run setup project) │ │
|
||||
│ │ 6. dispatch-run ──────────────────────────────────┐ │ │
|
||||
│ │ (pulls specs from Test System IO, runs locally, │ │ │
|
||||
│ │ posts result, loops until queue is empty) │ │ │
|
||||
│ │ 7. cloud-teardown │ │ │
|
||||
│ └────────────────────┬────────────────────────────────────┼───────┘ │
|
||||
│ │ │ │
|
||||
│ ▼ │ │
|
||||
│ ┌─────────────────────────────────────────────────┐ │ │
|
||||
│ │ report │ │ │
|
||||
│ │ • pull aggregated results from Test System IO │ ◄────┘ │
|
||||
│ │ • post commit status │ │
|
||||
│ │ • send webhook notification │ │
|
||||
│ └─────────────────────────────────────────────────┘ │
|
||||
└────────────────────────────────────────────────────────────────────────┬─┘
|
||||
│
|
||||
┌───────────────────────────▼───┐
|
||||
│ Test System IO (external) │
|
||||
│ • spec-level dispatch │
|
||||
│ • result aggregation │
|
||||
│ • retry orchestration │
|
||||
└───────────────────────────────┘
|
||||
```
|
||||
|
||||
### Key properties
|
||||
|
||||
- **Spec-level vs. job-level parallelism.** The matrix sizes the runner pool; Test System IO does the spec assignment. Slow specs don't block a worker — fast workers keep pulling the next spec from the queue.
|
||||
- **Cache-only workers.** `prep-deps` installs once per workflow run and saves to `actions/cache`. Every worker restores with `fail-on-cache-miss: true` and runs zero `npm ci`. Eliminates the 40-way `EEXIST/ENOENT` race in npm's shared cacache writer.
|
||||
- **dispatch-begin runs late.** It depends on `prep-deps` so the gap between Test System IO run registration and the first worker calling `dispatch-run` is just per-worker setup (~3–5 min). Registering earlier risks the run timing out before any worker checks in, bulk-failing every spec.
|
||||
- **Playwright slim slice.** Playwright only consumes `@mattermost/client` and `@mattermost/types` from webapp, so prep-deps caches just those two packages' built `lib/` and `node_modules` (~10–30 MB) instead of the full `webapp/node_modules` tree (~1–2 GB).
|
||||
- **Browser/binary caches.** Cypress caches `~/.cache/Cypress` (cypress binary lives outside node_modules); playwright caches `~/.cache/ms-playwright` (chromium only). Both keyed on the framework's lockfile so they invalidate on version bumps.
|
||||
- **No retry plumbing in the template.** Test System IO handles per-spec retries; the workflow only sees aggregated results.
|
||||
|
||||
---
|
||||
|
||||
## Pipeline 1: PR (`e2e-tests-ci.yml`)
|
||||
|
||||
Runs E2E tests for every PR commit after the enterprise docker image is built. Fails if the commit is not associated with an open PR.
|
||||
|
||||
@@ -1,360 +0,0 @@
|
||||
---
|
||||
name: E2E Tests
|
||||
on:
|
||||
# For PRs, this workflow gets triggered from the Argo Events platform.
|
||||
# Check the following repo for details: https://github.com/mattermost/delivery-platform
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
ref:
|
||||
type: string
|
||||
description: Git ref to test. Must be a full commit SHA for PR testing, and a tag for release testing. Ignored for daily tests.
|
||||
required: false
|
||||
PR_NUMBER:
|
||||
type: string
|
||||
description: PR number (if applicable)
|
||||
required: false
|
||||
ROLLING_RELEASE_FROM_TAG:
|
||||
type: string
|
||||
description: Mattermost release git tag for RollingRelease tests. Optional.
|
||||
required: false
|
||||
MM_ENV:
|
||||
type: string
|
||||
required: false
|
||||
description: A comma-separated list of environment variables to set for the server. Spaces are not supported.
|
||||
MM_SERVICE_OVERRIDES:
|
||||
type: string
|
||||
required: false
|
||||
description: A comma-separated list of service overrides. E.g. "-elasticsearch,+opensearch"
|
||||
REPORT_TYPE:
|
||||
type: choice
|
||||
description: The context this report is being generated in
|
||||
options:
|
||||
- PR
|
||||
- RELEASE
|
||||
- RELEASE_CLOUD
|
||||
- MASTER
|
||||
- MASTER_UNSTABLE
|
||||
- CLOUD
|
||||
- CLOUD_UNSTABLE
|
||||
- NONE
|
||||
default: NONE
|
||||
RUN_CYPRESS:
|
||||
type: string
|
||||
description: Enable Cypress run
|
||||
default: "true"
|
||||
RUN_PLAYWRIGHT:
|
||||
type: string
|
||||
description: Enable Playwright run
|
||||
default: "true"
|
||||
FIPS_ENABLED:
|
||||
type: string
|
||||
description: When true, use mattermost-enterprise-fips-edition image for testing instead of standard enterprise edition
|
||||
default: "false"
|
||||
required: false
|
||||
|
||||
concurrency:
|
||||
group: "${{ github.workflow }}-${{ inputs.REPORT_TYPE }}-${{ inputs.FIPS_ENABLED }}-${{ inputs.PR_NUMBER || inputs.ref }}-${{ inputs.MM_ENV }}"
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
generate-test-variables:
|
||||
runs-on: ubuntu-24.04
|
||||
permissions:
|
||||
issues: write
|
||||
pull-requests: write
|
||||
defaults:
|
||||
run:
|
||||
shell: bash
|
||||
outputs:
|
||||
commit_sha: "${{ steps.generate.outputs.commit_sha }}"
|
||||
BRANCH: "${{ steps.generate.outputs.BRANCH }}"
|
||||
SERVER_IMAGE: "${{ steps.generate.outputs.SERVER_IMAGE }}"
|
||||
status_check_context: "${{ steps.generate.outputs.status_check_context }}"
|
||||
workers_number: "${{ steps.generate.outputs.workers_number }}"
|
||||
server_uppercase: "${{ steps.generate.outputs.server_uppercase }}" # Required for license selection
|
||||
SERVER: "${{ steps.generate.outputs.SERVER }}"
|
||||
ENABLED_DOCKER_SERVICES: "${{ steps.generate.outputs.ENABLED_DOCKER_SERVICES }}"
|
||||
TEST_FILTER_CYPRESS: "${{ steps.generate.outputs.TEST_FILTER_CYPRESS }}"
|
||||
TEST_FILTER_PLAYWRIGHT: "tests"
|
||||
BUILD_ID: "${{ steps.generate.outputs.BUILD_ID }}"
|
||||
TM4J_ENABLE: "${{ steps.generate.outputs.TM4J_ENABLE }}"
|
||||
REPORT_TYPE: "${{ steps.generate.outputs.REPORT_TYPE }}"
|
||||
TESTCASE_FAILURE_FATAL: "${{ steps.generate.outputs.TESTCASE_FAILURE_FATAL }}"
|
||||
ROLLING_RELEASE_commit_sha: "${{ steps.generate.outputs.ROLLING_RELEASE_commit_sha }}"
|
||||
ROLLING_RELEASE_SERVER_IMAGE: "${{ steps.generate.outputs.ROLLING_RELEASE_SERVER_IMAGE }}"
|
||||
WORKFLOW_RUN_URL: "${{steps.generate.outputs.WORKFLOW_RUN_URL}}"
|
||||
CYCLE_URL: "${{steps.generate.outputs.CYCLE_URL}}"
|
||||
FIPS_SUFFIX: "${{ steps.generate.outputs.FIPS_SUFFIX }}"
|
||||
env:
|
||||
GH_TOKEN: "${{ github.token }}"
|
||||
REF: "${{ inputs.ref || github.sha }}"
|
||||
PR_NUMBER: "${{ inputs.PR_NUMBER || '' }}"
|
||||
REPORT_TYPE: "${{ inputs.REPORT_TYPE }}"
|
||||
ROLLING_RELEASE_FROM_TAG: "${{ inputs.ROLLING_RELEASE_FROM_TAG }}"
|
||||
AUTOMATION_DASHBOARD_URL: "${{ secrets.MM_E2E_AUTOMATION_DASHBOARD_URL }}"
|
||||
FIPS_ENABLED: "${{ inputs.FIPS_ENABLED }}"
|
||||
# We could exclude the @smoke group for PRs, but then we wouldn't have it in the report
|
||||
TEST_FILTER_CYPRESS_PR: >-
|
||||
--stage="@prod"
|
||||
--excludeGroup="@te_only,@cloud_only,@high_availability"
|
||||
--sortFirst="@compliance_export,@elasticsearch,@ldap_group,@ldap"
|
||||
--sortLast="@saml,@keycloak,@plugin,@plugins_uninstall,@mfa,@license_removal"
|
||||
TEST_FILTER_CYPRESS_PROD_ONPREM: >-
|
||||
--stage="@prod"
|
||||
--excludeGroup="@te_only,@cloud_only,@high_availability"
|
||||
--sortFirst="@compliance_export,@elasticsearch,@ldap_group,@ldap,@playbooks"
|
||||
--sortLast="@saml,@keycloak,@plugin,@plugins_uninstall,@mfa,@license_removal"
|
||||
TEST_FILTER_CYPRESS_PROD_CLOUD: >-
|
||||
--stage="@prod"
|
||||
--excludeGroup="@not_cloud,@cloud_trial,@e20_only,@te_only,@high_availability,@license_removal"
|
||||
--sortFirst="@compliance_export,@elasticsearch,@ldap_group,@ldap,@playbooks"
|
||||
--sortLast="@saml,@keycloak,@plugin,@plugins_uninstall,@mfa"
|
||||
MM_ENV: "${{ inputs.MM_ENV || '' }}"
|
||||
MM_SERVICE_OVERRIDES: "${{ inputs.MM_SERVICE_OVERRIDES }}"
|
||||
steps:
|
||||
- name: ci/checkout-repo
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
ref: "${{ inputs.ref || github.sha }}"
|
||||
fetch-depth: 0
|
||||
- name: ci/generate-test-variables
|
||||
id: generate
|
||||
run: |
|
||||
MM_ENV_HASH=$(md5sum -z <<<"$MM_ENV" | cut -c-8)
|
||||
TESTCASE_FAILURE_FATAL="true"
|
||||
if grep -q CLOUD <<<"$REPORT_TYPE"; then
|
||||
SERVER=cloud
|
||||
else
|
||||
SERVER=onprem
|
||||
fi
|
||||
case "$REPORT_TYPE" in
|
||||
NONE | PR)
|
||||
### Populate support variables
|
||||
_COMMIT_SHA_COMPUTED=$(git rev-parse --verify "$REF") # NB: not actually used for resolving the commit; it's only to double check the value of 'inputs.ref'
|
||||
### For image tag generation: utilize 'inputs.ref', assume that it is a full commit SHA
|
||||
COMMIT_SHA="${REF}"
|
||||
BRANCH="server-pr-${PR_NUMBER}" # For reference, the real branch name may be retrievable with command: 'jq -r .head.ref <pr.json'
|
||||
SERVER_IMAGE_TAG="${COMMIT_SHA::7}"
|
||||
SERVER_IMAGE_ORG=mattermostdevelopment
|
||||
BUILD_ID_SUFFIX="${REPORT_TYPE@L}-${SERVER}-ent"
|
||||
WORKERS_NUMBER=20
|
||||
TEST_FILTER_CYPRESS="$TEST_FILTER_CYPRESS_PR"
|
||||
COMPUTED_REPORT_TYPE="${REPORT_TYPE}"
|
||||
### Run sanity assertions after variable generations
|
||||
[ "$REF" = "${_COMMIT_SHA_COMPUTED}" ] # 'inputs.ref' must be a full commit hash, and the commit must exist
|
||||
[ "$REPORT_TYPE" != "PR" ] || [ "$PR_NUMBER" -gt "0" ] # If report type is PR, then PR_NUMBER must be set to a number
|
||||
;;
|
||||
MASTER | MASTER_UNSTABLE | CLOUD | CLOUD_UNSTABLE)
|
||||
### Populate support variables
|
||||
_IS_TEST_UNSTABLE=$(sed -n -E 's/^.*(UNSTABLE).*$/\1/p' <<< "$REPORT_TYPE") # The variable's value is 'UNSTABLE' if report type is for unstable tests, otherwise it's empty
|
||||
_TEST_FILTER_CYPRESS_VARIABLE="TEST_FILTER_CYPRESS_PROD_${SERVER@U}"
|
||||
### For ref and image tag generation: ignore 'inputs.ref', and use master branch directly. Note that 'COMMIT_SHA' will be used for reporting the test result, and for checking out the testing scripts and test cases
|
||||
COMMIT_SHA="$(git rev-parse --verify origin/master)"
|
||||
BRANCH=master
|
||||
SERVER_IMAGE_TAG=master
|
||||
SERVER_IMAGE_ORG=mattermostdevelopment
|
||||
BUILD_ID_SUFFIX="${_IS_TEST_UNSTABLE:+unstable-}daily-${SERVER}-ent"
|
||||
BUILD_ID_SUFFIX_IN_STATUS_CHECK=true
|
||||
WORKERS_NUMBER=10 # Daily tests are not time critical, and it's more efficient to run on fewer workers
|
||||
TEST_FILTER_CYPRESS="${!_TEST_FILTER_CYPRESS_VARIABLE} ${_IS_TEST_UNSTABLE:+--invert}"
|
||||
TM4J_ENABLE=true
|
||||
COMPUTED_REPORT_TYPE="${REPORT_TYPE}"
|
||||
[ -z "$_IS_TEST_UNSTABLE" ] || TESTCASE_FAILURE_FATAL="" # Assert that tests are stable. If they are not, the status check will be always green
|
||||
;;
|
||||
RELEASE | RELEASE_CLOUD)
|
||||
### Populate support variables
|
||||
_TEST_FILTER_CYPRESS_VARIABLE="TEST_FILTER_CYPRESS_PROD_${SERVER@U}"
|
||||
### For ref and image tag generation: assume the 'inputs.ref' is a tag, and use the first two digits to construct the branch name
|
||||
COMMIT_SHA="$(git rev-parse --verify HEAD)"
|
||||
BRANCH=$(sed -E "s/v([0-9]+)\.([0-9]+)\..+$/release-\1.\2/g" <<<$REF)
|
||||
SERVER_IMAGE_TAG="$(cut -c2- <<<$REF)" # Remove the leading 'v' from the given tag name, to generate the docker image tag
|
||||
SERVER_IMAGE_ORG=mattermost
|
||||
BUILD_ID_SUFFIX="release-${SERVER}-ent"
|
||||
BUILD_ID_SUFFIX_IN_STATUS_CHECK=true
|
||||
WORKERS_NUMBER=20
|
||||
TEST_FILTER_CYPRESS="${!_TEST_FILTER_CYPRESS_VARIABLE}"
|
||||
TM4J_ENABLE=true
|
||||
COMPUTED_REPORT_TYPE=RELEASE
|
||||
### Run sanity assertions after variable generations
|
||||
git show-ref --verify "refs/tags/${REF}" # 'inputs.ref' must be a tag, for release report types
|
||||
git show-ref --verify "refs/remotes/origin/${BRANCH}" # The release branch computed from the given tag must exist
|
||||
;;
|
||||
*)
|
||||
echo "Fatal: unimplemented test type. Aborting."
|
||||
exit 1
|
||||
esac
|
||||
if [ -n "$ROLLING_RELEASE_FROM_TAG" ]; then
|
||||
ROLLING_RELEASE_COMMIT_SHA=$(git rev-parse --verify "$ROLLING_RELEASE_FROM_TAG")
|
||||
ROLLING_RELEASE_SERVER_IMAGE_TAG=$(echo "$ROLLING_RELEASE_FROM_TAG" | sed 's/^v//') # Remove the leading 'v' from the given tag name, to generate the docker image tag
|
||||
ROLLING_RELEASE_SERVER_IMAGE="mattermost/mattermost-enterprise-edition:${ROLLING_RELEASE_SERVER_IMAGE_TAG}"
|
||||
BUILD_ID_SUFFIX="rolling${ROLLING_RELEASE_FROM_TAG/-/_}-$BUILD_ID_SUFFIX"
|
||||
BUILD_ID_SUFFIX_IN_STATUS_CHECK=true
|
||||
WORKERS_NUMBER=10 # Rolling release tests are particularly impacted by increased parallelism. It's more efficient to run on fewer workers
|
||||
### Run sanity assertions after variable generations
|
||||
git show-ref --verify "refs/tags/${ROLLING_RELEASE_FROM_TAG}" # 'inputs.ROLLING_RELEASE_FROM_TAG' must be a tag, for release report types
|
||||
fi
|
||||
ENABLED_DOCKER_SERVICES="postgres inbucket minio openldap elasticsearch keycloak"
|
||||
for SVC_OP in $(tr , ' '<<<"$MM_SERVICE_OVERRIDES"); do
|
||||
OP=$(cut -c1 <<<$SVC_OP)
|
||||
SVC=$(cut -c2- <<<$SVC_OP)
|
||||
case "$OP" in
|
||||
"+") ENABLED_DOCKER_SERVICES="$ENABLED_DOCKER_SERVICES $SVC" ;;
|
||||
"-") ENABLED_DOCKER_SERVICES=$(sed -E "s:(^| )${SVC}( |\$): :g" <<<"$ENABLED_DOCKER_SERVICES") ;;
|
||||
*) echo "Invalid MM_SERVICE_OVERRIDE value: $SVC_OP"; exit 1 ;;
|
||||
esac
|
||||
done
|
||||
# Determine server image name and FIPS suffix based on FIPS_ENABLED parameter
|
||||
if [ "$FIPS_ENABLED" = "true" ]; then
|
||||
SERVER_IMAGE_NAME="mattermost-enterprise-fips-edition"
|
||||
FIPS_SUFFIX="_fips"
|
||||
else
|
||||
SERVER_IMAGE_NAME="mattermost-enterprise-edition"
|
||||
FIPS_SUFFIX=""
|
||||
fi
|
||||
# BUILD_ID format: $pipelineID-$imageTag-$testType-$serverType-$serverEdition
|
||||
# Reference on BUILD_ID parsing: https://github.com/saturninoabril/automation-dashboard/blob/175891781bf1072c162c58c6ec0abfc5bcb3520e/lib/common_utils.ts#L3-L23
|
||||
BUILD_ID="${{ github.run_id }}_${{ github.run_attempt }}-${SERVER_IMAGE_TAG}${FIPS_SUFFIX}-${BUILD_ID_SUFFIX}"
|
||||
echo "commit_sha=${COMMIT_SHA}" >> $GITHUB_OUTPUT
|
||||
echo "BRANCH=${BRANCH}" >> $GITHUB_OUTPUT
|
||||
echo "SERVER_IMAGE=${SERVER_IMAGE_ORG}/${SERVER_IMAGE_NAME}:${SERVER_IMAGE_TAG}" >> $GITHUB_OUTPUT
|
||||
echo "FIPS_SUFFIX=${FIPS_SUFFIX}" >> $GITHUB_OUTPUT
|
||||
echo "SERVER=${SERVER}" >> $GITHUB_OUTPUT
|
||||
echo "server_uppercase=${SERVER@U}" >> $GITHUB_OUTPUT
|
||||
echo "ENABLED_DOCKER_SERVICES=${ENABLED_DOCKER_SERVICES}" >> $GITHUB_OUTPUT
|
||||
echo "status_check_context=E2E Tests/test${FIPS_SUFFIX}${BUILD_ID_SUFFIX_IN_STATUS_CHECK:+-$BUILD_ID_SUFFIX}${MM_ENV:+/$MM_ENV_HASH}" >> $GITHUB_OUTPUT
|
||||
echo "workers_number=${WORKERS_NUMBER}" >> $GITHUB_OUTPUT
|
||||
echo "TEST_FILTER_CYPRESS=${TEST_FILTER_CYPRESS}" >> $GITHUB_OUTPUT
|
||||
echo "TESTCASE_FAILURE_FATAL=${TESTCASE_FAILURE_FATAL}" >> $GITHUB_OUTPUT
|
||||
echo "TM4J_ENABLE=${TM4J_ENABLE:-}" >> $GITHUB_OUTPUT
|
||||
echo "REPORT_TYPE=${COMPUTED_REPORT_TYPE}" >> $GITHUB_OUTPUT
|
||||
echo "ROLLING_RELEASE_commit_sha=${ROLLING_RELEASE_COMMIT_SHA}" >> $GITHUB_OUTPUT
|
||||
echo "ROLLING_RELEASE_SERVER_IMAGE=${ROLLING_RELEASE_SERVER_IMAGE}" >> $GITHUB_OUTPUT
|
||||
echo "BUILD_ID=${BUILD_ID}" >> $GITHUB_OUTPUT
|
||||
# User notification variables
|
||||
echo "WORKFLOW_RUN_URL=${{ github.server_url }}/${{ github.repository }}/actions/runs/${{github.run_id}}" >> $GITHUB_OUTPUT
|
||||
echo "CYCLE_URL=${AUTOMATION_DASHBOARD_URL%%/api}/cycle/${BUILD_ID}" >> $GITHUB_OUTPUT
|
||||
- name: ci/notify-user
|
||||
env:
|
||||
COMMIT_SHA: "${{steps.generate.outputs.commit_sha}}"
|
||||
STATUS_CHECK_CONTEXT: "${{steps.generate.outputs.status_check_context}}"
|
||||
WORKFLOW_RUN_URL: "${{steps.generate.outputs.WORKFLOW_RUN_URL}}"
|
||||
CYCLE_URL: "${{steps.generate.outputs.CYCLE_URL}}"
|
||||
RUN_CYPRESS: "${{inputs.RUN_CYPRESS == 'true' || ''}}"
|
||||
RUN_PLAYWRIGHT: "${{inputs.RUN_PLAYWRIGHT == 'true' || ''}}"
|
||||
run: |
|
||||
if [ -n "$PR_NUMBER" ]; then
|
||||
gh issue -R "${{ github.repository }}" comment "$PR_NUMBER" --body-file - <<EOF
|
||||
E2E test run is starting for commit \`${COMMIT_SHA}\`${MM_ENV:+, with \`MM_ENV=$MM_ENV\`}${MM_SERVICE_OVERRIDES:+, Cypress service overrides \`$MM_SERVICE_OVERRIDES\`}.
|
||||
To check the run progress:
|
||||
- Cypress: ${RUN_CYPRESS:+look for commit status \`$STATUS_CHECK_CONTEXT\` or the access the [Automation Dashboard Cycle URL]($CYCLE_URL)}$([ -n "${RUN_CYPRESS:-}" ] || echo -n "will not run").
|
||||
- Playwright: ${RUN_PLAYWRIGHT:+look for commit status \`$STATUS_CHECK_CONTEXT-playwright\`}$([ -n "${RUN_PLAYWRIGHT:-}" ] || echo -n "will not run").
|
||||
|
||||
You can also look at the [E2E test's Workflow Run URL]($WORKFLOW_RUN_URL) (run ID \`${{ github.run_id }}\`).
|
||||
EOF
|
||||
fi
|
||||
|
||||
e2e-fulltest-cypress:
|
||||
needs:
|
||||
- generate-test-variables
|
||||
uses: ./.github/workflows/e2e-tests-ci-template.yml
|
||||
if: ${{ inputs.RUN_CYPRESS == 'true' }}
|
||||
with:
|
||||
commit_sha: "${{ needs.generate-test-variables.outputs.commit_sha }}"
|
||||
status_check_context: "${{ needs.generate-test-variables.outputs.status_check_context }}"
|
||||
workers_number: "${{ needs.generate-test-variables.outputs.workers_number }}"
|
||||
testcase_failure_fatal: "${{ needs.generate-test-variables.outputs.TESTCASE_FAILURE_FATAL == 'true' }}"
|
||||
enable_reporting: true
|
||||
SERVER: "${{ needs.generate-test-variables.outputs.SERVER }}"
|
||||
SERVER_IMAGE: "${{ needs.generate-test-variables.outputs.SERVER_IMAGE }}"
|
||||
ENABLED_DOCKER_SERVICES: "${{ needs.generate-test-variables.outputs.ENABLED_DOCKER_SERVICES }}"
|
||||
TEST: "cypress"
|
||||
TEST_FILTER: "${{ needs.generate-test-variables.outputs.TEST_FILTER_CYPRESS }}"
|
||||
MM_ENV: "${{ inputs.MM_ENV || '' }}"
|
||||
BRANCH: "${{ needs.generate-test-variables.outputs.BRANCH }}"
|
||||
BUILD_ID: "${{ needs.generate-test-variables.outputs.BUILD_ID }}"
|
||||
REPORT_TYPE: "${{ needs.generate-test-variables.outputs.REPORT_TYPE }}"
|
||||
ROLLING_RELEASE_commit_sha: "${{ needs.generate-test-variables.outputs.ROLLING_RELEASE_commit_sha }}"
|
||||
ROLLING_RELEASE_SERVER_IMAGE: "${{ needs.generate-test-variables.outputs.ROLLING_RELEASE_SERVER_IMAGE }}"
|
||||
PR_NUMBER: "${{ inputs.PR_NUMBER }}"
|
||||
secrets:
|
||||
MM_LICENSE: "${{ secrets[format('MM_E2E_TEST_LICENSE_{0}_ENT', needs.generate-test-variables.outputs.server_uppercase)] }}"
|
||||
AUTOMATION_DASHBOARD_URL: "${{ secrets.MM_E2E_AUTOMATION_DASHBOARD_URL }}"
|
||||
AUTOMATION_DASHBOARD_TOKEN: "${{ secrets.MM_E2E_AUTOMATION_DASHBOARD_TOKEN }}"
|
||||
PUSH_NOTIFICATION_SERVER: "${{ secrets.MM_E2E_PUSH_NOTIFICATION_SERVER }}"
|
||||
REPORT_WEBHOOK_URL: "${{ secrets.MM_E2E_REPORT_WEBHOOK_URL }}"
|
||||
REPORT_TM4J_API_KEY: "${{ needs.generate-test-variables.outputs.TM4J_ENABLE == 'true' && secrets.MM_E2E_TM4J_API_KEY || '' }}"
|
||||
REPORT_TM4J_TEST_CYCLE_LINK_PREFIX: "${{ secrets.MM_E2E_TEST_CYCLE_LINK_PREFIX }}"
|
||||
CWS_URL: "${{ needs.generate-test-variables.outputs.SERVER == 'cloud' && secrets.MM_E2E_CWS_URL || '' }}"
|
||||
CWS_EXTRA_HTTP_HEADERS: "${{ needs.generate-test-variables.outputs.SERVER == 'cloud' && secrets.MM_E2E_CWS_EXTRA_HTTP_HEADERS || '' }}"
|
||||
AWS_ACCESS_KEY_ID: "${{ secrets.CYPRESS_AWS_ACCESS_KEY_ID }}"
|
||||
AWS_SECRET_ACCESS_KEY: "${{ secrets.CYPRESS_AWS_SECRET_ACCESS_KEY }}"
|
||||
|
||||
e2e-fulltest-playwright:
|
||||
needs:
|
||||
- generate-test-variables
|
||||
uses: ./.github/workflows/e2e-tests-ci-template.yml
|
||||
if: ${{ inputs.RUN_PLAYWRIGHT == 'true' }}
|
||||
with:
|
||||
commit_sha: "${{ needs.generate-test-variables.outputs.commit_sha }}"
|
||||
status_check_context: "${{ needs.generate-test-variables.outputs.status_check_context }}-playwright"
|
||||
workers_number: "1"
|
||||
testcase_failure_fatal: "${{ needs.generate-test-variables.outputs.TESTCASE_FAILURE_FATAL == 'true' }}"
|
||||
enable_reporting: true
|
||||
SERVER: "${{ needs.generate-test-variables.outputs.SERVER }}"
|
||||
SERVER_IMAGE: "${{ needs.generate-test-variables.outputs.SERVER_IMAGE }}"
|
||||
TEST: "playwright"
|
||||
TEST_FILTER: "${{ needs.generate-test-variables.outputs.TEST_FILTER_PLAYWRIGHT }}"
|
||||
MM_ENV: "${{ inputs.MM_ENV || '' }}"
|
||||
BRANCH: "${{ needs.generate-test-variables.outputs.BRANCH }}"
|
||||
BUILD_ID: "${{ needs.generate-test-variables.outputs.BUILD_ID }}"
|
||||
REPORT_TYPE: "${{ needs.generate-test-variables.outputs.REPORT_TYPE }}"
|
||||
ROLLING_RELEASE_commit_sha: "${{ needs.generate-test-variables.outputs.ROLLING_RELEASE_commit_sha }}"
|
||||
ROLLING_RELEASE_SERVER_IMAGE: "${{ needs.generate-test-variables.outputs.ROLLING_RELEASE_SERVER_IMAGE }}"
|
||||
PR_NUMBER: "${{ inputs.PR_NUMBER }}"
|
||||
secrets:
|
||||
MM_LICENSE: "${{ secrets[format('MM_E2E_TEST_LICENSE_{0}_ENT', needs.generate-test-variables.outputs.server_uppercase)] }}"
|
||||
PUSH_NOTIFICATION_SERVER: "${{ secrets.MM_E2E_PUSH_NOTIFICATION_SERVER }}"
|
||||
REPORT_WEBHOOK_URL: "${{ secrets.MM_E2E_REPORT_WEBHOOK_URL }}"
|
||||
CWS_URL: "${{ needs.generate-test-variables.outputs.SERVER == 'cloud' && secrets.MM_E2E_CWS_URL || '' }}"
|
||||
CWS_EXTRA_HTTP_HEADERS: "${{ needs.generate-test-variables.outputs.SERVER == 'cloud' && secrets.MM_E2E_CWS_EXTRA_HTTP_HEADERS || '' }}"
|
||||
AWS_ACCESS_KEY_ID: "${{ secrets.CYPRESS_AWS_ACCESS_KEY_ID }}"
|
||||
AWS_SECRET_ACCESS_KEY: "${{ secrets.CYPRESS_AWS_SECRET_ACCESS_KEY }}"
|
||||
|
||||
notify-user:
|
||||
runs-on: ubuntu-24.04
|
||||
if: always()
|
||||
needs:
|
||||
- generate-test-variables
|
||||
- e2e-fulltest-cypress
|
||||
- e2e-fulltest-playwright
|
||||
permissions:
|
||||
issues: write
|
||||
pull-requests: write
|
||||
defaults:
|
||||
run:
|
||||
shell: bash
|
||||
env:
|
||||
GH_TOKEN: "${{ github.token }}"
|
||||
PR_NUMBER: "${{ inputs.PR_NUMBER || '' }}"
|
||||
MM_ENV: "${{ inputs.MM_ENV || '' }}"
|
||||
COMMIT_SHA: "${{ needs.generate-test-variables.outputs.commit_sha }}"
|
||||
STATUS_CHECK_CONTEXT: "${{ needs.generate-test-variables.outputs.status_check_context }}"
|
||||
WORKFLOW_RUN_URL: "${{ needs.generate-test-variables.outputs.WORKFLOW_RUN_URL }}"
|
||||
CYCLE_URL: "${{ needs.generate-test-variables.outputs.CYCLE_URL }}"
|
||||
RUN_CYPRESS: "${{inputs.RUN_CYPRESS == 'true' || ''}}"
|
||||
RUN_PLAYWRIGHT: "${{inputs.RUN_PLAYWRIGHT == 'true' || ''}}"
|
||||
PLAYWRIGHT_REPORT_URL: "${{ needs.e2e-fulltest-playwright.outputs.playwright_report_url }}"
|
||||
steps:
|
||||
- name: ci/notify-user-test-completion
|
||||
run: |
|
||||
if [ -n "$PR_NUMBER" ]; then
|
||||
gh issue -R "${{ github.repository }}" comment "$PR_NUMBER" --body-file - <<EOF
|
||||
E2E test has completed for commit \`${COMMIT_SHA}\`${MM_ENV:+, with \`MM_ENV=$MM_ENV\`}.
|
||||
Results summary:
|
||||
- Cypress: ${RUN_CYPRESS:+pass rate is \`${{ needs.e2e-fulltest-cypress.outputs.pass_rate || 'unknown' }}\` (see [Automation Dashboard]($CYCLE_URL) and commit status check \`$STATUS_CHECK_CONTEXT\`)}$([ -n "${RUN_CYPRESS:-}" ] || echo -n "did not run").
|
||||
- Playwright: ${RUN_PLAYWRIGHT:+pass rate is \`${{ needs.e2e-fulltest-playwright.outputs.pass_rate || 'unknown' }}\` (see [Playwright Report URL]($PLAYWRIGHT_REPORT_URL) and commit status check \`$STATUS_CHECK_CONTEXT-playwright\`)}$([ -n "${RUN_PLAYWRIGHT:-}" ] || echo -n "did not run").
|
||||
|
||||
The run summary artifacts are available in the corresponding [Workflow Run]($WORKFLOW_RUN_URL).
|
||||
EOF
|
||||
fi
|
||||
@@ -1,518 +0,0 @@
|
||||
---
|
||||
name: E2E Tests Template
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
# NB: this does not support using branch names that belong to forks.
|
||||
# In those cases, you should specify directly the commit SHA that you want to test, or
|
||||
# some wrapper workflow that does it for you (e.g. the slash command for initiating a PR test)
|
||||
commit_sha:
|
||||
type: string
|
||||
required: true
|
||||
status_check_context:
|
||||
type: string
|
||||
required: true
|
||||
workers_number:
|
||||
type: string # Should ideally be a number; see https://github.com/orgs/community/discussions/67182
|
||||
required: false
|
||||
default: "1"
|
||||
testcase_failure_fatal:
|
||||
type: boolean
|
||||
required: false
|
||||
default: true
|
||||
enable_reporting:
|
||||
type: boolean
|
||||
required: false
|
||||
default: false
|
||||
SERVER:
|
||||
type: string # Valid values are: onprem, cloud
|
||||
required: false
|
||||
default: onprem
|
||||
SERVER_IMAGE:
|
||||
type: string
|
||||
required: false
|
||||
ENABLED_DOCKER_SERVICES:
|
||||
type: string
|
||||
required: false
|
||||
TEST: # Valid values are: cypress, playwright
|
||||
type: string
|
||||
required: false
|
||||
default: "cypress"
|
||||
TEST_FILTER:
|
||||
type: string
|
||||
required: false
|
||||
MM_ENV:
|
||||
type: string
|
||||
required: false
|
||||
BRANCH:
|
||||
type: string
|
||||
required: false
|
||||
BUILD_ID:
|
||||
type: string
|
||||
required: false
|
||||
REPORT_TYPE:
|
||||
type: string
|
||||
required: false
|
||||
ROLLING_RELEASE_commit_sha:
|
||||
type: string
|
||||
required: false
|
||||
ROLLING_RELEASE_SERVER_IMAGE:
|
||||
type: string
|
||||
required: false
|
||||
PR_NUMBER:
|
||||
type: string
|
||||
required: false
|
||||
secrets:
|
||||
MM_LICENSE:
|
||||
required: false
|
||||
AUTOMATION_DASHBOARD_URL:
|
||||
required: false
|
||||
AUTOMATION_DASHBOARD_TOKEN:
|
||||
required: false
|
||||
PUSH_NOTIFICATION_SERVER:
|
||||
required: false
|
||||
REPORT_WEBHOOK_URL:
|
||||
required: false
|
||||
REPORT_TM4J_API_KEY:
|
||||
required: false
|
||||
REPORT_TM4J_TEST_CYCLE_LINK_PREFIX:
|
||||
required: false
|
||||
CWS_URL:
|
||||
required: false
|
||||
CWS_EXTRA_HTTP_HEADERS:
|
||||
required: false
|
||||
AWS_ACCESS_KEY_ID:
|
||||
required: false
|
||||
AWS_SECRET_ACCESS_KEY:
|
||||
required: false
|
||||
outputs:
|
||||
passed:
|
||||
value: "${{ jobs.report.outputs.passed }}"
|
||||
failed:
|
||||
value: "${{ jobs.report.outputs.failed }}"
|
||||
failed_expected:
|
||||
value: "${{ jobs.report.outputs.failed_expected }}"
|
||||
pass_rate:
|
||||
value: "${{ jobs.report.outputs.pass_rate }}"
|
||||
playwright_report_url:
|
||||
value: ${{ jobs.report.outputs.playwright_report_url }}
|
||||
|
||||
jobs:
|
||||
update-initial-status:
|
||||
runs-on: ubuntu-24.04
|
||||
steps:
|
||||
- uses: mattermost/actions/delivery/update-commit-status@f324ac89b05cc3511cb06e60642ac2fb829f0a63
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ github.token }}
|
||||
with:
|
||||
repository_full_name: ${{ github.repository }}
|
||||
commit_sha: ${{ inputs.commit_sha }}
|
||||
context: ${{ inputs.status_check_context }}
|
||||
description: E2E tests for mattermost server app
|
||||
status: pending
|
||||
|
||||
generate-build-variables:
|
||||
runs-on: ubuntu-24.04
|
||||
needs:
|
||||
- update-initial-status
|
||||
defaults:
|
||||
run:
|
||||
shell: bash
|
||||
outputs:
|
||||
workers: "${{ steps.generate.outputs.workers }}"
|
||||
steps:
|
||||
- name: ci/checkout-repo
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
ref: ${{ inputs.commit_sha }}
|
||||
fetch-depth: 0
|
||||
- name: ci/generate-build-variables
|
||||
id: generate
|
||||
env:
|
||||
WORKERS: ${{ inputs.workers_number }}
|
||||
run: |
|
||||
[ "$WORKERS" -gt "0" ] # Assert that the workers number is an integer greater than 0
|
||||
echo "workers="$(jq --slurp --compact-output '[range('"$WORKERS"')] | map(tostring)' /dev/null) >> $GITHUB_OUTPUT
|
||||
|
||||
generate-test-cycle:
|
||||
runs-on: ubuntu-24.04
|
||||
needs:
|
||||
- generate-build-variables
|
||||
defaults:
|
||||
run:
|
||||
shell: bash
|
||||
working-directory: e2e-tests
|
||||
outputs:
|
||||
status_check_url: "${{ steps.e2e-test-gencycle.outputs.status_check_url }}"
|
||||
steps:
|
||||
- name: ci/checkout-repo
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
ref: ${{ inputs.commit_sha }}
|
||||
fetch-depth: 0
|
||||
- name: ci/restore-npm-cache
|
||||
uses: ./.github/actions/restore-e2e-npm-cache
|
||||
- name: ci/setup-node
|
||||
uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
|
||||
id: setup_node
|
||||
with:
|
||||
node-version-file: ".nvmrc"
|
||||
- name: ci/e2e-test-gencycle
|
||||
id: e2e-test-gencycle
|
||||
env:
|
||||
AUTOMATION_DASHBOARD_URL: "${{ secrets.AUTOMATION_DASHBOARD_URL }}"
|
||||
AUTOMATION_DASHBOARD_TOKEN: "${{ secrets.AUTOMATION_DASHBOARD_TOKEN }}"
|
||||
BRANCH: "${{ inputs.BRANCH }}"
|
||||
BUILD_ID: "${{ inputs.BUILD_ID }}"
|
||||
TEST: "${{ inputs.TEST }}"
|
||||
TEST_FILTER: "${{ inputs.TEST_FILTER }}"
|
||||
run: |
|
||||
set -e -o pipefail
|
||||
make generate-test-cycle | tee generate-test-cycle.out
|
||||
# Extract cycle's dashboard URL, if present
|
||||
TEST_CYCLE_ID=$(sed -nE "s/^.*id: '([^']+)'.*$/\1/p" <generate-test-cycle.out)
|
||||
if [ -n "$TEST_CYCLE_ID" ]; then
|
||||
echo "status_check_url=https://automation-dashboard.vercel.app/cycles/${TEST_CYCLE_ID}" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "status_check_url=${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
|
||||
test:
|
||||
continue-on-error: true # Individual runner failures shouldn't prevent the completion of an E2E run
|
||||
strategy:
|
||||
fail-fast: false # Individual runner failures shouldn't prevent the completion of an E2E run
|
||||
matrix:
|
||||
#
|
||||
# Note that E2E tests should be run only on ubuntu, for QA purposes.
|
||||
# But it's useful to be able to run and debug the E2E tests for different OSes.
|
||||
# Notes:
|
||||
# - For MacOS: works on developer machines, but uses too many resources to be able to run on Github Actions
|
||||
# - for Windows: cannot currently run on Github Actions, since the runners do not support running linux containers, at the moment
|
||||
#
|
||||
#os: [ubuntu-24.04, windows-2022, macos-12-xl]
|
||||
os: [ubuntu-24.04]
|
||||
worker_index: ${{ fromJSON(needs.generate-build-variables.outputs.workers) }} # https://docs.github.com/en/actions/learn-github-actions/expressions#example-returning-a-json-object
|
||||
runs-on: "${{ matrix.os }}"
|
||||
timeout-minutes: 120
|
||||
needs:
|
||||
- generate-build-variables
|
||||
- generate-test-cycle
|
||||
defaults:
|
||||
run:
|
||||
shell: bash
|
||||
working-directory: e2e-tests
|
||||
env:
|
||||
AUTOMATION_DASHBOARD_URL: "${{ secrets.AUTOMATION_DASHBOARD_URL }}"
|
||||
AUTOMATION_DASHBOARD_TOKEN: "${{ secrets.AUTOMATION_DASHBOARD_TOKEN }}"
|
||||
SERVER: "${{ inputs.SERVER }}"
|
||||
SERVER_IMAGE: "${{ inputs.SERVER_IMAGE }}"
|
||||
MM_LICENSE: "${{ secrets.MM_LICENSE }}"
|
||||
ENABLED_DOCKER_SERVICES: "${{ inputs.ENABLED_DOCKER_SERVICES }}"
|
||||
TEST: "${{ inputs.TEST }}"
|
||||
TEST_FILTER: "${{ inputs.TEST_FILTER }}"
|
||||
MM_ENV: "${{ inputs.MM_ENV }}"
|
||||
BRANCH: "${{ inputs.BRANCH }}"
|
||||
BUILD_ID: "${{ inputs.BUILD_ID }}"
|
||||
CI_BASE_URL: "${{ matrix.os }}-${{ matrix.worker_index }}"
|
||||
CYPRESS_pushNotificationServer: "${{ secrets.PUSH_NOTIFICATION_SERVER }}"
|
||||
CWS_URL: "${{ secrets.CWS_URL }}"
|
||||
CWS_EXTRA_HTTP_HEADERS: "${{ secrets.CWS_EXTRA_HTTP_HEADERS }}"
|
||||
ROLLING_RELEASE_COMMIT_SHA: "${{ inputs.ROLLING_RELEASE_commit_sha }}"
|
||||
ROLLING_RELEASE_SERVER_IMAGE: "${{ inputs.ROLLING_RELEASE_SERVER_IMAGE }}"
|
||||
steps:
|
||||
- name: ci/checkout-actions
|
||||
# Sparse-checkout just .github/actions from the triggering ref (master)
|
||||
# so the composite action below is available before the full checkout
|
||||
# overwrites the workspace with inputs.commit_sha.
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
sparse-checkout: .github/actions
|
||||
sparse-checkout-cone-mode: true
|
||||
- name: ci/runner-prep-for-openldap
|
||||
uses: ./.github/actions/runner-prep-openldap
|
||||
- name: ci/checkout-repo
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
ref: ${{ inputs.commit_sha }}
|
||||
fetch-depth: 0
|
||||
- name: ci/setup-macos-docker
|
||||
if: runner.os == 'macos'
|
||||
# https://github.com/actions/runner-images/issues/17#issuecomment-1537238473
|
||||
run: |
|
||||
brew install docker docker-compose
|
||||
colima start
|
||||
mkdir -p ~/.docker/cli-plugins
|
||||
ln -sfn /usr/local/opt/docker-compose/bin/docker-compose ~/.docker/cli-plugins/docker-compose
|
||||
sudo ln -sf $HOME/.colima/default/docker.sock /var/run/docker.sock
|
||||
- name: ci/restore-npm-cache
|
||||
uses: ./.github/actions/restore-e2e-npm-cache
|
||||
- name: ci/setup-node
|
||||
uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
|
||||
id: setup_node
|
||||
with:
|
||||
node-version-file: ".nvmrc"
|
||||
- name: ci/e2e-test
|
||||
run: |
|
||||
make cloud-init
|
||||
if [ -n "$ROLLING_RELEASE_SERVER_IMAGE" ]; then
|
||||
echo "RollingRelease: checking out E2E test cases from revision ${ROLLING_RELEASE_COMMIT_SHA}, for initial smoketest"
|
||||
git checkout "${ROLLING_RELEASE_COMMIT_SHA}" -- "${TEST}/" && git status
|
||||
(
|
||||
echo "RollingRelease: running initial smoketest against image $ROLLING_RELEASE_SERVER_IMAGE"
|
||||
export SERVER_IMAGE="$ROLLING_RELEASE_SERVER_IMAGE"
|
||||
export TEST_FILTER=""
|
||||
export AUTOMATION_DASHBOARD_URL=""
|
||||
make
|
||||
)
|
||||
echo "RollingRelease: asserting smoketest result has zero failures."
|
||||
FAILURES=$(jq -r '.failed' "${TEST}/results/summary.json")
|
||||
if [ "$FAILURES" -ne "0" ]; then
|
||||
echo "RollingRelease: initial smoketest for rolling release E2E run has nonzero ($FAILURES) failures. Aborting test run." >&2
|
||||
exit 1
|
||||
fi
|
||||
rm -rfv "${TEST}/{results,logs}"
|
||||
echo "RollingRelease: reset the E2E test cases to the revision to test"
|
||||
git reset --hard HEAD && git status
|
||||
echo "RollingRelease: smoketest completed. Starting full E2E tests."
|
||||
fi
|
||||
make
|
||||
- name: ci/cloud-teardown
|
||||
if: always()
|
||||
run: make cloud-teardown
|
||||
- name: ci/dump-docker-state-on-failure
|
||||
# Always run a final docker-state capture so failures unrelated to
|
||||
# openldap startup (e.g. server container later crashes) still produce
|
||||
# logs we can inspect. The script's own retry loop dumps openldap
|
||||
# state per-attempt; this step is a backstop covering the whole job.
|
||||
if: failure()
|
||||
run: |
|
||||
set +e
|
||||
DIAG="e2e-tests/docker-diagnostics/job-failure"
|
||||
mkdir -p "$DIAG"
|
||||
docker ps -a >"$DIAG/docker.ps.txt" 2>&1
|
||||
docker version >"$DIAG/docker.version.txt" 2>&1
|
||||
docker info >"$DIAG/docker.info.txt" 2>&1
|
||||
for c in $(docker ps -a --format '{{.Names}}'); do
|
||||
docker inspect "$c" >"$DIAG/$c.inspect.json" 2>&1
|
||||
docker logs "$c" >"$DIAG/$c.log" 2>&1
|
||||
done
|
||||
uname -a >"$DIAG/host.uname.txt" 2>&1
|
||||
free -m >"$DIAG/host.free.txt" 2>&1
|
||||
df -h >"$DIAG/host.df.txt" 2>&1
|
||||
sudo dmesg | tail -500 >"$DIAG/host.dmesg.tail.txt" 2>&1
|
||||
sudo dmesg | grep -iE 'apparmor|denied|oom|killed|openldap|slapd' >"$DIAG/host.dmesg.relevant.txt" 2>&1
|
||||
- name: ci/upload-docker-diagnostics
|
||||
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
|
||||
if: always()
|
||||
with:
|
||||
name: docker-diagnostics-${{ inputs.TEST }}-${{ matrix.os }}-${{ matrix.worker_index }}
|
||||
path: e2e-tests/docker-diagnostics/
|
||||
retention-days: 7
|
||||
if-no-files-found: ignore
|
||||
- name: ci/e2e-test-store-results
|
||||
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
|
||||
if: always()
|
||||
with:
|
||||
name: e2e-test-results-${{ inputs.TEST }}-${{ matrix.os }}-${{ matrix.worker_index }}
|
||||
path: |
|
||||
e2e-tests/${{ inputs.TEST }}/logs/
|
||||
e2e-tests/${{ inputs.TEST }}/results/
|
||||
retention-days: 1
|
||||
|
||||
report:
|
||||
runs-on: ubuntu-24.04
|
||||
needs:
|
||||
- test
|
||||
- generate-build-variables
|
||||
defaults:
|
||||
run:
|
||||
shell: bash
|
||||
working-directory: e2e-tests
|
||||
outputs:
|
||||
passed: "${{ steps.calculate-results.outputs.passed }}"
|
||||
failed: "${{ steps.calculate-results.outputs.failed }}"
|
||||
failed_expected: "${{ steps.calculate-results.outputs.failed_expected }}"
|
||||
pass_rate: "${{ steps.calculate-results.outputs.pass_rate }}"
|
||||
commit_status_message: "${{ steps.calculate-results.outputs.commit_status_message }}"
|
||||
playwright_report_url: "${{ steps.upload-to-s3.outputs.report_url }}"
|
||||
steps:
|
||||
- name: ci/checkout-repo
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
ref: ${{ inputs.commit_sha }}
|
||||
fetch-depth: 0
|
||||
- name: ci/download-artifacts
|
||||
uses: actions/download-artifact@70fc10c6e5e1ce46ad2ea6f2b72d43f7d47b13c3 # v8.0.0
|
||||
with:
|
||||
pattern: e2e-test-results-${{ inputs.TEST }}-*
|
||||
path: e2e-tests/${{ inputs.TEST }}/
|
||||
merge-multiple: true
|
||||
- name: ci/upload-report-global
|
||||
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
|
||||
with:
|
||||
name: e2e-test-results-${{ inputs.TEST }}
|
||||
path: |
|
||||
e2e-tests/${{ inputs.TEST }}/logs/
|
||||
e2e-tests/${{ inputs.TEST }}/results/
|
||||
- name: ci/restore-npm-cache
|
||||
if: "${{ inputs.enable_reporting }}"
|
||||
uses: ./.github/actions/restore-e2e-npm-cache
|
||||
- name: ci/setup-node
|
||||
if: "${{ inputs.enable_reporting }}"
|
||||
uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
|
||||
id: setup_node
|
||||
with:
|
||||
node-version-file: ".nvmrc"
|
||||
- name: ci/publish-report
|
||||
if: "${{ inputs.enable_reporting }}"
|
||||
env:
|
||||
TYPE: "${{ inputs.REPORT_TYPE }}"
|
||||
TEST: "${{ inputs.TEST }}"
|
||||
SERVER: "${{ inputs.SERVER }}"
|
||||
SERVER_IMAGE: "${{ inputs.SERVER_IMAGE }}"
|
||||
AUTOMATION_DASHBOARD_URL: "${{ secrets.AUTOMATION_DASHBOARD_URL }}"
|
||||
WEBHOOK_URL: "${{ secrets.REPORT_WEBHOOK_URL }}"
|
||||
PR_NUMBER: "${{ inputs.PR_NUMBER }}"
|
||||
BRANCH: "${{ inputs.BRANCH }}"
|
||||
BUILD_ID: "${{ inputs.BUILD_ID }}"
|
||||
MM_ENV: "${{ inputs.MM_ENV }}"
|
||||
TM4J_API_KEY: "${{ secrets.REPORT_TM4J_API_KEY }}"
|
||||
TEST_CYCLE_LINK_PREFIX: "${{ secrets.REPORT_TM4J_TEST_CYCLE_LINK_PREFIX }}"
|
||||
run: |
|
||||
echo "DEBUG: TYPE=${TYPE}, PR_NUMBER=${PR_NUMBER:-<not set>}"
|
||||
make report
|
||||
# The results dir may have been modified as part of the reporting: re-upload
|
||||
- name: ci/upload-report-global
|
||||
if: "${{ inputs.enable_reporting }}"
|
||||
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
|
||||
with:
|
||||
name: e2e-test-results-${{ inputs.TEST }}
|
||||
path: |
|
||||
e2e-tests/${{ inputs.TEST }}/logs/
|
||||
e2e-tests/${{ inputs.TEST }}/results/
|
||||
overwrite: true
|
||||
|
||||
# Configure AWS credentials
|
||||
- name: ci/aws-configure
|
||||
if: (inputs.TEST == 'playwright')
|
||||
uses: aws-actions/configure-aws-credentials@8df5847569e6427dd6c4fb1cf565c83acfa8afa7 # v6.0.0
|
||||
with:
|
||||
aws-region: us-east-1
|
||||
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
||||
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
||||
|
||||
# Upload the playwright reports to S3
|
||||
- name: ci/upload-results-to-s3
|
||||
if: (inputs.TEST == 'playwright')
|
||||
id: upload-to-s3
|
||||
run: |
|
||||
echo "🔍 Checking if results directory exists..."
|
||||
|
||||
PR_NUMBER="${{ inputs.PR_NUMBER }}"
|
||||
LOCAL_RESULTS_PATH="${{ inputs.TEST }}/results/"
|
||||
LOCAL_LOGS_PATH="${{ inputs.TEST }}/logs/"
|
||||
RUN_ID="${{ github.run_id }}"
|
||||
S3_PATH="server-pr-${PR_NUMBER}/e2e-reports/${{ inputs.TEST }}/${RUN_ID}"
|
||||
|
||||
echo "📤 Uploading to s3://${AWS_S3_BUCKET}/${S3_PATH}/"
|
||||
|
||||
if [[ -d "$LOCAL_RESULTS_PATH" ]]; then
|
||||
aws s3 sync "$LOCAL_RESULTS_PATH" "s3://${AWS_S3_BUCKET}/${S3_PATH}/results/" \
|
||||
--acl public-read \
|
||||
--cache-control "no-cache"
|
||||
fi
|
||||
|
||||
REPORT_URL="https://${AWS_S3_BUCKET}.s3.amazonaws.com/${S3_PATH}/results/reporter/index.html"
|
||||
echo "✅ Report uploaded to: $REPORT_URL"
|
||||
|
||||
echo "report_url=$REPORT_URL" >> "$GITHUB_OUTPUT"
|
||||
env:
|
||||
AWS_REGION: us-east-1
|
||||
AWS_S3_BUCKET: mattermost-cypress-report
|
||||
|
||||
- name: ci/report-calculate-results
|
||||
id: calculate-results
|
||||
env:
|
||||
TEST: "${{ inputs.TEST }}"
|
||||
run: |
|
||||
AD_CYCLE_FILE="${TEST}/results/ad_cycle.json"
|
||||
if [ -f "$AD_CYCLE_FILE" ]; then
|
||||
# Prefer using the Automation Dashboard's results to calculate failures
|
||||
export PASSED=$(jq -r .pass "$AD_CYCLE_FILE")
|
||||
export FAILED=$(jq -r .fail "$AD_CYCLE_FILE")
|
||||
export FAILED_EXPECTED=$(jq -r ".known + .flaky + .skipped" "$AD_CYCLE_FILE")
|
||||
else
|
||||
# Otherwise, utilize summary.json to calculate the failures
|
||||
# NB: in this job, this file only makes sense if a single worker is used, as with Playwright
|
||||
export PASSED=$(jq '.passed' "${TEST}/results/summary.json")
|
||||
export FAILED=$(jq '.failed' "${TEST}/results/summary.json")
|
||||
export FAILED_EXPECTED=$(jq '.failed_expected' "${TEST}/results/summary.json")
|
||||
fi
|
||||
export TOTAL_SPECS=$(( PASSED + FAILED ))
|
||||
export PASS_RATE=$(jq -r '100 * (env.PASSED | tonumber) / (env.TOTAL_SPECS | tonumber)' <<<'{}' | xargs -l printf '%.2f')
|
||||
if [ "$FAILED" = "0" ]; then
|
||||
export COMMIT_STATUS_MESSAGE="All test cases passed"
|
||||
else
|
||||
export COMMIT_STATUS_MESSAGE="${FAILED} test cases failed. Please check the workflow logs"
|
||||
fi
|
||||
echo "passed=${PASSED:?}" >> $GITHUB_OUTPUT
|
||||
echo "failed=${FAILED:?}" >> $GITHUB_OUTPUT
|
||||
echo "failed_expected=${FAILED_EXPECTED:?}" >> $GITHUB_OUTPUT
|
||||
echo "pass_rate=${PASS_RATE:?}%" >> $GITHUB_OUTPUT
|
||||
echo "commit_status_message=${COMMIT_STATUS_MESSAGE:?}" >> $GITHUB_OUTPUT
|
||||
echo "$COMMIT_STATUS_MESSAGE"
|
||||
- name: ci/e2e-test-assert-results
|
||||
if: "${{ inputs.testcase_failure_fatal }}"
|
||||
run: |
|
||||
# Assert that the run contained 0 failures
|
||||
[ "${{ steps.calculate-results.outputs.failed }}" = "0" ]
|
||||
|
||||
update-failure-final-status:
|
||||
runs-on: ubuntu-24.04
|
||||
if: failure() || cancelled()
|
||||
needs:
|
||||
- generate-test-cycle
|
||||
- test
|
||||
- report
|
||||
steps:
|
||||
- uses: mattermost/actions/delivery/update-commit-status@f324ac89b05cc3511cb06e60642ac2fb829f0a63
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ github.token }}
|
||||
with:
|
||||
repository_full_name: ${{ github.repository }}
|
||||
commit_sha: ${{ inputs.commit_sha }}
|
||||
context: ${{ inputs.status_check_context }}
|
||||
description: ${{ needs.report.outputs.commit_status_message || 'Error during test execution' }}
|
||||
status: failure
|
||||
target_url: >-
|
||||
${{ inputs.TEST == 'playwright'
|
||||
&& needs.report.outputs.playwright_report_url
|
||||
|| needs.generate-test-cycle.outputs.status_check_url }}
|
||||
|
||||
|
||||
update-success-final-status:
|
||||
runs-on: ubuntu-24.04
|
||||
if: success()
|
||||
needs:
|
||||
- generate-test-cycle
|
||||
- test
|
||||
- report
|
||||
steps:
|
||||
- uses: mattermost/actions/delivery/update-commit-status@f324ac89b05cc3511cb06e60642ac2fb829f0a63
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ github.token }}
|
||||
with:
|
||||
repository_full_name: ${{ github.repository }}
|
||||
commit_sha: ${{ inputs.commit_sha }}
|
||||
context: ${{ inputs.status_check_context }}
|
||||
description: ${{ needs.report.outputs.commit_status_message || 'Error during test execution' }}
|
||||
status: success
|
||||
target_url: >-
|
||||
${{ inputs.TEST == 'playwright'
|
||||
&& needs.report.outputs.playwright_report_url
|
||||
|| needs.generate-test-cycle.outputs.status_check_url }}
|
||||
@@ -255,8 +255,6 @@ jobs:
|
||||
should_run: "${{ needs.check-changes.outputs.should_run }}"
|
||||
secrets:
|
||||
MM_LICENSE: "${{ secrets.MM_E2E_TEST_LICENSE_ONPREM_ENT }}"
|
||||
AWS_ACCESS_KEY_ID: "${{ secrets.CYPRESS_AWS_ACCESS_KEY_ID }}"
|
||||
AWS_SECRET_ACCESS_KEY: "${{ secrets.CYPRESS_AWS_SECRET_ACCESS_KEY }}"
|
||||
REPORT_WEBHOOK_URL: "${{ secrets.MM_E2E_REPORT_WEBHOOK_URL }}"
|
||||
|
||||
# Enterprise FIPS Edition
|
||||
@@ -309,6 +307,4 @@ jobs:
|
||||
pr_number: "${{ needs.resolve-pr.outputs.PR_NUMBER }}"
|
||||
secrets:
|
||||
MM_LICENSE: "${{ secrets.MM_E2E_TEST_LICENSE_ONPREM_ENT }}"
|
||||
AWS_ACCESS_KEY_ID: "${{ secrets.CYPRESS_AWS_ACCESS_KEY_ID }}"
|
||||
AWS_SECRET_ACCESS_KEY: "${{ secrets.CYPRESS_AWS_SECRET_ACCESS_KEY }}"
|
||||
REPORT_WEBHOOK_URL: "${{ secrets.MM_E2E_REPORT_WEBHOOK_URL }}"
|
||||
|
||||
@@ -1,348 +0,0 @@
|
||||
---
|
||||
name: E2E Tests - Cypress Template (v2 - test system io dispatch)
|
||||
|
||||
# Delegates Cypress spec dispatch + reporting to test system io.
|
||||
# Authenticates via GitHub Actions OIDC; calling job MUST grant
|
||||
# `id-token: write`.
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
test_type:
|
||||
description: "Type of test run (smoke or full)"
|
||||
type: string
|
||||
required: true
|
||||
workers:
|
||||
description: "Number of parallel test system io dispatch workers"
|
||||
type: number
|
||||
required: false
|
||||
default: 40
|
||||
enabled_docker_services:
|
||||
description: "Space-separated list of docker services to enable"
|
||||
type: string
|
||||
required: false
|
||||
default: "postgres inbucket minio openldap elasticsearch keycloak"
|
||||
|
||||
commit_sha:
|
||||
type: string
|
||||
required: true
|
||||
branch:
|
||||
type: string
|
||||
required: true
|
||||
build_id:
|
||||
type: string
|
||||
required: true
|
||||
server_image_tag:
|
||||
description: "Server image tag (e.g., master or short SHA)"
|
||||
type: string
|
||||
required: true
|
||||
server:
|
||||
type: string
|
||||
required: false
|
||||
default: onprem
|
||||
server_edition:
|
||||
description: "Server edition: enterprise (default), fips, or team"
|
||||
type: string
|
||||
required: false
|
||||
default: enterprise
|
||||
server_image_repo:
|
||||
description: "Docker registry: mattermostdevelopment (default) or mattermost"
|
||||
type: string
|
||||
required: false
|
||||
default: mattermostdevelopment
|
||||
server_image_aliases:
|
||||
description: "Comma-separated alias tags for description"
|
||||
type: string
|
||||
required: false
|
||||
|
||||
enable_reporting:
|
||||
type: boolean
|
||||
required: false
|
||||
default: false
|
||||
report_type:
|
||||
type: string
|
||||
required: false
|
||||
ref_branch:
|
||||
type: string
|
||||
required: false
|
||||
pr_number:
|
||||
type: string
|
||||
required: false
|
||||
context_name:
|
||||
description: "GitHub commit status context name"
|
||||
type: string
|
||||
required: true
|
||||
|
||||
cypress_stage:
|
||||
description: "Comma-separated `// Stage:` tags; spec must share at least one. Empty disables filter."
|
||||
type: string
|
||||
required: false
|
||||
default: "@prod"
|
||||
cypress_include_group:
|
||||
description: "Comma-separated `// Group:` tags; spec must share at least one. Empty disables filter."
|
||||
type: string
|
||||
required: false
|
||||
default: ""
|
||||
cypress_exclude_group:
|
||||
description: "Comma-separated `// Group:` tags; spec dropped if it shares any."
|
||||
type: string
|
||||
required: false
|
||||
default: "@te_only,@cloud_only,@high_availability"
|
||||
cypress_skip_on:
|
||||
description: "Comma-separated active-env tag(s); spec dropped if its `// Skip:` line shares any."
|
||||
type: string
|
||||
required: false
|
||||
default: "@headless"
|
||||
cypress_sort_first:
|
||||
description: "Comma-separated `// Group:` tags; matching specs dispatch first."
|
||||
type: string
|
||||
required: false
|
||||
default: "@compliance_export,@elasticsearch,@ldap_group,@ldap"
|
||||
cypress_sort_last:
|
||||
description: "Comma-separated `// Group:` tags; matching specs dispatch last."
|
||||
type: string
|
||||
required: false
|
||||
default: "@saml,@keycloak,@plugin,@plugins_uninstall,@mfa,@license_removal"
|
||||
retest_on_fail:
|
||||
description: "Re-dispatch failed dispatch units once (whole-spec retry, on top of cypress.config retries)"
|
||||
type: boolean
|
||||
required: false
|
||||
default: true
|
||||
|
||||
secrets:
|
||||
MM_LICENSE:
|
||||
required: false
|
||||
AUTOMATION_DASHBOARD_URL:
|
||||
required: false
|
||||
AUTOMATION_DASHBOARD_TOKEN:
|
||||
required: false
|
||||
PUSH_NOTIFICATION_SERVER:
|
||||
required: false
|
||||
REPORT_WEBHOOK_URL:
|
||||
required: false
|
||||
CWS_URL:
|
||||
required: false
|
||||
CWS_EXTRA_HTTP_HEADERS:
|
||||
required: false
|
||||
|
||||
# Callers must grant: contents: read, statuses: write, id-token: write
|
||||
permissions:
|
||||
contents: read
|
||||
statuses: write
|
||||
id-token: write
|
||||
|
||||
env:
|
||||
SERVER_IMAGE: "${{ inputs.server_image_repo }}/${{ inputs.server_edition == 'fips' && 'mattermost-enterprise-fips-edition' || inputs.server_edition == 'team' && 'mattermost-team-edition' || 'mattermost-enterprise-edition' }}:${{ inputs.server_image_tag }}"
|
||||
|
||||
jobs:
|
||||
dispatch-begin:
|
||||
runs-on: ubuntu-24.04
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
statuses: write
|
||||
outputs:
|
||||
composite-identity-json: ${{ steps.composite-identity.outputs.composite-identity-json }}
|
||||
workers-matrix: ${{ steps.matrix.outputs.workers }}
|
||||
start_time: ${{ steps.matrix.outputs.start_time }}
|
||||
steps:
|
||||
- name: ci/checkout-repo
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
ref: ${{ inputs.commit_sha }}
|
||||
fetch-depth: 1
|
||||
- name: ci/composite-identity
|
||||
id: composite-identity
|
||||
env:
|
||||
CONTEXT_NAME: ${{ inputs.context_name }}
|
||||
MM_SHA: ${{ inputs.commit_sha }}
|
||||
MM_BRANCH: ${{ inputs.branch }}
|
||||
PR_NUMBER: ${{ inputs.pr_number }}
|
||||
run: |
|
||||
# Derive the test-system-io run name from the GitHub commit-status
|
||||
# context: drop the `e2e-test/` prefix (the framework name already
|
||||
# implies E2E in the dashboard) and swap remaining `/` for `-` so
|
||||
# the dashboard URL is path-safe. The commit-status context itself
|
||||
# stays unchanged elsewhere — branch protection rules depend on it.
|
||||
NAME="${CONTEXT_NAME#e2e-test/}"
|
||||
NAME="${NAME//\//-}"
|
||||
# gh_pr_number is optional; include it only when present.
|
||||
if [ -n "$PR_NUMBER" ]; then
|
||||
COMPOSITE_IDENTITY=$(jq -nc \
|
||||
--arg repo "${{ github.repository }}" \
|
||||
--arg sha "${MM_SHA}" \
|
||||
--arg run_id "${GITHUB_RUN_ID}" \
|
||||
--arg name "${NAME}" \
|
||||
--arg attempt "${GITHUB_RUN_ATTEMPT}" \
|
||||
--arg branch "${MM_BRANCH}" \
|
||||
--arg pr "${PR_NUMBER}" \
|
||||
'{repository:$repo, commit_sha:$sha, gh_run_id:$run_id, name:$name, gh_run_attempt:$attempt, branch:$branch, gh_pr_number:$pr}')
|
||||
else
|
||||
COMPOSITE_IDENTITY=$(jq -nc \
|
||||
--arg repo "${{ github.repository }}" \
|
||||
--arg sha "${MM_SHA}" \
|
||||
--arg run_id "${GITHUB_RUN_ID}" \
|
||||
--arg name "${NAME}" \
|
||||
--arg attempt "${GITHUB_RUN_ATTEMPT}" \
|
||||
--arg branch "${MM_BRANCH}" \
|
||||
'{repository:$repo, commit_sha:$sha, gh_run_id:$run_id, name:$name, gh_run_attempt:$attempt, branch:$branch}')
|
||||
fi
|
||||
echo "composite-identity-json=${COMPOSITE_IDENTITY}" >> $GITHUB_OUTPUT
|
||||
- name: ci/matrix
|
||||
id: matrix
|
||||
run: |
|
||||
echo "workers=$(jq -nc --argjson n ${{ inputs.workers }} '[range(1; $n+1)]')" >> $GITHUB_OUTPUT
|
||||
echo "start_time=$(date +%s)" >> $GITHUB_OUTPUT
|
||||
- name: ci/dispatch-begin
|
||||
uses: mattermost/mattermost-test-system-io/.github/actions/test-system-io-dispatch-begin@main
|
||||
with:
|
||||
use-staging: ${{ vars.E2E_USE_STAGING_TEST_IO_URL != 'false' }}
|
||||
framework: cypress
|
||||
repo-dir: ${{ github.workspace }}
|
||||
composite-identity: ${{ steps.composite-identity.outputs.composite-identity-json }}
|
||||
total-reports-expected: ${{ inputs.workers }}
|
||||
retest-on-fail: ${{ inputs.retest_on_fail }}
|
||||
cypress-stage: ${{ inputs.cypress_stage }}
|
||||
cypress-include-group: ${{ inputs.cypress_include_group }}
|
||||
cypress-exclude-group: ${{ inputs.cypress_exclude_group }}
|
||||
cypress-skip-on: ${{ inputs.cypress_skip_on }}
|
||||
cypress-sort-first: ${{ inputs.cypress_sort_first }}
|
||||
cypress-sort-last: ${{ inputs.cypress_sort_last }}
|
||||
github-token: ${{ secrets.GITHUB_TOKEN }}
|
||||
commit-status-context: ${{ inputs.context_name }}
|
||||
image-tag: ${{ inputs.server_image_tag }}
|
||||
image-aliases: ${{ inputs.server_image_aliases }}
|
||||
|
||||
workers:
|
||||
name: dispatch-run-${{ matrix.worker_index }}
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 30
|
||||
needs: dispatch-begin
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
worker_index: ${{ fromJSON(needs.dispatch-begin.outputs.workers-matrix) }}
|
||||
env:
|
||||
COMPOSITE_IDENTITY: ${{ needs.dispatch-begin.outputs.composite-identity-json }}
|
||||
SERVER: "${{ inputs.server }}"
|
||||
MM_LICENSE: "${{ secrets.MM_LICENSE }}"
|
||||
ENABLED_DOCKER_SERVICES: "${{ inputs.enabled_docker_services }}"
|
||||
TEST: cypress
|
||||
# The dispatch adapter invokes `npx cypress run` directly (no
|
||||
# cross-env), so pin TZ here — several specs assume UTC.
|
||||
TZ: Etc/UTC
|
||||
BRANCH: "${{ inputs.branch }}"
|
||||
BUILD_ID: "${{ inputs.build_id }}"
|
||||
CI_BASE_URL: "${{ inputs.test_type }}-test-${{ matrix.worker_index }}"
|
||||
CYPRESS_pushNotificationServer: "${{ secrets.PUSH_NOTIFICATION_SERVER }}"
|
||||
CWS_URL: "${{ secrets.CWS_URL }}"
|
||||
CWS_EXTRA_HTTP_HEADERS: "${{ secrets.CWS_EXTRA_HTTP_HEADERS }}"
|
||||
steps:
|
||||
- name: ci/checkout-actions
|
||||
# Sparse-checkout just .github/actions from the triggering ref (master)
|
||||
# so the composite action below is available before the full checkout
|
||||
# overwrites the workspace with inputs.commit_sha.
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
sparse-checkout: .github/actions
|
||||
sparse-checkout-cone-mode: true
|
||||
- name: ci/runner-prep-for-openldap
|
||||
uses: ./.github/actions/runner-prep-openldap
|
||||
- name: ci/checkout-repo
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
ref: ${{ inputs.commit_sha }}
|
||||
fetch-depth: 0
|
||||
- name: ci/setup-node
|
||||
uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
|
||||
with:
|
||||
node-version-file: ".nvmrc"
|
||||
cache: npm
|
||||
cache-dependency-path: "e2e-tests/cypress/package-lock.json"
|
||||
- name: ci/get-webapp-node-modules
|
||||
working-directory: webapp
|
||||
run: make node_modules
|
||||
- name: ci/cloud-init
|
||||
working-directory: e2e-tests
|
||||
run: make cloud-init
|
||||
- name: ci/start-server
|
||||
working-directory: e2e-tests
|
||||
run: make start-server
|
||||
# `npm ci` in the host context replaces the container-built native
|
||||
# binaries with host-built ones, since the dispatch adapter spawns
|
||||
# `npx cypress run` on the host.
|
||||
- name: ci/prepare-cypress
|
||||
working-directory: e2e-tests/cypress
|
||||
run: npm ci
|
||||
- name: ci/dispatch-run
|
||||
uses: mattermost/mattermost-test-system-io/.github/actions/test-system-io-dispatch-run@main
|
||||
with:
|
||||
use-staging: ${{ vars.E2E_USE_STAGING_TEST_IO_URL != 'false' }}
|
||||
framework: cypress
|
||||
composite-identity: ${{ needs.dispatch-begin.outputs.composite-identity-json }}
|
||||
repo-dir: ${{ github.workspace }}
|
||||
artifacts-root: ${{ github.workspace }}/worker-artifacts
|
||||
github-token: ${{ secrets.GITHUB_TOKEN }}
|
||||
gh-job-name: dispatch-run-${{ matrix.worker_index }}
|
||||
- name: ci/cloud-teardown
|
||||
if: always()
|
||||
working-directory: e2e-tests
|
||||
run: make cloud-teardown
|
||||
- name: ci/upload-debug-artifacts
|
||||
if: always()
|
||||
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
|
||||
with:
|
||||
name: cypress-${{ inputs.test_type }}-${{ inputs.server_edition }}-debug-${{ matrix.worker_index }}
|
||||
path: |
|
||||
e2e-tests/cypress/logs/
|
||||
e2e-tests/cypress/results/
|
||||
e2e-tests/cypress/tests/screenshots/
|
||||
worker-artifacts/
|
||||
retention-days: 5
|
||||
if-no-files-found: ignore
|
||||
|
||||
report:
|
||||
runs-on: ubuntu-24.04
|
||||
needs: [dispatch-begin, workers]
|
||||
if: always()
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
statuses: write
|
||||
outputs:
|
||||
commit_status_description: ${{ steps.summary.outputs.commit_status_description }}
|
||||
webhook_payload: ${{ steps.summary.outputs.webhook_payload }}
|
||||
steps:
|
||||
- name: ci/checkout-repo
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
- name: ci/run-summary
|
||||
id: summary
|
||||
continue-on-error: true
|
||||
uses: mattermost/mattermost-test-system-io/.github/actions/test-system-io-summary@main
|
||||
with:
|
||||
use-staging: ${{ vars.E2E_USE_STAGING_TEST_IO_URL != 'false' }}
|
||||
composite-identity: ${{ needs.dispatch-begin.outputs.composite-identity-json }}
|
||||
framework: cypress
|
||||
report-type: ${{ inputs.report_type }}
|
||||
image-tag: ${{ inputs.server_image_tag }}
|
||||
image-aliases: ${{ inputs.server_image_aliases }}
|
||||
server-image: ${{ env.SERVER_IMAGE }}
|
||||
pr-number: ${{ inputs.pr_number }}
|
||||
ref-branch: ${{ inputs.ref_branch }}
|
||||
commit-status-context: ${{ inputs.context_name }}
|
||||
github-token: ${{ secrets.GITHUB_TOKEN }}
|
||||
- name: ci/publish-webhook
|
||||
if: inputs.enable_reporting && env.REPORT_WEBHOOK_URL != ''
|
||||
env:
|
||||
REPORT_WEBHOOK_URL: ${{ secrets.REPORT_WEBHOOK_URL }}
|
||||
PAYLOAD: ${{ steps.summary.outputs.webhook_payload }}
|
||||
run: |
|
||||
curl -X POST -H "Content-Type: application/json" -d "$PAYLOAD" "$REPORT_WEBHOOK_URL"
|
||||
- name: ci/assert-results
|
||||
env:
|
||||
SUMMARY_OUTCOME: ${{ steps.summary.outcome }}
|
||||
run: |
|
||||
[ "$SUMMARY_OUTCOME" = "success" ]
|
||||
@@ -1,29 +1,28 @@
|
||||
---
|
||||
name: E2E Tests - Cypress Template
|
||||
name: E2E Tests - Cypress Template (test system io dispatch)
|
||||
|
||||
# Delegates Cypress spec dispatch + reporting to test system io.
|
||||
# Authenticates via GitHub Actions OIDC; calling job MUST grant
|
||||
# `id-token: write`.
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
# Test configuration
|
||||
test_type:
|
||||
description: "Type of test run (smoke or full)"
|
||||
type: string
|
||||
required: true
|
||||
test_filter:
|
||||
description: "Test filter arguments"
|
||||
type: string
|
||||
required: true
|
||||
workers:
|
||||
description: "Number of parallel workers"
|
||||
description: "Number of parallel test system io dispatch workers"
|
||||
type: number
|
||||
required: false
|
||||
default: 1
|
||||
default: 40
|
||||
enabled_docker_services:
|
||||
description: "Space-separated list of docker services to enable"
|
||||
type: string
|
||||
required: false
|
||||
default: "postgres inbucket"
|
||||
default: "postgres inbucket minio openldap elasticsearch keycloak"
|
||||
|
||||
# Common build variables
|
||||
commit_sha:
|
||||
type: string
|
||||
required: true
|
||||
@@ -52,11 +51,10 @@ on:
|
||||
required: false
|
||||
default: mattermostdevelopment
|
||||
server_image_aliases:
|
||||
description: "Comma-separated alias tags for description (e.g., 'release-11.4, release-11')"
|
||||
description: "Comma-separated alias tags for description"
|
||||
type: string
|
||||
required: false
|
||||
|
||||
# Reporting options
|
||||
enable_reporting:
|
||||
type: boolean
|
||||
required: false
|
||||
@@ -65,28 +63,51 @@ on:
|
||||
type: string
|
||||
required: false
|
||||
ref_branch:
|
||||
description: "Source branch name for webhook messages (e.g., 'master' or 'release-11.4')"
|
||||
type: string
|
||||
required: false
|
||||
pr_number:
|
||||
type: string
|
||||
required: false
|
||||
# Commit status configuration
|
||||
context_name:
|
||||
description: "GitHub commit status context name"
|
||||
type: string
|
||||
required: true
|
||||
|
||||
outputs:
|
||||
passed:
|
||||
description: "Number of passed tests"
|
||||
value: ${{ jobs.report.outputs.passed }}
|
||||
failed:
|
||||
description: "Number of failed tests"
|
||||
value: ${{ jobs.report.outputs.failed }}
|
||||
status_check_url:
|
||||
description: "URL to test results"
|
||||
value: ${{ jobs.generate-test-cycle.outputs.status_check_url }}
|
||||
cypress_stage:
|
||||
description: "Comma-separated `// Stage:` tags; spec must share at least one. Empty disables filter."
|
||||
type: string
|
||||
required: false
|
||||
default: "@prod"
|
||||
cypress_include_group:
|
||||
description: "Comma-separated `// Group:` tags; spec must share at least one. Empty disables filter."
|
||||
type: string
|
||||
required: false
|
||||
default: ""
|
||||
cypress_exclude_group:
|
||||
description: "Comma-separated `// Group:` tags; spec dropped if it shares any."
|
||||
type: string
|
||||
required: false
|
||||
default: "@te_only,@cloud_only,@high_availability"
|
||||
cypress_skip_on:
|
||||
description: "Comma-separated active-env tag(s); spec dropped if its `// Skip:` line shares any."
|
||||
type: string
|
||||
required: false
|
||||
default: "@headless"
|
||||
cypress_sort_first:
|
||||
description: "Comma-separated `// Group:` tags; matching specs dispatch first."
|
||||
type: string
|
||||
required: false
|
||||
default: "@compliance_export,@elasticsearch,@ldap_group,@ldap"
|
||||
cypress_sort_last:
|
||||
description: "Comma-separated `// Group:` tags; matching specs dispatch last."
|
||||
type: string
|
||||
required: false
|
||||
default: "@saml,@keycloak,@plugin,@plugins_uninstall,@mfa,@license_removal"
|
||||
retest_on_fail:
|
||||
description: "Re-dispatch failed dispatch units once (whole-spec retry, on top of cypress.config retries)"
|
||||
type: boolean
|
||||
required: false
|
||||
default: true
|
||||
|
||||
secrets:
|
||||
MM_LICENSE:
|
||||
@@ -104,575 +125,224 @@ on:
|
||||
CWS_EXTRA_HTTP_HEADERS:
|
||||
required: false
|
||||
|
||||
# Callers must grant at least these scopes on the job that uses this workflow
|
||||
# Callers must grant: contents: read, statuses: write, id-token: write
|
||||
permissions:
|
||||
contents: read
|
||||
statuses: write
|
||||
id-token: write
|
||||
|
||||
env:
|
||||
SERVER_IMAGE: "${{ inputs.server_image_repo }}/${{ inputs.server_edition == 'fips' && 'mattermost-enterprise-fips-edition' || inputs.server_edition == 'team' && 'mattermost-team-edition' || 'mattermost-enterprise-edition' }}:${{ inputs.server_image_tag }}"
|
||||
|
||||
jobs:
|
||||
update-initial-status:
|
||||
dispatch-begin:
|
||||
runs-on: ubuntu-24.04
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
statuses: write
|
||||
steps:
|
||||
- name: ci/set-initial-status
|
||||
uses: mattermost/actions/delivery/update-commit-status@f324ac89b05cc3511cb06e60642ac2fb829f0a63
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ github.token }}
|
||||
with:
|
||||
repository_full_name: ${{ github.repository }}
|
||||
commit_sha: ${{ inputs.commit_sha }}
|
||||
context: ${{ inputs.context_name }}
|
||||
description: "tests running, image_tag:${{ inputs.server_image_tag }}${{ inputs.server_image_aliases && format(' ({0})', inputs.server_image_aliases) || '' }}"
|
||||
status: pending
|
||||
|
||||
generate-test-cycle:
|
||||
runs-on: ubuntu-24.04
|
||||
outputs:
|
||||
status_check_url: "${{ steps.generate-cycle.outputs.status_check_url }}"
|
||||
workers: "${{ steps.generate-workers.outputs.workers }}"
|
||||
start_time: "${{ steps.generate-workers.outputs.start_time }}"
|
||||
composite-identity-json: ${{ steps.composite-identity.outputs.composite-identity-json }}
|
||||
workers-matrix: ${{ steps.matrix.outputs.workers }}
|
||||
start_time: ${{ steps.matrix.outputs.start_time }}
|
||||
steps:
|
||||
- name: ci/generate-workers
|
||||
id: generate-workers
|
||||
run: |
|
||||
echo "workers=$(jq -nc '[range(${{ inputs.workers }})]')" >> $GITHUB_OUTPUT
|
||||
echo "start_time=$(date +%s)" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: ci/checkout-repo
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
ref: ${{ inputs.commit_sha }}
|
||||
fetch-depth: 0
|
||||
- name: ci/restore-npm-cache
|
||||
uses: ./.github/actions/restore-e2e-npm-cache
|
||||
- name: ci/setup-node
|
||||
uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
|
||||
with:
|
||||
node-version-file: ".nvmrc"
|
||||
|
||||
- name: ci/generate-test-cycle
|
||||
id: generate-cycle
|
||||
working-directory: e2e-tests
|
||||
fetch-depth: 1
|
||||
- name: ci/composite-identity
|
||||
id: composite-identity
|
||||
env:
|
||||
AUTOMATION_DASHBOARD_URL: "${{ secrets.AUTOMATION_DASHBOARD_URL }}"
|
||||
AUTOMATION_DASHBOARD_TOKEN: "${{ secrets.AUTOMATION_DASHBOARD_TOKEN }}"
|
||||
BRANCH: "${{ inputs.branch }}-${{ inputs.test_type }}"
|
||||
BUILD_ID: "${{ inputs.build_id }}"
|
||||
TEST: cypress
|
||||
TEST_FILTER: "${{ inputs.test_filter }}"
|
||||
CONTEXT_NAME: ${{ inputs.context_name }}
|
||||
MM_SHA: ${{ inputs.commit_sha }}
|
||||
MM_BRANCH: ${{ inputs.branch }}
|
||||
PR_NUMBER: ${{ inputs.pr_number }}
|
||||
run: |
|
||||
set -e -o pipefail
|
||||
make generate-test-cycle | tee generate-test-cycle.out
|
||||
TEST_CYCLE_ID=$(sed -nE "s/^.*id: '([^']+)'.*$/\1/p" <generate-test-cycle.out)
|
||||
if [ -n "$TEST_CYCLE_ID" ]; then
|
||||
echo "status_check_url=https://automation-dashboard.vercel.app/cycles/${TEST_CYCLE_ID}" >> $GITHUB_OUTPUT
|
||||
# Derive the test-system-io run name from the GitHub commit-status
|
||||
# context: drop the `e2e-test/` prefix (the framework name already
|
||||
# implies E2E in the dashboard) and swap remaining `/` for `-` so
|
||||
# the dashboard URL is path-safe. The commit-status context itself
|
||||
# stays unchanged elsewhere — branch protection rules depend on it.
|
||||
NAME="${CONTEXT_NAME#e2e-test/}"
|
||||
NAME="${NAME//\//-}"
|
||||
# gh_pr_number is optional; include it only when present.
|
||||
if [ -n "$PR_NUMBER" ]; then
|
||||
COMPOSITE_IDENTITY=$(jq -nc \
|
||||
--arg repo "${{ github.repository }}" \
|
||||
--arg sha "${MM_SHA}" \
|
||||
--arg run_id "${GITHUB_RUN_ID}" \
|
||||
--arg name "${NAME}" \
|
||||
--arg attempt "${GITHUB_RUN_ATTEMPT}" \
|
||||
--arg branch "${MM_BRANCH}" \
|
||||
--arg pr "${PR_NUMBER}" \
|
||||
'{repository:$repo, commit_sha:$sha, gh_run_id:$run_id, name:$name, gh_run_attempt:$attempt, branch:$branch, gh_pr_number:$pr}')
|
||||
else
|
||||
echo "status_check_url=${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}" >> $GITHUB_OUTPUT
|
||||
COMPOSITE_IDENTITY=$(jq -nc \
|
||||
--arg repo "${{ github.repository }}" \
|
||||
--arg sha "${MM_SHA}" \
|
||||
--arg run_id "${GITHUB_RUN_ID}" \
|
||||
--arg name "${NAME}" \
|
||||
--arg attempt "${GITHUB_RUN_ATTEMPT}" \
|
||||
--arg branch "${MM_BRANCH}" \
|
||||
'{repository:$repo, commit_sha:$sha, gh_run_id:$run_id, name:$name, gh_run_attempt:$attempt, branch:$branch}')
|
||||
fi
|
||||
echo "composite-identity-json=${COMPOSITE_IDENTITY}" >> $GITHUB_OUTPUT
|
||||
- name: ci/matrix
|
||||
id: matrix
|
||||
run: |
|
||||
echo "workers=$(jq -nc --argjson n ${{ inputs.workers }} '[range(1; $n+1)]')" >> $GITHUB_OUTPUT
|
||||
echo "start_time=$(date +%s)" >> $GITHUB_OUTPUT
|
||||
- name: ci/dispatch-begin
|
||||
uses: mattermost/mattermost-test-system-io/.github/actions/test-system-io-dispatch-begin@main
|
||||
with:
|
||||
use-staging: ${{ vars.E2E_USE_STAGING_TEST_IO_URL != 'false' }}
|
||||
framework: cypress
|
||||
repo-dir: ${{ github.workspace }}
|
||||
composite-identity: ${{ steps.composite-identity.outputs.composite-identity-json }}
|
||||
total-reports-expected: ${{ inputs.workers }}
|
||||
retest-on-fail: ${{ inputs.retest_on_fail }}
|
||||
cypress-stage: ${{ inputs.cypress_stage }}
|
||||
cypress-include-group: ${{ inputs.cypress_include_group }}
|
||||
cypress-exclude-group: ${{ inputs.cypress_exclude_group }}
|
||||
cypress-skip-on: ${{ inputs.cypress_skip_on }}
|
||||
cypress-sort-first: ${{ inputs.cypress_sort_first }}
|
||||
cypress-sort-last: ${{ inputs.cypress_sort_last }}
|
||||
github-token: ${{ secrets.GITHUB_TOKEN }}
|
||||
commit-status-context: ${{ inputs.context_name }}
|
||||
image-tag: ${{ inputs.server_image_tag }}
|
||||
image-aliases: ${{ inputs.server_image_aliases }}
|
||||
|
||||
run-tests:
|
||||
workers:
|
||||
name: dispatch-run-${{ matrix.worker_index }}
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 30
|
||||
continue-on-error: ${{ inputs.workers > 1 }}
|
||||
needs:
|
||||
- generate-test-cycle
|
||||
if: needs.generate-test-cycle.result == 'success'
|
||||
needs: dispatch-begin
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
worker_index: ${{ fromJSON(needs.generate-test-cycle.outputs.workers) }}
|
||||
defaults:
|
||||
run:
|
||||
working-directory: e2e-tests
|
||||
worker_index: ${{ fromJSON(needs.dispatch-begin.outputs.workers-matrix) }}
|
||||
env:
|
||||
AUTOMATION_DASHBOARD_URL: "${{ secrets.AUTOMATION_DASHBOARD_URL }}"
|
||||
AUTOMATION_DASHBOARD_TOKEN: "${{ secrets.AUTOMATION_DASHBOARD_TOKEN }}"
|
||||
COMPOSITE_IDENTITY: ${{ needs.dispatch-begin.outputs.composite-identity-json }}
|
||||
SERVER: "${{ inputs.server }}"
|
||||
MM_LICENSE: "${{ secrets.MM_LICENSE }}"
|
||||
ENABLED_DOCKER_SERVICES: "${{ inputs.enabled_docker_services }}"
|
||||
TEST: cypress
|
||||
TEST_FILTER: "${{ inputs.test_filter }}"
|
||||
BRANCH: "${{ inputs.branch }}-${{ inputs.test_type }}"
|
||||
# The dispatch adapter invokes `npx cypress run` directly (no
|
||||
# cross-env), so pin TZ here — several specs assume UTC.
|
||||
TZ: Etc/UTC
|
||||
BRANCH: "${{ inputs.branch }}"
|
||||
BUILD_ID: "${{ inputs.build_id }}"
|
||||
CI_BASE_URL: "${{ inputs.test_type }}-test-${{ matrix.worker_index }}"
|
||||
CYPRESS_pushNotificationServer: "${{ secrets.PUSH_NOTIFICATION_SERVER }}"
|
||||
CWS_URL: "${{ secrets.CWS_URL }}"
|
||||
CWS_EXTRA_HTTP_HEADERS: "${{ secrets.CWS_EXTRA_HTTP_HEADERS }}"
|
||||
steps:
|
||||
- name: ci/checkout-actions
|
||||
# Sparse-checkout just .github/actions from the triggering ref (master)
|
||||
# so the composite action below is available before the full checkout
|
||||
# overwrites the workspace with inputs.commit_sha.
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
sparse-checkout: .github/actions
|
||||
sparse-checkout-cone-mode: true
|
||||
- name: ci/runner-prep-for-openldap
|
||||
uses: ./.github/actions/runner-prep-openldap
|
||||
- name: ci/checkout-repo
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
ref: ${{ inputs.commit_sha }}
|
||||
fetch-depth: 0
|
||||
- name: ci/restore-npm-cache
|
||||
uses: ./.github/actions/restore-e2e-npm-cache
|
||||
- name: ci/setup-node
|
||||
uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
|
||||
with:
|
||||
node-version-file: ".nvmrc"
|
||||
- name: ci/npm-cache-verify
|
||||
# Heal any partial/dangling entries left in the restored ~/.npm cache
|
||||
# before running `npm ci`. Avoids the intermittent EEXIST/ENOENT
|
||||
# failures in npm's cacache writer.
|
||||
run: npm cache verify
|
||||
cache: npm
|
||||
cache-dependency-path: "e2e-tests/cypress/package-lock.json"
|
||||
- name: ci/get-webapp-node-modules
|
||||
working-directory: webapp
|
||||
run: make node_modules
|
||||
- name: ci/run-tests
|
||||
run: |
|
||||
make cloud-init
|
||||
make
|
||||
- name: ci/cloud-teardown
|
||||
if: always()
|
||||
run: make cloud-teardown
|
||||
- name: ci/upload-results
|
||||
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
|
||||
if: always()
|
||||
with:
|
||||
name: cypress-${{ inputs.test_type }}-${{ inputs.server_edition }}-results-${{ matrix.worker_index }}
|
||||
path: |
|
||||
e2e-tests/cypress/logs/
|
||||
e2e-tests/cypress/results/
|
||||
retention-days: 5
|
||||
|
||||
calculate-results:
|
||||
runs-on: ubuntu-24.04
|
||||
needs:
|
||||
- generate-test-cycle
|
||||
- run-tests
|
||||
if: always() && needs.generate-test-cycle.result == 'success'
|
||||
outputs:
|
||||
passed: ${{ steps.calculate.outputs.passed }}
|
||||
failed: ${{ steps.calculate.outputs.failed }}
|
||||
pending: ${{ steps.calculate.outputs.pending }}
|
||||
total_specs: ${{ steps.calculate.outputs.total_specs }}
|
||||
failed_specs: ${{ steps.calculate.outputs.failed_specs }}
|
||||
failed_specs_count: ${{ steps.calculate.outputs.failed_specs_count }}
|
||||
failed_tests: ${{ steps.calculate.outputs.failed_tests }}
|
||||
commit_status_message: ${{ steps.calculate.outputs.commit_status_message }}
|
||||
total: ${{ steps.calculate.outputs.total }}
|
||||
pass_rate: ${{ steps.calculate.outputs.pass_rate }}
|
||||
color: ${{ steps.calculate.outputs.color }}
|
||||
test_duration: ${{ steps.calculate.outputs.test_duration }}
|
||||
end_time: ${{ steps.record-end-time.outputs.end_time }}
|
||||
steps:
|
||||
- name: ci/checkout-repo
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
- name: ci/download-results
|
||||
uses: actions/download-artifact@70fc10c6e5e1ce46ad2ea6f2b72d43f7d47b13c3 # v8.0.0
|
||||
with:
|
||||
pattern: cypress-${{ inputs.test_type }}-${{ inputs.server_edition }}-results-*
|
||||
path: e2e-tests/cypress/
|
||||
merge-multiple: true
|
||||
- name: ci/calculate
|
||||
id: calculate
|
||||
uses: ./.github/actions/calculate-cypress-results
|
||||
with:
|
||||
original-results-path: e2e-tests/cypress/results
|
||||
- name: ci/record-end-time
|
||||
id: record-end-time
|
||||
run: echo "end_time=$(date +%s)" >> $GITHUB_OUTPUT
|
||||
|
||||
run-failed-tests:
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 30
|
||||
needs:
|
||||
- generate-test-cycle
|
||||
- run-tests
|
||||
- calculate-results
|
||||
if: >-
|
||||
always() &&
|
||||
needs.calculate-results.result == 'success' &&
|
||||
needs.calculate-results.outputs.failed != '0' &&
|
||||
fromJSON(needs.calculate-results.outputs.failed_specs_count) <= 20
|
||||
defaults:
|
||||
run:
|
||||
- name: ci/cloud-init
|
||||
working-directory: e2e-tests
|
||||
env:
|
||||
AUTOMATION_DASHBOARD_URL: "${{ secrets.AUTOMATION_DASHBOARD_URL }}"
|
||||
AUTOMATION_DASHBOARD_TOKEN: "${{ secrets.AUTOMATION_DASHBOARD_TOKEN }}"
|
||||
SERVER: "${{ inputs.server }}"
|
||||
MM_LICENSE: "${{ secrets.MM_LICENSE }}"
|
||||
ENABLED_DOCKER_SERVICES: "${{ inputs.enabled_docker_services }}"
|
||||
TEST: cypress
|
||||
BRANCH: "${{ inputs.branch }}-${{ inputs.test_type }}-retest"
|
||||
BUILD_ID: "${{ inputs.build_id }}-retest"
|
||||
CYPRESS_pushNotificationServer: "${{ secrets.PUSH_NOTIFICATION_SERVER }}"
|
||||
CWS_URL: "${{ secrets.CWS_URL }}"
|
||||
CWS_EXTRA_HTTP_HEADERS: "${{ secrets.CWS_EXTRA_HTTP_HEADERS }}"
|
||||
steps:
|
||||
- name: ci/checkout-repo
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
run: make cloud-init
|
||||
- name: ci/start-server
|
||||
working-directory: e2e-tests
|
||||
run: make start-server
|
||||
# `npm ci` in the host context replaces the container-built native
|
||||
# binaries with host-built ones, since the dispatch adapter spawns
|
||||
# `npx cypress run` on the host.
|
||||
- name: ci/prepare-cypress
|
||||
working-directory: e2e-tests/cypress
|
||||
run: npm ci
|
||||
- name: ci/dispatch-run
|
||||
uses: mattermost/mattermost-test-system-io/.github/actions/test-system-io-dispatch-run@main
|
||||
with:
|
||||
persist-credentials: false
|
||||
ref: ${{ inputs.commit_sha }}
|
||||
fetch-depth: 0
|
||||
- name: ci/restore-npm-cache
|
||||
uses: ./.github/actions/restore-e2e-npm-cache
|
||||
- name: ci/setup-node
|
||||
uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
|
||||
with:
|
||||
node-version-file: ".nvmrc"
|
||||
- name: ci/run-failed-specs
|
||||
env:
|
||||
SPEC_FILES: ${{ needs.calculate-results.outputs.failed_specs }}
|
||||
run: |
|
||||
echo "Retesting failed specs: $SPEC_FILES"
|
||||
make cloud-init
|
||||
make start-server run-specs
|
||||
use-staging: ${{ vars.E2E_USE_STAGING_TEST_IO_URL != 'false' }}
|
||||
framework: cypress
|
||||
composite-identity: ${{ needs.dispatch-begin.outputs.composite-identity-json }}
|
||||
repo-dir: ${{ github.workspace }}
|
||||
artifacts-root: ${{ github.workspace }}/worker-artifacts
|
||||
github-token: ${{ secrets.GITHUB_TOKEN }}
|
||||
gh-job-name: dispatch-run-${{ matrix.worker_index }}
|
||||
- name: ci/cloud-teardown
|
||||
if: always()
|
||||
working-directory: e2e-tests
|
||||
run: make cloud-teardown
|
||||
- name: ci/upload-retest-results
|
||||
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
|
||||
- name: ci/upload-debug-artifacts
|
||||
if: always()
|
||||
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
|
||||
with:
|
||||
name: cypress-${{ inputs.test_type }}-${{ inputs.server_edition }}-retest-results
|
||||
name: cypress-${{ inputs.test_type }}-${{ inputs.server_edition }}-debug-${{ matrix.worker_index }}
|
||||
path: |
|
||||
e2e-tests/cypress/logs/
|
||||
e2e-tests/cypress/results/
|
||||
e2e-tests/cypress/tests/screenshots/
|
||||
worker-artifacts/
|
||||
retention-days: 5
|
||||
if-no-files-found: ignore
|
||||
|
||||
report:
|
||||
runs-on: ubuntu-24.04
|
||||
needs:
|
||||
- generate-test-cycle
|
||||
- run-tests
|
||||
- calculate-results
|
||||
- run-failed-tests
|
||||
if: always() && needs.calculate-results.result == 'success'
|
||||
needs: [dispatch-begin, workers]
|
||||
if: always()
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
statuses: write
|
||||
outputs:
|
||||
passed: "${{ steps.final-results.outputs.passed }}"
|
||||
failed: "${{ steps.final-results.outputs.failed }}"
|
||||
commit_status_message: "${{ steps.final-results.outputs.commit_status_message }}"
|
||||
duration: "${{ steps.duration.outputs.duration }}"
|
||||
duration_display: "${{ steps.duration.outputs.duration_display }}"
|
||||
retest_display: "${{ steps.duration.outputs.retest_display }}"
|
||||
defaults:
|
||||
run:
|
||||
working-directory: e2e-tests
|
||||
commit_status_description: ${{ steps.summary.outputs.commit_status_description }}
|
||||
webhook_payload: ${{ steps.summary.outputs.webhook_payload }}
|
||||
steps:
|
||||
- name: ci/checkout-repo
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
- name: ci/run-summary
|
||||
id: summary
|
||||
continue-on-error: true
|
||||
uses: mattermost/mattermost-test-system-io/.github/actions/test-system-io-summary@main
|
||||
with:
|
||||
persist-credentials: false
|
||||
- name: ci/restore-npm-cache
|
||||
uses: ./.github/actions/restore-e2e-npm-cache
|
||||
- name: ci/setup-node
|
||||
uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
|
||||
with:
|
||||
node-version-file: ".nvmrc"
|
||||
|
||||
# PATH A: run-failed-tests was skipped (no failures to retest)
|
||||
- name: ci/download-results-path-a
|
||||
if: needs.run-failed-tests.result == 'skipped'
|
||||
uses: actions/download-artifact@70fc10c6e5e1ce46ad2ea6f2b72d43f7d47b13c3 # v8.0.0
|
||||
with:
|
||||
pattern: cypress-${{ inputs.test_type }}-${{ inputs.server_edition }}-results-*
|
||||
path: e2e-tests/cypress/
|
||||
merge-multiple: true
|
||||
- name: ci/use-previous-calculation
|
||||
if: needs.run-failed-tests.result == 'skipped'
|
||||
id: use-previous
|
||||
run: |
|
||||
echo "passed=${{ needs.calculate-results.outputs.passed }}" >> $GITHUB_OUTPUT
|
||||
echo "failed=${{ needs.calculate-results.outputs.failed }}" >> $GITHUB_OUTPUT
|
||||
echo "pending=${{ needs.calculate-results.outputs.pending }}" >> $GITHUB_OUTPUT
|
||||
echo "total_specs=${{ needs.calculate-results.outputs.total_specs }}" >> $GITHUB_OUTPUT
|
||||
echo "failed_specs=${{ needs.calculate-results.outputs.failed_specs }}" >> $GITHUB_OUTPUT
|
||||
echo "failed_specs_count=${{ needs.calculate-results.outputs.failed_specs_count }}" >> $GITHUB_OUTPUT
|
||||
echo "commit_status_message=${{ needs.calculate-results.outputs.commit_status_message }}" >> $GITHUB_OUTPUT
|
||||
echo "total=${{ needs.calculate-results.outputs.total }}" >> $GITHUB_OUTPUT
|
||||
echo "pass_rate=${{ needs.calculate-results.outputs.pass_rate }}" >> $GITHUB_OUTPUT
|
||||
echo "color=${{ needs.calculate-results.outputs.color }}" >> $GITHUB_OUTPUT
|
||||
echo "test_duration=${{ needs.calculate-results.outputs.test_duration }}" >> $GITHUB_OUTPUT
|
||||
{
|
||||
echo "failed_tests<<EOF"
|
||||
echo "${{ needs.calculate-results.outputs.failed_tests }}"
|
||||
echo "EOF"
|
||||
} >> $GITHUB_OUTPUT
|
||||
|
||||
# PATH B: run-failed-tests ran, need to merge and recalculate
|
||||
- name: ci/download-original-results
|
||||
if: needs.run-failed-tests.result != 'skipped'
|
||||
uses: actions/download-artifact@70fc10c6e5e1ce46ad2ea6f2b72d43f7d47b13c3 # v8.0.0
|
||||
with:
|
||||
pattern: cypress-${{ inputs.test_type }}-${{ inputs.server_edition }}-results-*
|
||||
path: e2e-tests/cypress/
|
||||
merge-multiple: true
|
||||
- name: ci/download-retest-results
|
||||
if: needs.run-failed-tests.result != 'skipped'
|
||||
uses: actions/download-artifact@70fc10c6e5e1ce46ad2ea6f2b72d43f7d47b13c3 # v8.0.0
|
||||
with:
|
||||
name: cypress-${{ inputs.test_type }}-${{ inputs.server_edition }}-retest-results
|
||||
path: e2e-tests/cypress/retest-results/
|
||||
- name: ci/calculate-results
|
||||
if: needs.run-failed-tests.result != 'skipped'
|
||||
id: recalculate
|
||||
uses: ./.github/actions/calculate-cypress-results
|
||||
with:
|
||||
original-results-path: e2e-tests/cypress/results
|
||||
retest-results-path: e2e-tests/cypress/retest-results/results
|
||||
|
||||
# Set final outputs from either path
|
||||
- name: ci/set-final-results
|
||||
id: final-results
|
||||
env:
|
||||
USE_PREVIOUS_FAILED_TESTS: ${{ steps.use-previous.outputs.failed_tests }}
|
||||
RECALCULATE_FAILED_TESTS: ${{ steps.recalculate.outputs.failed_tests }}
|
||||
run: |
|
||||
if [ "${{ needs.run-failed-tests.result }}" == "skipped" ]; then
|
||||
echo "passed=${{ steps.use-previous.outputs.passed }}" >> $GITHUB_OUTPUT
|
||||
echo "failed=${{ steps.use-previous.outputs.failed }}" >> $GITHUB_OUTPUT
|
||||
echo "pending=${{ steps.use-previous.outputs.pending }}" >> $GITHUB_OUTPUT
|
||||
echo "total_specs=${{ steps.use-previous.outputs.total_specs }}" >> $GITHUB_OUTPUT
|
||||
echo "failed_specs=${{ steps.use-previous.outputs.failed_specs }}" >> $GITHUB_OUTPUT
|
||||
echo "failed_specs_count=${{ steps.use-previous.outputs.failed_specs_count }}" >> $GITHUB_OUTPUT
|
||||
echo "commit_status_message=${{ steps.use-previous.outputs.commit_status_message }}" >> $GITHUB_OUTPUT
|
||||
echo "total=${{ steps.use-previous.outputs.total }}" >> $GITHUB_OUTPUT
|
||||
echo "pass_rate=${{ steps.use-previous.outputs.pass_rate }}" >> $GITHUB_OUTPUT
|
||||
echo "color=${{ steps.use-previous.outputs.color }}" >> $GITHUB_OUTPUT
|
||||
echo "test_duration=${{ steps.use-previous.outputs.test_duration }}" >> $GITHUB_OUTPUT
|
||||
{
|
||||
echo "failed_tests<<EOF"
|
||||
echo "$USE_PREVIOUS_FAILED_TESTS"
|
||||
echo "EOF"
|
||||
} >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "passed=${{ steps.recalculate.outputs.passed }}" >> $GITHUB_OUTPUT
|
||||
echo "failed=${{ steps.recalculate.outputs.failed }}" >> $GITHUB_OUTPUT
|
||||
echo "pending=${{ steps.recalculate.outputs.pending }}" >> $GITHUB_OUTPUT
|
||||
echo "total_specs=${{ steps.recalculate.outputs.total_specs }}" >> $GITHUB_OUTPUT
|
||||
echo "failed_specs=${{ steps.recalculate.outputs.failed_specs }}" >> $GITHUB_OUTPUT
|
||||
echo "failed_specs_count=${{ steps.recalculate.outputs.failed_specs_count }}" >> $GITHUB_OUTPUT
|
||||
echo "commit_status_message=${{ steps.recalculate.outputs.commit_status_message }}" >> $GITHUB_OUTPUT
|
||||
echo "total=${{ steps.recalculate.outputs.total }}" >> $GITHUB_OUTPUT
|
||||
echo "pass_rate=${{ steps.recalculate.outputs.pass_rate }}" >> $GITHUB_OUTPUT
|
||||
echo "color=${{ steps.recalculate.outputs.color }}" >> $GITHUB_OUTPUT
|
||||
echo "test_duration=${{ steps.recalculate.outputs.test_duration }}" >> $GITHUB_OUTPUT
|
||||
{
|
||||
echo "failed_tests<<EOF"
|
||||
echo "$RECALCULATE_FAILED_TESTS"
|
||||
echo "EOF"
|
||||
} >> $GITHUB_OUTPUT
|
||||
fi
|
||||
|
||||
- name: ci/compute-duration
|
||||
id: duration
|
||||
env:
|
||||
START_TIME: ${{ needs.generate-test-cycle.outputs.start_time }}
|
||||
FIRST_PASS_END_TIME: ${{ needs.calculate-results.outputs.end_time }}
|
||||
RETEST_RESULT: ${{ needs.run-failed-tests.result }}
|
||||
RETEST_SPEC_COUNT: ${{ needs.calculate-results.outputs.failed_specs_count }}
|
||||
TEST_DURATION: ${{ steps.final-results.outputs.test_duration }}
|
||||
run: |
|
||||
NOW=$(date +%s)
|
||||
ELAPSED=$((NOW - START_TIME))
|
||||
MINUTES=$((ELAPSED / 60))
|
||||
SECONDS=$((ELAPSED % 60))
|
||||
DURATION="${MINUTES}m ${SECONDS}s"
|
||||
|
||||
# Compute first-pass and re-run durations
|
||||
FIRST_PASS_ELAPSED=$((FIRST_PASS_END_TIME - START_TIME))
|
||||
FP_MIN=$((FIRST_PASS_ELAPSED / 60))
|
||||
FP_SEC=$((FIRST_PASS_ELAPSED % 60))
|
||||
FIRST_PASS="${FP_MIN}m ${FP_SEC}s"
|
||||
|
||||
if [ "$RETEST_RESULT" != "skipped" ]; then
|
||||
RERUN_ELAPSED=$((NOW - FIRST_PASS_END_TIME))
|
||||
RR_MIN=$((RERUN_ELAPSED / 60))
|
||||
RR_SEC=$((RERUN_ELAPSED % 60))
|
||||
RUN_BREAKDOWN=" (first-pass: ${FIRST_PASS}, re-run: ${RR_MIN}m ${RR_SEC}s)"
|
||||
else
|
||||
RUN_BREAKDOWN=""
|
||||
fi
|
||||
|
||||
# Duration icons: >20m high alert, >15m warning, otherwise clock
|
||||
if [ "$MINUTES" -ge 20 ]; then
|
||||
DURATION_DISPLAY=":rotating_light: ${DURATION}${RUN_BREAKDOWN} | test: ${TEST_DURATION}"
|
||||
elif [ "$MINUTES" -ge 15 ]; then
|
||||
DURATION_DISPLAY=":warning: ${DURATION}${RUN_BREAKDOWN} | test: ${TEST_DURATION}"
|
||||
else
|
||||
DURATION_DISPLAY=":clock3: ${DURATION}${RUN_BREAKDOWN} | test: ${TEST_DURATION}"
|
||||
fi
|
||||
|
||||
# Retest indicator with spec count
|
||||
if [ "$RETEST_RESULT" != "skipped" ]; then
|
||||
RETEST_DISPLAY=":repeat: re-run ${RETEST_SPEC_COUNT} spec(s)"
|
||||
else
|
||||
RETEST_DISPLAY=""
|
||||
fi
|
||||
|
||||
echo "duration=${DURATION}" >> $GITHUB_OUTPUT
|
||||
echo "duration_display=${DURATION_DISPLAY}" >> $GITHUB_OUTPUT
|
||||
echo "retest_display=${RETEST_DISPLAY}" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: ci/upload-combined-results
|
||||
if: inputs.workers > 1
|
||||
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
|
||||
with:
|
||||
name: cypress-${{ inputs.test_type }}-${{ inputs.server_edition }}-results
|
||||
path: |
|
||||
e2e-tests/cypress/logs/
|
||||
e2e-tests/cypress/results/
|
||||
- name: ci/publish-report
|
||||
use-staging: ${{ vars.E2E_USE_STAGING_TEST_IO_URL != 'false' }}
|
||||
composite-identity: ${{ needs.dispatch-begin.outputs.composite-identity-json }}
|
||||
framework: cypress
|
||||
report-type: ${{ inputs.report_type }}
|
||||
image-tag: ${{ inputs.server_image_tag }}
|
||||
image-aliases: ${{ inputs.server_image_aliases }}
|
||||
server-image: ${{ env.SERVER_IMAGE }}
|
||||
pr-number: ${{ inputs.pr_number }}
|
||||
ref-branch: ${{ inputs.ref_branch }}
|
||||
commit-status-context: ${{ inputs.context_name }}
|
||||
github-token: ${{ secrets.GITHUB_TOKEN }}
|
||||
- name: ci/publish-webhook
|
||||
if: inputs.enable_reporting && env.REPORT_WEBHOOK_URL != ''
|
||||
env:
|
||||
REPORT_WEBHOOK_URL: ${{ secrets.REPORT_WEBHOOK_URL }}
|
||||
COMMIT_STATUS_MESSAGE: ${{ steps.final-results.outputs.commit_status_message }}
|
||||
COLOR: ${{ steps.final-results.outputs.color }}
|
||||
REPORT_URL: ${{ needs.generate-test-cycle.outputs.status_check_url }}
|
||||
TEST_TYPE: ${{ inputs.test_type }}
|
||||
REPORT_TYPE: ${{ inputs.report_type }}
|
||||
COMMIT_SHA: ${{ inputs.commit_sha }}
|
||||
REF_BRANCH: ${{ inputs.ref_branch }}
|
||||
PR_NUMBER: ${{ inputs.pr_number }}
|
||||
DURATION_DISPLAY: ${{ steps.duration.outputs.duration_display }}
|
||||
RETEST_DISPLAY: ${{ steps.duration.outputs.retest_display }}
|
||||
PAYLOAD: ${{ steps.summary.outputs.webhook_payload }}
|
||||
run: |
|
||||
# Capitalize test type
|
||||
TEST_TYPE_CAP=$(echo "$TEST_TYPE" | sed 's/.*/\u&/')
|
||||
|
||||
# Build source line based on report type
|
||||
COMMIT_SHORT="${COMMIT_SHA::7}"
|
||||
COMMIT_URL="https://github.com/${{ github.repository }}/commit/${COMMIT_SHA}"
|
||||
if [ "$REPORT_TYPE" = "RELEASE_CUT" ]; then
|
||||
SOURCE_LINE=":github_round: [${COMMIT_SHORT}](${COMMIT_URL}) on \`${REF_BRANCH}\`"
|
||||
elif [ "$REPORT_TYPE" = "MASTER" ] || [ "$REPORT_TYPE" = "RELEASE" ]; then
|
||||
SOURCE_LINE=":git_merge: [${COMMIT_SHORT}](${COMMIT_URL}) on \`${REF_BRANCH}\`"
|
||||
else
|
||||
SOURCE_LINE=":open-pull-request: [mattermost-pr-${PR_NUMBER}](https://github.com/${{ github.repository }}/pull/${PR_NUMBER})"
|
||||
fi
|
||||
|
||||
# Build retest part for message
|
||||
RETEST_PART=""
|
||||
if [ -n "$RETEST_DISPLAY" ]; then
|
||||
RETEST_PART=" | ${RETEST_DISPLAY}"
|
||||
fi
|
||||
|
||||
# Build payload with attachments
|
||||
PAYLOAD=$(cat <<EOF
|
||||
{
|
||||
"username": "E2E Test",
|
||||
"icon_url": "https://mattermost.com/wp-content/uploads/2022/02/icon_WS.png",
|
||||
"attachments": [{
|
||||
"color": "${COLOR}",
|
||||
"text": "**Results - Cypress ${TEST_TYPE_CAP} Tests**\n\n${SOURCE_LINE}\n:docker: \`${{ env.SERVER_IMAGE }}\`\n${COMMIT_STATUS_MESSAGE}${RETEST_PART} | [full report](${REPORT_URL})\n${DURATION_DISPLAY}"
|
||||
}]
|
||||
}
|
||||
EOF
|
||||
)
|
||||
|
||||
# Send to webhook
|
||||
curl -X POST -H "Content-Type: application/json" -d "$PAYLOAD" "$REPORT_WEBHOOK_URL"
|
||||
- name: ci/write-job-summary
|
||||
if: always()
|
||||
env:
|
||||
STATUS_CHECK_URL: ${{ needs.generate-test-cycle.outputs.status_check_url }}
|
||||
TEST_TYPE: ${{ inputs.test_type }}
|
||||
PASSED: ${{ steps.final-results.outputs.passed }}
|
||||
FAILED: ${{ steps.final-results.outputs.failed }}
|
||||
PENDING: ${{ steps.final-results.outputs.pending }}
|
||||
TOTAL_SPECS: ${{ steps.final-results.outputs.total_specs }}
|
||||
FAILED_SPECS_COUNT: ${{ steps.final-results.outputs.failed_specs_count }}
|
||||
FAILED_SPECS: ${{ steps.final-results.outputs.failed_specs }}
|
||||
COMMIT_STATUS_MESSAGE: ${{ steps.final-results.outputs.commit_status_message }}
|
||||
FAILED_TESTS: ${{ steps.final-results.outputs.failed_tests }}
|
||||
DURATION_DISPLAY: ${{ steps.duration.outputs.duration_display }}
|
||||
RETEST_RESULT: ${{ needs.run-failed-tests.result }}
|
||||
run: |
|
||||
{
|
||||
echo "## E2E Test Results - Cypress ${TEST_TYPE}"
|
||||
echo ""
|
||||
|
||||
if [ "$FAILED" = "0" ]; then
|
||||
echo "All tests passed: **${PASSED} passed**"
|
||||
else
|
||||
echo "<details>"
|
||||
echo "<summary>${FAILED} failed, ${PASSED} passed</summary>"
|
||||
echo ""
|
||||
echo "| Test | File |"
|
||||
echo "|------|------|"
|
||||
echo "${FAILED_TESTS}"
|
||||
echo "</details>"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "### Calculation Outputs"
|
||||
echo ""
|
||||
echo "| Output | Value |"
|
||||
echo "|--------|-------|"
|
||||
echo "| passed | ${PASSED} |"
|
||||
echo "| failed | ${FAILED} |"
|
||||
echo "| pending | ${PENDING} |"
|
||||
echo "| total_specs | ${TOTAL_SPECS} |"
|
||||
echo "| failed_specs_count | ${FAILED_SPECS_COUNT} |"
|
||||
echo "| commit_status_message | ${COMMIT_STATUS_MESSAGE} |"
|
||||
echo "| failed_specs | ${FAILED_SPECS:-none} |"
|
||||
echo "| duration | ${DURATION_DISPLAY} |"
|
||||
if [ "$RETEST_RESULT" != "skipped" ]; then
|
||||
echo "| retested | Yes |"
|
||||
else
|
||||
echo "| retested | No |"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "---"
|
||||
echo "[View Full Report](${STATUS_CHECK_URL})"
|
||||
} >> $GITHUB_STEP_SUMMARY
|
||||
- name: ci/assert-results
|
||||
env:
|
||||
SUMMARY_OUTCOME: ${{ steps.summary.outcome }}
|
||||
run: |
|
||||
[ "${{ steps.final-results.outputs.failed }}" = "0" ]
|
||||
|
||||
update-success-status:
|
||||
runs-on: ubuntu-24.04
|
||||
permissions:
|
||||
contents: read
|
||||
statuses: write
|
||||
if: always() && needs.report.result == 'success' && needs.calculate-results.result == 'success'
|
||||
needs:
|
||||
- generate-test-cycle
|
||||
- calculate-results
|
||||
- report
|
||||
steps:
|
||||
- uses: mattermost/actions/delivery/update-commit-status@f324ac89b05cc3511cb06e60642ac2fb829f0a63
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ github.token }}
|
||||
with:
|
||||
repository_full_name: ${{ github.repository }}
|
||||
commit_sha: ${{ inputs.commit_sha }}
|
||||
context: ${{ inputs.context_name }}
|
||||
description: "${{ needs.report.outputs.commit_status_message }}, ${{ needs.report.outputs.duration }}, image_tag:${{ inputs.server_image_tag }}${{ inputs.server_image_aliases && format(' ({0})', inputs.server_image_aliases) || '' }}"
|
||||
status: success
|
||||
target_url: ${{ needs.generate-test-cycle.outputs.status_check_url }}
|
||||
|
||||
update-failure-status:
|
||||
runs-on: ubuntu-24.04
|
||||
permissions:
|
||||
contents: read
|
||||
statuses: write
|
||||
if: always() && (needs.report.result != 'success' || needs.calculate-results.result != 'success')
|
||||
needs:
|
||||
- generate-test-cycle
|
||||
- calculate-results
|
||||
- report
|
||||
steps:
|
||||
- uses: mattermost/actions/delivery/update-commit-status@f324ac89b05cc3511cb06e60642ac2fb829f0a63
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ github.token }}
|
||||
with:
|
||||
repository_full_name: ${{ github.repository }}
|
||||
commit_sha: ${{ inputs.commit_sha }}
|
||||
context: ${{ inputs.context_name }}
|
||||
description: "${{ needs.report.outputs.commit_status_message }}, ${{ needs.report.outputs.duration }}, image_tag:${{ inputs.server_image_tag }}${{ inputs.server_image_aliases && format(' ({0})', inputs.server_image_aliases) || '' }}"
|
||||
status: failure
|
||||
target_url: ${{ needs.generate-test-cycle.outputs.status_check_url }}
|
||||
[ "$SUMMARY_OUTCOME" = "success" ]
|
||||
|
||||
@@ -62,9 +62,7 @@ on:
|
||||
CWS_EXTRA_HTTP_HEADERS:
|
||||
required: false
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
statuses: write
|
||||
permissions: {}
|
||||
|
||||
jobs:
|
||||
generate-build-variables:
|
||||
@@ -80,10 +78,15 @@ jobs:
|
||||
id: build-vars
|
||||
env:
|
||||
COMMIT_SHA: ${{ inputs.commit_sha }}
|
||||
PR_NUMBER: ${{ inputs.pr_number }}
|
||||
INPUT_REF_BRANCH: ${{ inputs.ref_branch }}
|
||||
INPUT_REPORT_TYPE: ${{ inputs.report_type }}
|
||||
INPUT_SERVER_EDITION: ${{ inputs.server_edition }}
|
||||
INPUT_SERVER_IMAGE_ALIASES: ${{ inputs.server_image_aliases }}
|
||||
INPUT_SERVER_IMAGE_REPO: ${{ inputs.server_image_repo }}
|
||||
INPUT_SERVER_IMAGE_TAG: ${{ inputs.server_image_tag }}
|
||||
RUN_ID: ${{ github.run_id }}
|
||||
PR_NUMBER: ${{ inputs.pr_number }}
|
||||
RUN_ATTEMPT: ${{ github.run_attempt }}
|
||||
RUN_ID: ${{ github.run_id }}
|
||||
run: |
|
||||
# Use provided server_image_tag or derive from commit SHA
|
||||
if [ -n "$INPUT_SERVER_IMAGE_TAG" ]; then
|
||||
@@ -105,7 +108,7 @@ jobs:
|
||||
# build on that branch instead of treating each image tag as its
|
||||
# own "branch". PR and commit-only fallback paths keep their
|
||||
# synthetic prefix because there's no real branch to use.
|
||||
REF_BRANCH="${{ inputs.ref_branch }}"
|
||||
REF_BRANCH="${INPUT_REF_BRANCH}"
|
||||
if [ -n "$PR_NUMBER" ]; then
|
||||
echo "branch=pr-${PR_NUMBER}" >> $GITHUB_OUTPUT
|
||||
elif [ -n "$REF_BRANCH" ]; then
|
||||
@@ -115,8 +118,8 @@ jobs:
|
||||
fi
|
||||
|
||||
# Determine server image name
|
||||
EDITION="${{ inputs.server_edition }}"
|
||||
REPO="${{ inputs.server_image_repo }}"
|
||||
EDITION="${INPUT_SERVER_EDITION}"
|
||||
REPO="${INPUT_SERVER_IMAGE_REPO}"
|
||||
REPO="${REPO:-mattermostdevelopment}"
|
||||
case "$EDITION" in
|
||||
fips) IMAGE_NAME="mattermost-enterprise-fips-edition" ;;
|
||||
@@ -127,7 +130,7 @@ jobs:
|
||||
echo "server_image=${SERVER_IMAGE}" >> $GITHUB_OUTPUT
|
||||
|
||||
# Validate server_image_aliases format if provided
|
||||
ALIASES="${{ inputs.server_image_aliases }}"
|
||||
ALIASES="${INPUT_SERVER_IMAGE_ALIASES}"
|
||||
if [ -n "$ALIASES" ] && ! [[ "$ALIASES" =~ ^[a-zA-Z0-9._,\ -]+$ ]]; then
|
||||
echo "::error::Invalid server_image_aliases format: ${ALIASES}"
|
||||
exit 1
|
||||
@@ -141,7 +144,7 @@ jobs:
|
||||
fi
|
||||
|
||||
# Generate context name suffix based on report type
|
||||
REPORT_TYPE="${{ inputs.report_type }}"
|
||||
REPORT_TYPE="${INPUT_REPORT_TYPE}"
|
||||
case "$REPORT_TYPE" in
|
||||
MASTER) echo "context_suffix=/master" >> $GITHUB_OUTPUT ;;
|
||||
RELEASE) echo "context_suffix=/release" >> $GITHUB_OUTPUT ;;
|
||||
@@ -160,67 +163,29 @@ jobs:
|
||||
steps:
|
||||
- name: ci/post-skip-status
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
COMMIT_SHA: ${{ inputs.commit_sha }}
|
||||
CONTEXT_NAME: "e2e-test/cypress-full/${{ inputs.server_edition || 'enterprise' }}${{ needs.generate-build-variables.outputs.context_suffix }}"
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
GITHUB_RUN_ID: ${{ github.run_id }}
|
||||
run: |
|
||||
gh api repos/${{ github.repository }}/statuses/${COMMIT_SHA} \
|
||||
gh api "repos/${GITHUB_REPOSITORY}/statuses/${COMMIT_SHA}" \
|
||||
-f state=success \
|
||||
-f context="${CONTEXT_NAME}" \
|
||||
-f description="No E2E-relevant changes - skipped" \
|
||||
-f target_url="https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}"
|
||||
-f target_url="https://github.com/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}"
|
||||
echo "Posted success for ${CONTEXT_NAME}"
|
||||
|
||||
# ── Routing fork ─────────────────────────────────────────────────────
|
||||
# vars.E2E_USE_TEST_IO_DISPATCH selects between v1 (legacy) and v2.
|
||||
# vars.E2E_USE_STAGING_TEST_IO_URL toggles v2's staging vs production
|
||||
# endpoint (default: staging).
|
||||
|
||||
cypress-full-v1:
|
||||
cypress-full:
|
||||
needs:
|
||||
- generate-build-variables
|
||||
if: inputs.should_run != 'false' && vars.E2E_USE_TEST_IO_DISPATCH != 'true'
|
||||
permissions:
|
||||
contents: read
|
||||
statuses: write
|
||||
uses: ./.github/workflows/e2e-tests-cypress-template.yml
|
||||
with:
|
||||
test_type: full
|
||||
test_filter: '--stage="@prod" --excludeGroup="@te_only,@cloud_only,@high_availability" --sortFirst="@compliance_export,@elasticsearch,@ldap_group,@ldap" --sortLast="@saml,@keycloak,@plugin,@plugins_uninstall,@mfa,@license_removal"'
|
||||
workers: 40
|
||||
enabled_docker_services: "postgres inbucket minio openldap elasticsearch keycloak"
|
||||
commit_sha: ${{ inputs.commit_sha }}
|
||||
branch: ${{ needs.generate-build-variables.outputs.branch }}
|
||||
build_id: ${{ needs.generate-build-variables.outputs.build_id }}
|
||||
server_image_tag: ${{ needs.generate-build-variables.outputs.server_image_tag }}
|
||||
server_edition: ${{ inputs.server_edition }}
|
||||
server_image_repo: ${{ inputs.server_image_repo }}
|
||||
server_image_aliases: ${{ inputs.server_image_aliases }}
|
||||
server: ${{ inputs.server }}
|
||||
enable_reporting: ${{ inputs.enable_reporting }}
|
||||
report_type: ${{ inputs.report_type }}
|
||||
ref_branch: ${{ inputs.ref_branch }}
|
||||
pr_number: ${{ inputs.pr_number }}
|
||||
context_name: "e2e-test/cypress-full/${{ inputs.server_edition || 'enterprise' }}${{ needs.generate-build-variables.outputs.context_suffix }}"
|
||||
secrets:
|
||||
MM_LICENSE: ${{ secrets.MM_LICENSE }}
|
||||
AUTOMATION_DASHBOARD_URL: ${{ secrets.AUTOMATION_DASHBOARD_URL }}
|
||||
AUTOMATION_DASHBOARD_TOKEN: ${{ secrets.AUTOMATION_DASHBOARD_TOKEN }}
|
||||
PUSH_NOTIFICATION_SERVER: ${{ secrets.PUSH_NOTIFICATION_SERVER }}
|
||||
REPORT_WEBHOOK_URL: ${{ secrets.REPORT_WEBHOOK_URL }}
|
||||
CWS_URL: ${{ secrets.CWS_URL }}
|
||||
CWS_EXTRA_HTTP_HEADERS: ${{ secrets.CWS_EXTRA_HTTP_HEADERS }}
|
||||
|
||||
cypress-full-v2:
|
||||
needs:
|
||||
- generate-build-variables
|
||||
if: inputs.should_run != 'false' && vars.E2E_USE_TEST_IO_DISPATCH == 'true'
|
||||
if: inputs.should_run != 'false'
|
||||
permissions:
|
||||
contents: read
|
||||
statuses: write
|
||||
id-token: write
|
||||
pull-requests: write
|
||||
uses: ./.github/workflows/e2e-tests-cypress-template-v2.yml
|
||||
uses: ./.github/workflows/e2e-tests-cypress-template.yml
|
||||
with:
|
||||
test_type: full
|
||||
workers: 40
|
||||
|
||||
@@ -106,8 +106,6 @@ jobs:
|
||||
ref_branch: ${{ needs.generate-build-variables.outputs.ref_branch }}
|
||||
secrets:
|
||||
MM_LICENSE: "${{ secrets.MM_E2E_TEST_LICENSE_ONPREM_ENT }}"
|
||||
AWS_ACCESS_KEY_ID: "${{ secrets.CYPRESS_AWS_ACCESS_KEY_ID }}"
|
||||
AWS_SECRET_ACCESS_KEY: "${{ secrets.CYPRESS_AWS_SECRET_ACCESS_KEY }}"
|
||||
REPORT_WEBHOOK_URL: "${{ secrets.MM_E2E_REPORT_WEBHOOK_URL }}"
|
||||
|
||||
# Enterprise FIPS Edition
|
||||
@@ -157,6 +155,4 @@ jobs:
|
||||
ref_branch: ${{ needs.generate-build-variables.outputs.ref_branch }}
|
||||
secrets:
|
||||
MM_LICENSE: "${{ secrets.MM_E2E_TEST_LICENSE_ONPREM_ENT }}"
|
||||
AWS_ACCESS_KEY_ID: "${{ secrets.CYPRESS_AWS_ACCESS_KEY_ID }}"
|
||||
AWS_SECRET_ACCESS_KEY: "${{ secrets.CYPRESS_AWS_SECRET_ACCESS_KEY }}"
|
||||
REPORT_WEBHOOK_URL: "${{ secrets.MM_E2E_REPORT_WEBHOOK_URL }}"
|
||||
|
||||
@@ -105,8 +105,6 @@ jobs:
|
||||
ref_branch: ${{ needs.validate.outputs.ref_branch }}
|
||||
secrets:
|
||||
MM_LICENSE: "${{ secrets.MM_E2E_TEST_LICENSE_ONPREM_ENT }}"
|
||||
AWS_ACCESS_KEY_ID: "${{ secrets.CYPRESS_AWS_ACCESS_KEY_ID }}"
|
||||
AWS_SECRET_ACCESS_KEY: "${{ secrets.CYPRESS_AWS_SECRET_ACCESS_KEY }}"
|
||||
REPORT_WEBHOOK_URL: "${{ secrets.MM_E2E_REPORT_WEBHOOK_URL }}"
|
||||
|
||||
# Enterprise FIPS Edition
|
||||
@@ -160,6 +158,4 @@ jobs:
|
||||
ref_branch: ${{ needs.validate.outputs.ref_branch }}
|
||||
secrets:
|
||||
MM_LICENSE: "${{ secrets.MM_E2E_TEST_LICENSE_ONPREM_ENT }}"
|
||||
AWS_ACCESS_KEY_ID: "${{ secrets.CYPRESS_AWS_ACCESS_KEY_ID }}"
|
||||
AWS_SECRET_ACCESS_KEY: "${{ secrets.CYPRESS_AWS_SECRET_ACCESS_KEY }}"
|
||||
REPORT_WEBHOOK_URL: "${{ secrets.MM_E2E_REPORT_WEBHOOK_URL }}"
|
||||
|
||||
@@ -1,305 +0,0 @@
|
||||
---
|
||||
name: E2E Tests - Playwright Template (v2 - test system io dispatch)
|
||||
|
||||
# Delegates Playwright spec dispatch + reporting to test system io.
|
||||
# Authenticates via GitHub Actions OIDC; calling job MUST grant
|
||||
# `id-token: write`.
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
workers:
|
||||
description: "Number of parallel test system io dispatch workers"
|
||||
type: number
|
||||
required: false
|
||||
default: 8
|
||||
enabled_docker_services:
|
||||
description: "Space-separated list of docker services to enable"
|
||||
type: string
|
||||
required: false
|
||||
default: "postgres inbucket"
|
||||
|
||||
commit_sha:
|
||||
type: string
|
||||
required: true
|
||||
branch:
|
||||
type: string
|
||||
required: true
|
||||
build_id:
|
||||
type: string
|
||||
required: true
|
||||
server_image_tag:
|
||||
description: "Server image tag (e.g., master or short SHA)"
|
||||
type: string
|
||||
required: true
|
||||
server:
|
||||
type: string
|
||||
required: false
|
||||
default: onprem
|
||||
server_edition:
|
||||
description: "Server edition: enterprise (default), fips, or team"
|
||||
type: string
|
||||
required: false
|
||||
default: enterprise
|
||||
server_image_repo:
|
||||
description: "Docker registry: mattermostdevelopment (default) or mattermost"
|
||||
type: string
|
||||
required: false
|
||||
default: mattermostdevelopment
|
||||
server_image_aliases:
|
||||
description: "Comma-separated alias tags for description"
|
||||
type: string
|
||||
required: false
|
||||
|
||||
enable_reporting:
|
||||
type: boolean
|
||||
required: false
|
||||
default: false
|
||||
report_type:
|
||||
type: string
|
||||
required: false
|
||||
ref_branch:
|
||||
type: string
|
||||
required: false
|
||||
pr_number:
|
||||
type: string
|
||||
required: false
|
||||
context_name:
|
||||
description: "GitHub commit status context name"
|
||||
type: string
|
||||
required: true
|
||||
|
||||
playwright_project:
|
||||
description: "Playwright project name (passed to dispatch-begin metadata and dispatch-run --project=)."
|
||||
type: string
|
||||
required: false
|
||||
default: chrome
|
||||
playwright_retries:
|
||||
description: "Playwright --retries=N (per-spec, in-process retry of flaky tests)"
|
||||
type: number
|
||||
required: false
|
||||
default: 1
|
||||
retest_on_fail:
|
||||
description: "Re-dispatch failed dispatch units once (whole-spec retry, on top of Playwright --retries)"
|
||||
type: boolean
|
||||
required: false
|
||||
default: true
|
||||
|
||||
secrets:
|
||||
MM_LICENSE:
|
||||
required: false
|
||||
REPORT_WEBHOOK_URL:
|
||||
required: false
|
||||
|
||||
# Callers must grant: contents: read, statuses: write, id-token: write
|
||||
permissions:
|
||||
contents: read
|
||||
statuses: write
|
||||
id-token: write
|
||||
|
||||
env:
|
||||
SERVER_IMAGE: "${{ inputs.server_image_repo }}/${{ inputs.server_edition == 'fips' && 'mattermost-enterprise-fips-edition' || inputs.server_edition == 'team' && 'mattermost-team-edition' || 'mattermost-enterprise-edition' }}:${{ inputs.server_image_tag }}"
|
||||
|
||||
jobs:
|
||||
dispatch-begin:
|
||||
runs-on: ubuntu-24.04
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
statuses: write
|
||||
outputs:
|
||||
composite-identity-json: ${{ steps.composite-identity.outputs.composite-identity-json }}
|
||||
workers-matrix: ${{ steps.matrix.outputs.workers }}
|
||||
start_time: ${{ steps.matrix.outputs.start_time }}
|
||||
steps:
|
||||
- name: ci/checkout-repo
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
ref: ${{ inputs.commit_sha }}
|
||||
fetch-depth: 1
|
||||
- name: ci/composite-identity
|
||||
id: composite-identity
|
||||
env:
|
||||
CONTEXT_NAME: ${{ inputs.context_name }}
|
||||
MM_SHA: ${{ inputs.commit_sha }}
|
||||
MM_BRANCH: ${{ inputs.branch }}
|
||||
PR_NUMBER: ${{ inputs.pr_number }}
|
||||
run: |
|
||||
# Derive the test-system-io run name from the GitHub commit-status
|
||||
# context: drop the `e2e-test/` prefix (the framework name already
|
||||
# implies E2E in the dashboard) and swap remaining `/` for `-` so
|
||||
# the dashboard URL is path-safe. The commit-status context itself
|
||||
# stays unchanged elsewhere — branch protection rules depend on it.
|
||||
NAME="${CONTEXT_NAME#e2e-test/}"
|
||||
NAME="${NAME//\//-}"
|
||||
if [ -n "$PR_NUMBER" ]; then
|
||||
COMPOSITE_IDENTITY=$(jq -nc \
|
||||
--arg repo "${{ github.repository }}" \
|
||||
--arg sha "${MM_SHA}" \
|
||||
--arg run_id "${GITHUB_RUN_ID}" \
|
||||
--arg name "${NAME}" \
|
||||
--arg attempt "${GITHUB_RUN_ATTEMPT}" \
|
||||
--arg branch "${MM_BRANCH}" \
|
||||
--arg pr "${PR_NUMBER}" \
|
||||
'{repository:$repo, commit_sha:$sha, gh_run_id:$run_id, name:$name, gh_run_attempt:$attempt, branch:$branch, gh_pr_number:$pr}')
|
||||
else
|
||||
COMPOSITE_IDENTITY=$(jq -nc \
|
||||
--arg repo "${{ github.repository }}" \
|
||||
--arg sha "${MM_SHA}" \
|
||||
--arg run_id "${GITHUB_RUN_ID}" \
|
||||
--arg name "${NAME}" \
|
||||
--arg attempt "${GITHUB_RUN_ATTEMPT}" \
|
||||
--arg branch "${MM_BRANCH}" \
|
||||
'{repository:$repo, commit_sha:$sha, gh_run_id:$run_id, name:$name, gh_run_attempt:$attempt, branch:$branch}')
|
||||
fi
|
||||
echo "composite-identity-json=${COMPOSITE_IDENTITY}" >> $GITHUB_OUTPUT
|
||||
- name: ci/matrix
|
||||
id: matrix
|
||||
run: |
|
||||
echo "workers=$(jq -nc --argjson n ${{ inputs.workers }} '[range(1; $n+1)]')" >> $GITHUB_OUTPUT
|
||||
echo "start_time=$(date +%s)" >> $GITHUB_OUTPUT
|
||||
- name: ci/dispatch-begin
|
||||
uses: mattermost/mattermost-test-system-io/.github/actions/test-system-io-dispatch-begin@main
|
||||
with:
|
||||
use-staging: ${{ vars.E2E_USE_STAGING_TEST_IO_URL != 'false' }}
|
||||
framework: playwright
|
||||
repo-dir: ${{ github.workspace }}
|
||||
composite-identity: ${{ steps.composite-identity.outputs.composite-identity-json }}
|
||||
total-reports-expected: ${{ inputs.workers }}
|
||||
retest-on-fail: ${{ inputs.retest_on_fail }}
|
||||
playwright-project: ${{ inputs.playwright_project }}
|
||||
github-token: ${{ secrets.GITHUB_TOKEN }}
|
||||
commit-status-context: ${{ inputs.context_name }}
|
||||
image-tag: ${{ inputs.server_image_tag }}
|
||||
image-aliases: ${{ inputs.server_image_aliases }}
|
||||
|
||||
workers:
|
||||
name: dispatch-run-${{ matrix.worker_index }}
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 30
|
||||
needs: dispatch-begin
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
worker_index: ${{ fromJSON(needs.dispatch-begin.outputs.workers-matrix) }}
|
||||
env:
|
||||
COMPOSITE_IDENTITY: ${{ needs.dispatch-begin.outputs.composite-identity-json }}
|
||||
SERVER: "${{ inputs.server }}"
|
||||
MM_LICENSE: "${{ secrets.MM_LICENSE }}"
|
||||
ENABLED_DOCKER_SERVICES: "${{ inputs.enabled_docker_services }}"
|
||||
TEST: playwright
|
||||
BRANCH: "${{ inputs.branch }}"
|
||||
BUILD_ID: "${{ inputs.build_id }}"
|
||||
CI_BASE_URL: "full-test-${{ matrix.worker_index }}"
|
||||
steps:
|
||||
- name: ci/checkout-actions
|
||||
# Sparse-checkout just .github/actions from the triggering ref (master)
|
||||
# so the composite action below is available before the full checkout
|
||||
# overwrites the workspace with inputs.commit_sha.
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
sparse-checkout: .github/actions
|
||||
sparse-checkout-cone-mode: true
|
||||
- name: ci/runner-prep-for-openldap
|
||||
uses: ./.github/actions/runner-prep-openldap
|
||||
- name: ci/checkout-repo
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
ref: ${{ inputs.commit_sha }}
|
||||
fetch-depth: 0
|
||||
- name: ci/setup-node
|
||||
uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
|
||||
with:
|
||||
node-version-file: ".nvmrc"
|
||||
cache: npm
|
||||
cache-dependency-path: "e2e-tests/playwright/package-lock.json"
|
||||
- name: ci/get-webapp-node-modules
|
||||
working-directory: webapp
|
||||
run: make node_modules
|
||||
- name: ci/cloud-init
|
||||
working-directory: e2e-tests
|
||||
run: make cloud-init
|
||||
- name: ci/start-server
|
||||
working-directory: e2e-tests
|
||||
run: make start-server
|
||||
# Build once + run the `setup` project so per-spec dispatches can
|
||||
# pass --no-deps and skip plugin-load + server-deployment checks.
|
||||
- name: ci/prepare-playwright
|
||||
working-directory: e2e-tests/playwright
|
||||
run: |
|
||||
npm ci
|
||||
npm run build
|
||||
npx playwright test --project=setup
|
||||
- name: ci/dispatch-run
|
||||
uses: mattermost/mattermost-test-system-io/.github/actions/test-system-io-dispatch-run@main
|
||||
with:
|
||||
use-staging: ${{ vars.E2E_USE_STAGING_TEST_IO_URL != 'false' }}
|
||||
framework: playwright
|
||||
composite-identity: ${{ needs.dispatch-begin.outputs.composite-identity-json }}
|
||||
repo-dir: ${{ github.workspace }}
|
||||
artifacts-root: ${{ github.workspace }}/worker-artifacts
|
||||
github-token: ${{ secrets.GITHUB_TOKEN }}
|
||||
gh-job-name: dispatch-run-${{ matrix.worker_index }}
|
||||
playwright-retries: ${{ inputs.playwright_retries }}
|
||||
playwright-project: ${{ inputs.playwright_project }}
|
||||
- name: ci/cloud-teardown
|
||||
if: always()
|
||||
working-directory: e2e-tests
|
||||
run: make cloud-teardown
|
||||
- name: ci/upload-debug-artifacts
|
||||
if: always()
|
||||
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
|
||||
with:
|
||||
name: playwright-full-${{ inputs.server_edition }}-debug-${{ matrix.worker_index }}
|
||||
path: |
|
||||
e2e-tests/playwright/logs/
|
||||
e2e-tests/playwright/results/
|
||||
worker-artifacts/
|
||||
retention-days: 5
|
||||
if-no-files-found: ignore
|
||||
|
||||
report:
|
||||
runs-on: ubuntu-24.04
|
||||
needs: [dispatch-begin, workers]
|
||||
if: always()
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
statuses: write
|
||||
outputs:
|
||||
commit_status_description: ${{ steps.summary.outputs.commit_status_description }}
|
||||
webhook_payload: ${{ steps.summary.outputs.webhook_payload }}
|
||||
steps:
|
||||
- name: ci/checkout-repo
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
- name: ci/run-summary
|
||||
id: summary
|
||||
continue-on-error: true
|
||||
uses: mattermost/mattermost-test-system-io/.github/actions/test-system-io-summary@main
|
||||
with:
|
||||
use-staging: ${{ vars.E2E_USE_STAGING_TEST_IO_URL != 'false' }}
|
||||
composite-identity: ${{ needs.dispatch-begin.outputs.composite-identity-json }}
|
||||
framework: playwright
|
||||
report-type: ${{ inputs.report_type }}
|
||||
image-tag: ${{ inputs.server_image_tag }}
|
||||
image-aliases: ${{ inputs.server_image_aliases }}
|
||||
server-image: ${{ env.SERVER_IMAGE }}
|
||||
pr-number: ${{ inputs.pr_number }}
|
||||
ref-branch: ${{ inputs.ref_branch }}
|
||||
commit-status-context: ${{ inputs.context_name }}
|
||||
github-token: ${{ secrets.GITHUB_TOKEN }}
|
||||
- name: ci/publish-webhook
|
||||
if: inputs.enable_reporting && env.REPORT_WEBHOOK_URL != ''
|
||||
env:
|
||||
REPORT_WEBHOOK_URL: ${{ secrets.REPORT_WEBHOOK_URL }}
|
||||
PAYLOAD: ${{ steps.summary.outputs.webhook_payload }}
|
||||
run: |
|
||||
curl -X POST -H "Content-Type: application/json" -d "$PAYLOAD" "$REPORT_WEBHOOK_URL"
|
||||
- name: ci/assert-results
|
||||
env:
|
||||
SUMMARY_OUTCOME: ${{ steps.summary.outcome }}
|
||||
run: |
|
||||
[ "$SUMMARY_OUTCOME" = "success" ]
|
||||
@@ -1,29 +1,24 @@
|
||||
---
|
||||
name: E2E Tests - Playwright Template
|
||||
name: E2E Tests - Playwright Template (test system io dispatch)
|
||||
|
||||
# Delegates Playwright spec dispatch + reporting to test system io.
|
||||
# Authenticates via GitHub Actions OIDC; calling job MUST grant
|
||||
# `id-token: write`.
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
# Test configuration
|
||||
test_type:
|
||||
description: "Type of test run (smoke or full)"
|
||||
type: string
|
||||
required: true
|
||||
test_filter:
|
||||
description: "Test filter arguments (e.g., --grep @smoke)"
|
||||
type: string
|
||||
required: true
|
||||
workers:
|
||||
description: "Number of parallel shards"
|
||||
description: "Number of parallel test system io dispatch workers"
|
||||
type: number
|
||||
required: false
|
||||
default: 2
|
||||
default: 8
|
||||
enabled_docker_services:
|
||||
description: "Space-separated list of docker services to enable"
|
||||
type: string
|
||||
required: false
|
||||
default: "postgres inbucket"
|
||||
|
||||
# Common build variables
|
||||
commit_sha:
|
||||
type: string
|
||||
required: true
|
||||
@@ -52,11 +47,10 @@ on:
|
||||
required: false
|
||||
default: mattermostdevelopment
|
||||
server_image_aliases:
|
||||
description: "Comma-separated alias tags for description (e.g., 'release-11.4, release-11')"
|
||||
description: "Comma-separated alias tags for description"
|
||||
type: string
|
||||
required: false
|
||||
|
||||
# Reporting options
|
||||
enable_reporting:
|
||||
type: boolean
|
||||
required: false
|
||||
@@ -65,99 +59,222 @@ on:
|
||||
type: string
|
||||
required: false
|
||||
ref_branch:
|
||||
description: "Source branch name for webhook messages (e.g., 'master' or 'release-11.4')"
|
||||
type: string
|
||||
required: false
|
||||
pr_number:
|
||||
type: string
|
||||
required: false
|
||||
|
||||
# Commit status configuration
|
||||
context_name:
|
||||
description: "GitHub commit status context name"
|
||||
type: string
|
||||
required: true
|
||||
|
||||
outputs:
|
||||
passed:
|
||||
description: "Number of passed tests"
|
||||
value: ${{ jobs.report.outputs.passed }}
|
||||
failed:
|
||||
description: "Number of failed tests"
|
||||
value: ${{ jobs.report.outputs.failed }}
|
||||
report_url:
|
||||
description: "URL to test report on S3"
|
||||
value: ${{ jobs.report.outputs.report_url }}
|
||||
playwright_project:
|
||||
description: "Playwright project name (passed to dispatch-begin metadata and dispatch-run --project=)."
|
||||
type: string
|
||||
required: false
|
||||
default: chrome
|
||||
playwright_retries:
|
||||
description: "Playwright --retries=N (per-spec, in-process retry of flaky tests)"
|
||||
type: number
|
||||
required: false
|
||||
default: 1
|
||||
retest_on_fail:
|
||||
description: "Re-dispatch failed dispatch units once (whole-spec retry, on top of Playwright --retries)"
|
||||
type: boolean
|
||||
required: false
|
||||
default: true
|
||||
|
||||
secrets:
|
||||
MM_LICENSE:
|
||||
required: false
|
||||
REPORT_WEBHOOK_URL:
|
||||
required: false
|
||||
AWS_ACCESS_KEY_ID:
|
||||
required: true
|
||||
AWS_SECRET_ACCESS_KEY:
|
||||
required: true
|
||||
|
||||
# Callers must grant at least these scopes on the job that uses this workflow
|
||||
# Callers must grant: contents: read, statuses: write, id-token: write
|
||||
permissions:
|
||||
contents: read
|
||||
statuses: write
|
||||
id-token: write
|
||||
|
||||
env:
|
||||
SERVER_IMAGE: "${{ inputs.server_image_repo }}/${{ inputs.server_edition == 'fips' && 'mattermost-enterprise-fips-edition' || inputs.server_edition == 'team' && 'mattermost-team-edition' || 'mattermost-enterprise-edition' }}:${{ inputs.server_image_tag }}"
|
||||
|
||||
jobs:
|
||||
update-initial-status:
|
||||
runs-on: ubuntu-24.04
|
||||
steps:
|
||||
- name: ci/set-initial-status
|
||||
uses: mattermost/actions/delivery/update-commit-status@f324ac89b05cc3511cb06e60642ac2fb829f0a63
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ github.token }}
|
||||
with:
|
||||
repository_full_name: ${{ github.repository }}
|
||||
commit_sha: ${{ inputs.commit_sha }}
|
||||
context: ${{ inputs.context_name }}
|
||||
description: "tests running, image_tag:${{ inputs.server_image_tag }}${{ inputs.server_image_aliases && format(' ({0})', inputs.server_image_aliases) || '' }}"
|
||||
status: pending
|
||||
|
||||
generate-test-variables:
|
||||
prepare-run:
|
||||
runs-on: ubuntu-24.04
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
statuses: write
|
||||
outputs:
|
||||
workers: "${{ steps.generate-workers.outputs.workers }}"
|
||||
start_time: "${{ steps.generate-workers.outputs.start_time }}"
|
||||
composite-identity-json: ${{ steps.composite-identity.outputs.composite-identity-json }}
|
||||
workers-matrix: ${{ steps.matrix.outputs.workers }}
|
||||
start_time: ${{ steps.matrix.outputs.start_time }}
|
||||
steps:
|
||||
- name: ci/generate-workers
|
||||
id: generate-workers
|
||||
- name: ci/checkout-repo
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
ref: ${{ inputs.commit_sha }}
|
||||
fetch-depth: 1
|
||||
- name: ci/composite-identity
|
||||
id: composite-identity
|
||||
env:
|
||||
CONTEXT_NAME: ${{ inputs.context_name }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
MM_BRANCH: ${{ inputs.branch }}
|
||||
MM_SHA: ${{ inputs.commit_sha }}
|
||||
PR_NUMBER: ${{ inputs.pr_number }}
|
||||
run: |
|
||||
echo "workers=$(jq -nc '[range(1; ${{ inputs.workers }} + 1)]')" >> $GITHUB_OUTPUT
|
||||
# Derive the test-system-io run name from the GitHub commit-status
|
||||
# context: drop the `e2e-test/` prefix (the framework name already
|
||||
# implies E2E in the dashboard) and swap remaining `/` for `-` so
|
||||
# the dashboard URL is path-safe. The commit-status context itself
|
||||
# stays unchanged elsewhere — branch protection rules depend on it.
|
||||
NAME="${CONTEXT_NAME#e2e-test/}"
|
||||
NAME="${NAME//\//-}"
|
||||
if [ -n "$PR_NUMBER" ]; then
|
||||
COMPOSITE_IDENTITY=$(jq -nc \
|
||||
--arg repo "${GITHUB_REPOSITORY}" \
|
||||
--arg sha "${MM_SHA}" \
|
||||
--arg run_id "${GITHUB_RUN_ID}" \
|
||||
--arg name "${NAME}" \
|
||||
--arg attempt "${GITHUB_RUN_ATTEMPT}" \
|
||||
--arg branch "${MM_BRANCH}" \
|
||||
--arg pr "${PR_NUMBER}" \
|
||||
'{repository:$repo, commit_sha:$sha, gh_run_id:$run_id, name:$name, gh_run_attempt:$attempt, branch:$branch, gh_pr_number:$pr}')
|
||||
else
|
||||
COMPOSITE_IDENTITY=$(jq -nc \
|
||||
--arg repo "${GITHUB_REPOSITORY}" \
|
||||
--arg sha "${MM_SHA}" \
|
||||
--arg run_id "${GITHUB_RUN_ID}" \
|
||||
--arg name "${NAME}" \
|
||||
--arg attempt "${GITHUB_RUN_ATTEMPT}" \
|
||||
--arg branch "${MM_BRANCH}" \
|
||||
'{repository:$repo, commit_sha:$sha, gh_run_id:$run_id, name:$name, gh_run_attempt:$attempt, branch:$branch}')
|
||||
fi
|
||||
echo "composite-identity-json=${COMPOSITE_IDENTITY}" >> $GITHUB_OUTPUT
|
||||
- name: ci/matrix
|
||||
id: matrix
|
||||
env:
|
||||
INPUT_WORKERS: ${{ inputs.workers }}
|
||||
run: |
|
||||
echo "workers=$(jq -nc --argjson n "${INPUT_WORKERS}" '[range(1; $n+1)]')" >> $GITHUB_OUTPUT
|
||||
echo "start_time=$(date +%s)" >> $GITHUB_OUTPUT
|
||||
|
||||
run-tests:
|
||||
# Install webapp node_modules once via the shared webapp-setup action, then
|
||||
# workers restore the same stable cache. The node_modules cache is keyed only
|
||||
# on webapp/package-lock.json and is shared with webapp-ci.yml jobs.
|
||||
prep-deps:
|
||||
name: prep-deps
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 60
|
||||
continue-on-error: true
|
||||
needs:
|
||||
- generate-test-variables
|
||||
if: needs.generate-test-variables.result == 'success'
|
||||
timeout-minutes: 15
|
||||
permissions:
|
||||
contents: read
|
||||
steps:
|
||||
- name: ci/checkout-repo
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
ref: ${{ inputs.commit_sha }}
|
||||
fetch-depth: 1
|
||||
- name: ci/setup-webapp-node-modules
|
||||
uses: ./.github/actions/webapp-setup
|
||||
- name: ci/cache-playwright-deps
|
||||
# Caches node_modules + the rolled-up @mattermost/playwright-lib dist
|
||||
# so workers don't re-run rollup on every job.
|
||||
id: cache-playwright
|
||||
uses: actions/cache@cdf6c1fa76f9f475f3d7449005a359c84ca0f306 # v5.0.3
|
||||
with:
|
||||
path: |
|
||||
e2e-tests/playwright/node_modules
|
||||
e2e-tests/playwright/lib/dist
|
||||
e2e-tests/playwright/lib/node_modules
|
||||
key: e2e-playwright-deps-${{ runner.os }}-${{ hashFiles('e2e-tests/playwright/package-lock.json', 'e2e-tests/playwright/lib/src/**', 'e2e-tests/playwright/lib/package.json', 'e2e-tests/playwright/lib/rollup.config.js', 'e2e-tests/playwright/lib/tsconfig.json') }}
|
||||
- name: ci/install-playwright-deps
|
||||
# `npm ci` creates symlinks at node_modules/@mattermost/{client,types}
|
||||
# → webapp/platform/{client,types}; targets must already be built.
|
||||
# The postinstall then builds lib/dist via rollup. Skip browser
|
||||
# download here — chromium is cached separately below.
|
||||
if: steps.cache-playwright.outputs.cache-hit != 'true'
|
||||
working-directory: e2e-tests/playwright
|
||||
env:
|
||||
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD: "1"
|
||||
run: npm ci
|
||||
- name: ci/cache-playwright-browsers
|
||||
# Cache chromium binary (~150MB) keyed on the playwright lockfile so a
|
||||
# version bump invalidates. Restored by workers; no docker image needed.
|
||||
id: cache-pw-browsers
|
||||
uses: actions/cache@cdf6c1fa76f9f475f3d7449005a359c84ca0f306 # v5.0.3
|
||||
with:
|
||||
path: ~/.cache/ms-playwright
|
||||
key: playwright-browsers-${{ runner.os }}-${{ hashFiles('e2e-tests/playwright/package-lock.json') }}
|
||||
- name: ci/install-playwright-chromium
|
||||
if: steps.cache-pw-browsers.outputs.cache-hit != 'true'
|
||||
working-directory: e2e-tests/playwright
|
||||
run: npx playwright install chromium
|
||||
|
||||
# Register the Test System IO run AFTER prep-deps so workers reach
|
||||
# dispatch-run within Test System IO's inactivity window.
|
||||
dispatch-begin:
|
||||
runs-on: ubuntu-24.04
|
||||
needs: [prepare-run, prep-deps]
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
statuses: write
|
||||
steps:
|
||||
- name: ci/checkout-repo
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
ref: ${{ inputs.commit_sha }}
|
||||
fetch-depth: 1
|
||||
- name: ci/dispatch-begin
|
||||
uses: mattermost/mattermost-test-system-io/.github/actions/test-system-io-dispatch-begin@1631d8fcea24f4545a0b3b7f77e41c2fe0be4418 # 2026-07-28
|
||||
with:
|
||||
use-staging: ${{ vars.E2E_USE_STAGING_TEST_IO_URL != 'false' }}
|
||||
framework: playwright
|
||||
repo-dir: ${{ github.workspace }}
|
||||
composite-identity: ${{ needs.prepare-run.outputs.composite-identity-json }}
|
||||
total-reports-expected: ${{ inputs.workers }}
|
||||
retest-on-fail: ${{ inputs.retest_on_fail }}
|
||||
playwright-project: ${{ inputs.playwright_project }}
|
||||
github-token: ${{ secrets.GITHUB_TOKEN }}
|
||||
commit-status-context: ${{ inputs.context_name }}
|
||||
image-tag: ${{ inputs.server_image_tag }}
|
||||
image-aliases: ${{ inputs.server_image_aliases }}
|
||||
|
||||
workers:
|
||||
name: dispatch-run-${{ matrix.worker_index }}
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 30
|
||||
needs: [prepare-run, dispatch-begin]
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
worker_index: ${{ fromJSON(needs.generate-test-variables.outputs.workers) }}
|
||||
defaults:
|
||||
run:
|
||||
working-directory: e2e-tests
|
||||
worker_index: ${{ fromJSON(needs.prepare-run.outputs.workers-matrix) }}
|
||||
env:
|
||||
SERVER: "${{ inputs.server }}"
|
||||
COMPOSITE_IDENTITY: ${{ needs.prepare-run.outputs.composite-identity-json }}
|
||||
MM_LICENSE: "${{ secrets.MM_LICENSE }}"
|
||||
ENABLED_DOCKER_SERVICES: "${{ inputs.enabled_docker_services }}"
|
||||
TEST: playwright
|
||||
TEST_FILTER: "${{ inputs.test_filter }}"
|
||||
PW_SHARD: "${{ format('--shard={0}/{1}', matrix.worker_index, inputs.workers) }}"
|
||||
BRANCH: "${{ inputs.branch }}-${{ inputs.test_type }}"
|
||||
BRANCH: "${{ inputs.branch }}"
|
||||
BUILD_ID: "${{ inputs.build_id }}"
|
||||
CI_BASE_URL: "${{ inputs.test_type }}-test-${{ matrix.worker_index }}"
|
||||
CI_BASE_URL: "full-test-${{ matrix.worker_index }}"
|
||||
# testcontainers mode: global setup brings up Postgres, Inbucket, and the Mattermost
|
||||
# server. Optional sidecar services (openldap/keycloak/elasticsearch/opensearch/minio/
|
||||
# azurite) are left off on this release — related specs are not part of the suite here.
|
||||
PW_USE_TESTCONTAINERS: "true"
|
||||
PW_TESTCONTAINERS_SERVICES: ""
|
||||
# Keeps the server alive across this worker's per-spec dispatch invocations instead of
|
||||
# tearing down after each — otherwise ci/prepare-playwright's own global-teardown call would
|
||||
# stop the server that ci/dispatch-run needs next. ci/testcontainers-teardown does the final
|
||||
# teardown once the whole queue is done.
|
||||
PW_TESTCONTAINERS_REUSE: "true"
|
||||
steps:
|
||||
- name: ci/checkout-actions
|
||||
# Sparse-checkout just .github/actions from the triggering ref (master)
|
||||
@@ -168,431 +285,111 @@ jobs:
|
||||
persist-credentials: false
|
||||
sparse-checkout: .github/actions
|
||||
sparse-checkout-cone-mode: true
|
||||
- name: ci/runner-prep-for-openldap
|
||||
uses: ./.github/actions/runner-prep-openldap
|
||||
- name: ci/checkout-repo
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
ref: ${{ inputs.commit_sha }}
|
||||
fetch-depth: 0
|
||||
- name: ci/restore-npm-cache
|
||||
uses: ./.github/actions/restore-e2e-npm-cache
|
||||
- name: ci/setup-node
|
||||
uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
|
||||
- name: ci/setup-webapp-node-modules
|
||||
uses: ./.github/actions/webapp-setup
|
||||
with:
|
||||
node-version-file: ".nvmrc"
|
||||
- name: ci/npm-cache-verify
|
||||
# Heal any partial/dangling entries left in the restored ~/.npm cache
|
||||
# before running `npm ci`. Avoids the intermittent EEXIST/ENOENT
|
||||
# failures in npm's cacache writer.
|
||||
run: npm cache verify
|
||||
- name: ci/get-webapp-node-modules
|
||||
working-directory: webapp
|
||||
run: make node_modules
|
||||
- name: ci/restore-playwright-image-cache
|
||||
# Cache the Playwright Docker image tar by the SHA of the files that pin
|
||||
# its version. Cache busts automatically when either file is edited to bump
|
||||
# the version. Avoids repeated MCR pulls which are frequently blocked by
|
||||
# Microsoft's CDN ("The request is blocked").
|
||||
id: playwright-image-cache
|
||||
uses: actions/cache@5a3ec84eff668545956fd18022155c47e93e2684 # v4.2.3
|
||||
read-only: "true"
|
||||
- name: ci/restore-playwright-deps
|
||||
uses: actions/cache/restore@cdf6c1fa76f9f475f3d7449005a359c84ca0f306 # v5.0.3
|
||||
with:
|
||||
path: /tmp/playwright-docker-image.tar
|
||||
key: playwright-docker-image-${{ hashFiles('e2e-tests/.ci/server.generate.sh', '.github/workflows/e2e-tests-playwright-template.yml') }}-${{ runner.os }}
|
||||
- name: ci/pre-pull-playwright-image
|
||||
# Load from cache when available; pull from MCR only on cache miss.
|
||||
# A single pull attempt is enough because the image is saved to the cache
|
||||
# tar for all future runs — no need for a retry loop.
|
||||
run: |
|
||||
set -euo pipefail
|
||||
IMAGE="mcr.microsoft.com/playwright:v1.59.1-noble"
|
||||
TAR="/tmp/playwright-docker-image.tar"
|
||||
if [ -f "${TAR}" ]; then
|
||||
echo "Loading Playwright image from GitHub Actions cache"
|
||||
docker load --input "${TAR}"
|
||||
else
|
||||
echo "Cache miss — pulling from MCR"
|
||||
docker pull "${IMAGE}"
|
||||
echo "Saving image to cache for future runs"
|
||||
docker save "${IMAGE}" --output "${TAR}"
|
||||
fi
|
||||
- name: ci/run-tests
|
||||
run: |
|
||||
make cloud-init
|
||||
make
|
||||
- name: ci/cloud-teardown
|
||||
path: |
|
||||
e2e-tests/playwright/node_modules
|
||||
e2e-tests/playwright/lib/dist
|
||||
e2e-tests/playwright/lib/node_modules
|
||||
key: e2e-playwright-deps-${{ runner.os }}-${{ hashFiles('e2e-tests/playwright/package-lock.json', 'e2e-tests/playwright/lib/src/**', 'e2e-tests/playwright/lib/package.json', 'e2e-tests/playwright/lib/rollup.config.js', 'e2e-tests/playwright/lib/tsconfig.json') }}
|
||||
fail-on-cache-miss: true
|
||||
- name: ci/restore-playwright-browsers
|
||||
uses: actions/cache/restore@cdf6c1fa76f9f475f3d7449005a359c84ca0f306 # v5.0.3
|
||||
with:
|
||||
path: ~/.cache/ms-playwright
|
||||
key: playwright-browsers-${{ runner.os }}-${{ hashFiles('e2e-tests/playwright/package-lock.json') }}
|
||||
fail-on-cache-miss: true
|
||||
# Brings up the stack via Testcontainers in `testcontainers` mode (global setup). Also runs the
|
||||
# `setup` project so per-spec dispatches can pass --no-deps and skip plugin-load +
|
||||
# server-deployment checks. node_modules, lib/dist, and chromium are all restored from cache.
|
||||
- name: ci/prepare-playwright
|
||||
working-directory: e2e-tests/playwright
|
||||
run: npx playwright test --project=setup
|
||||
- name: ci/dispatch-run
|
||||
uses: mattermost/mattermost-test-system-io/.github/actions/test-system-io-dispatch-run@1631d8fcea24f4545a0b3b7f77e41c2fe0be4418 # 2026-07-28
|
||||
with:
|
||||
use-staging: ${{ vars.E2E_USE_STAGING_TEST_IO_URL != 'false' }}
|
||||
framework: playwright
|
||||
composite-identity: ${{ needs.prepare-run.outputs.composite-identity-json }}
|
||||
repo-dir: ${{ github.workspace }}
|
||||
artifacts-root: ${{ github.workspace }}/worker-artifacts
|
||||
github-token: ${{ secrets.GITHUB_TOKEN }}
|
||||
gh-job-name: dispatch-run-${{ matrix.worker_index }}
|
||||
playwright-retries: ${{ inputs.playwright_retries }}
|
||||
playwright-project: ${{ inputs.playwright_project }}
|
||||
# The only step that actually tears down the stack: PW_TESTCONTAINERS_REUSE means every
|
||||
# earlier step (including ci/prepare-playwright's own global-teardown call) just adopts the
|
||||
# already-running server instead of stopping it. Also collects container logs and archives
|
||||
# the boot-env drift history (.env.testcontainers) before removing containers.
|
||||
- name: ci/testcontainers-teardown
|
||||
if: always()
|
||||
working-directory: e2e-tests/playwright
|
||||
run: npm run testcontainers:down
|
||||
|
||||
- name: ci/upload-debug-artifacts
|
||||
if: always()
|
||||
run: make cloud-teardown
|
||||
- name: ci/dump-docker-state-on-failure
|
||||
# Always run a final docker-state capture so failures unrelated to
|
||||
# openldap startup (e.g. server container later crashes) still produce
|
||||
# logs we can inspect. The script's own retry loop dumps openldap
|
||||
# state per-attempt; this step is a backstop covering the whole job.
|
||||
if: failure()
|
||||
run: |
|
||||
set +e
|
||||
DIAG="e2e-tests/docker-diagnostics/job-failure"
|
||||
mkdir -p "$DIAG"
|
||||
docker ps -a >"$DIAG/docker.ps.txt" 2>&1
|
||||
docker version >"$DIAG/docker.version.txt" 2>&1
|
||||
docker info >"$DIAG/docker.info.txt" 2>&1
|
||||
for c in $(docker ps -a --format '{{.Names}}'); do
|
||||
docker inspect "$c" >"$DIAG/$c.inspect.json" 2>&1
|
||||
docker logs "$c" >"$DIAG/$c.log" 2>&1
|
||||
done
|
||||
uname -a >"$DIAG/host.uname.txt" 2>&1
|
||||
free -m >"$DIAG/host.free.txt" 2>&1
|
||||
df -h >"$DIAG/host.df.txt" 2>&1
|
||||
sudo dmesg | tail -500 >"$DIAG/host.dmesg.tail.txt" 2>&1
|
||||
sudo dmesg | grep -iE 'apparmor|denied|oom|killed|openldap|slapd' >"$DIAG/host.dmesg.relevant.txt" 2>&1
|
||||
- name: ci/upload-docker-diagnostics
|
||||
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
|
||||
if: always()
|
||||
with:
|
||||
name: docker-diagnostics-playwright-${{ inputs.test_type }}-${{ inputs.server_edition }}-${{ matrix.worker_index }}
|
||||
path: e2e-tests/docker-diagnostics/
|
||||
retention-days: 7
|
||||
if-no-files-found: ignore
|
||||
- name: ci/upload-results
|
||||
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
|
||||
if: always()
|
||||
with:
|
||||
name: playwright-${{ inputs.test_type }}-${{ inputs.server_edition }}-results-${{ matrix.worker_index }}
|
||||
name: playwright-full-${{ inputs.server_edition }}-debug-${{ matrix.worker_index }}
|
||||
path: |
|
||||
e2e-tests/playwright/logs/
|
||||
e2e-tests/playwright/results/
|
||||
worker-artifacts/
|
||||
retention-days: 5
|
||||
|
||||
calculate-results:
|
||||
runs-on: ubuntu-24.04
|
||||
needs:
|
||||
- generate-test-variables
|
||||
- run-tests
|
||||
if: always() && needs.generate-test-variables.result == 'success'
|
||||
outputs:
|
||||
passed: ${{ steps.calculate.outputs.passed }}
|
||||
failed: ${{ steps.calculate.outputs.failed }}
|
||||
flaky: ${{ steps.calculate.outputs.flaky }}
|
||||
skipped: ${{ steps.calculate.outputs.skipped }}
|
||||
total_specs: ${{ steps.calculate.outputs.total_specs }}
|
||||
failed_specs: ${{ steps.calculate.outputs.failed_specs }}
|
||||
failed_specs_count: ${{ steps.calculate.outputs.failed_specs_count }}
|
||||
failed_tests: ${{ steps.calculate.outputs.failed_tests }}
|
||||
commit_status_message: ${{ steps.calculate.outputs.commit_status_message }}
|
||||
total: ${{ steps.calculate.outputs.total }}
|
||||
pass_rate: ${{ steps.calculate.outputs.pass_rate }}
|
||||
passing: ${{ steps.calculate.outputs.passing }}
|
||||
color: ${{ steps.calculate.outputs.color }}
|
||||
test_duration: ${{ steps.calculate.outputs.test_duration }}
|
||||
end_time: ${{ steps.record-end-time.outputs.end_time }}
|
||||
steps:
|
||||
- name: ci/checkout-repo
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
- name: ci/restore-npm-cache
|
||||
uses: ./.github/actions/restore-e2e-npm-cache
|
||||
- name: ci/setup-node
|
||||
uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
|
||||
with:
|
||||
node-version-file: ".nvmrc"
|
||||
- name: ci/download-shard-results
|
||||
uses: actions/download-artifact@70fc10c6e5e1ce46ad2ea6f2b72d43f7d47b13c3 # v8.0.0
|
||||
with:
|
||||
pattern: playwright-${{ inputs.test_type }}-${{ inputs.server_edition }}-results-*
|
||||
path: e2e-tests/playwright/shard-results/
|
||||
merge-multiple: true
|
||||
- name: ci/merge-shard-results
|
||||
working-directory: e2e-tests/playwright
|
||||
run: |
|
||||
mkdir -p results/reporter
|
||||
|
||||
# Merge blob reports using Playwright merge-reports (per docs)
|
||||
npm install --no-save @playwright/test
|
||||
npx playwright merge-reports --config merge.config.mjs ./shard-results/results/blob-report/
|
||||
- name: ci/calculate
|
||||
id: calculate
|
||||
uses: ./.github/actions/calculate-playwright-results
|
||||
with:
|
||||
original-results-path: e2e-tests/playwright/results/reporter/results.json
|
||||
- name: ci/upload-merged-results
|
||||
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
|
||||
with:
|
||||
name: playwright-${{ inputs.test_type }}-${{ inputs.server_edition }}-results
|
||||
path: e2e-tests/playwright/results/
|
||||
retention-days: 5
|
||||
- name: ci/record-end-time
|
||||
id: record-end-time
|
||||
run: echo "end_time=$(date +%s)" >> $GITHUB_OUTPUT
|
||||
|
||||
# NB: retries for failing specs happen INLINE inside each shard's
|
||||
# `ci/run-tests` step (see e2e-tests/.ci/server.run_playwright.sh).
|
||||
# That reuses the already-running server+docker stack instead of
|
||||
# paying ~4-7 min to provision a fresh one here, and it correctly
|
||||
# handles the chrome + chrome-serial project split. The old
|
||||
# standalone `run-failed-tests` job was removed because it was
|
||||
# invoking `--project=chrome` against specs that only exist in
|
||||
# chrome-serial, causing the retest to run zero tests.
|
||||
if-no-files-found: ignore
|
||||
|
||||
report:
|
||||
runs-on: ubuntu-24.04
|
||||
needs:
|
||||
- generate-test-variables
|
||||
- run-tests
|
||||
- calculate-results
|
||||
if: always() && needs.calculate-results.result == 'success'
|
||||
needs: [prepare-run, dispatch-begin, workers]
|
||||
if: always()
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
statuses: write
|
||||
outputs:
|
||||
passed: "${{ steps.final-results.outputs.passed }}"
|
||||
failed: "${{ steps.final-results.outputs.failed }}"
|
||||
commit_status_message: "${{ steps.final-results.outputs.commit_status_message }}"
|
||||
report_url: "${{ steps.upload-to-s3.outputs.report_url }}"
|
||||
duration: "${{ steps.duration.outputs.duration }}"
|
||||
duration_display: "${{ steps.duration.outputs.duration_display }}"
|
||||
retest_display: "${{ steps.duration.outputs.retest_display }}"
|
||||
defaults:
|
||||
run:
|
||||
working-directory: e2e-tests
|
||||
commit_status_description: ${{ steps.summary.outputs.commit_status_description }}
|
||||
webhook_payload: ${{ steps.summary.outputs.webhook_payload }}
|
||||
steps:
|
||||
- name: ci/checkout-repo
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
- name: ci/restore-npm-cache
|
||||
uses: ./.github/actions/restore-e2e-npm-cache
|
||||
- name: ci/setup-node
|
||||
uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
|
||||
- name: ci/run-summary
|
||||
id: summary
|
||||
continue-on-error: true
|
||||
uses: mattermost/mattermost-test-system-io/.github/actions/test-system-io-summary@1631d8fcea24f4545a0b3b7f77e41c2fe0be4418 # 2026-07-28
|
||||
with:
|
||||
node-version-file: ".nvmrc"
|
||||
|
||||
# Download merged results (uploaded by calculate-results). These blob
|
||||
# reports already include the inline per-shard retry results, so no
|
||||
# separate retest download/merge is needed here.
|
||||
- name: ci/download-results
|
||||
uses: actions/download-artifact@70fc10c6e5e1ce46ad2ea6f2b72d43f7d47b13c3 # v8.0.0
|
||||
with:
|
||||
name: playwright-${{ inputs.test_type }}-${{ inputs.server_edition }}-results
|
||||
path: e2e-tests/playwright/results/
|
||||
|
||||
# Calculate final results. Tests that failed in the first pass but
|
||||
# passed on inline retry are reported as `flaky`, not `failed`, so
|
||||
# no retest-results-path is needed.
|
||||
- name: ci/calculate-results
|
||||
id: final-results
|
||||
uses: ./.github/actions/calculate-playwright-results
|
||||
with:
|
||||
original-results-path: e2e-tests/playwright/results/reporter/results.json
|
||||
|
||||
- name: ci/aws-configure
|
||||
uses: aws-actions/configure-aws-credentials@8df5847569e6427dd6c4fb1cf565c83acfa8afa7 # v6.0.0
|
||||
with:
|
||||
aws-region: us-east-1
|
||||
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
||||
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
||||
- name: ci/upload-to-s3
|
||||
id: upload-to-s3
|
||||
env:
|
||||
AWS_REGION: us-east-1
|
||||
AWS_S3_BUCKET: mattermost-cypress-report
|
||||
PR_NUMBER: "${{ inputs.pr_number }}"
|
||||
RUN_ID: "${{ github.run_id }}"
|
||||
COMMIT_SHA: "${{ inputs.commit_sha }}"
|
||||
TEST_TYPE: "${{ inputs.test_type }}"
|
||||
run: |
|
||||
LOCAL_RESULTS_PATH="playwright/results/"
|
||||
|
||||
# Use PR number if available, otherwise use commit SHA prefix
|
||||
if [ -n "$PR_NUMBER" ]; then
|
||||
S3_PATH="server-pr-${PR_NUMBER}/e2e-reports/playwright-${TEST_TYPE}/${RUN_ID}"
|
||||
else
|
||||
S3_PATH="server-commit-${COMMIT_SHA::7}/e2e-reports/playwright-${TEST_TYPE}/${RUN_ID}"
|
||||
fi
|
||||
|
||||
if [[ -d "$LOCAL_RESULTS_PATH" ]]; then
|
||||
aws s3 sync "$LOCAL_RESULTS_PATH" "s3://${AWS_S3_BUCKET}/${S3_PATH}/results/" \
|
||||
--acl public-read --cache-control "no-cache"
|
||||
fi
|
||||
|
||||
REPORT_URL="https://${AWS_S3_BUCKET}.s3.amazonaws.com/${S3_PATH}/results/reporter/index.html"
|
||||
echo "report_url=$REPORT_URL" >> "$GITHUB_OUTPUT"
|
||||
- name: ci/compute-duration
|
||||
id: duration
|
||||
env:
|
||||
START_TIME: ${{ needs.generate-test-variables.outputs.start_time }}
|
||||
FLAKY_COUNT: ${{ steps.final-results.outputs.flaky }}
|
||||
TEST_DURATION: ${{ steps.final-results.outputs.test_duration }}
|
||||
run: |
|
||||
NOW=$(date +%s)
|
||||
ELAPSED=$((NOW - START_TIME))
|
||||
MINUTES=$((ELAPSED / 60))
|
||||
SECONDS=$((ELAPSED % 60))
|
||||
DURATION="${MINUTES}m ${SECONDS}s"
|
||||
|
||||
# Duration icons: >20m high alert, >15m warning, otherwise clock.
|
||||
# Retries now happen inline per-shard, so there's no separate
|
||||
# first-pass/re-run breakdown — the shard wall-clock already
|
||||
# includes any retries it needed.
|
||||
if [ "$MINUTES" -ge 20 ]; then
|
||||
DURATION_DISPLAY=":rotating_light: ${DURATION} | test: ${TEST_DURATION}"
|
||||
elif [ "$MINUTES" -ge 15 ]; then
|
||||
DURATION_DISPLAY=":warning: ${DURATION} | test: ${TEST_DURATION}"
|
||||
else
|
||||
DURATION_DISPLAY=":clock3: ${DURATION} | test: ${TEST_DURATION}"
|
||||
fi
|
||||
|
||||
# Flaky indicator: tests that failed first pass but passed on
|
||||
# inline retry. Signals retries did run.
|
||||
if [ -n "$FLAKY_COUNT" ] && [ "$FLAKY_COUNT" -gt 0 ] 2>/dev/null; then
|
||||
RETEST_DISPLAY=":repeat: ${FLAKY_COUNT} flaky"
|
||||
else
|
||||
RETEST_DISPLAY=""
|
||||
fi
|
||||
|
||||
echo "duration=${DURATION}" >> $GITHUB_OUTPUT
|
||||
echo "duration_display=${DURATION_DISPLAY}" >> $GITHUB_OUTPUT
|
||||
echo "retest_display=${RETEST_DISPLAY}" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: ci/publish-report
|
||||
use-staging: ${{ vars.E2E_USE_STAGING_TEST_IO_URL != 'false' }}
|
||||
composite-identity: ${{ needs.prepare-run.outputs.composite-identity-json }}
|
||||
framework: playwright
|
||||
report-type: ${{ inputs.report_type }}
|
||||
image-tag: ${{ inputs.server_image_tag }}
|
||||
image-aliases: ${{ inputs.server_image_aliases }}
|
||||
server-image: ${{ env.SERVER_IMAGE }}
|
||||
pr-number: ${{ inputs.pr_number }}
|
||||
ref-branch: ${{ inputs.ref_branch }}
|
||||
commit-status-context: ${{ inputs.context_name }}
|
||||
github-token: ${{ secrets.GITHUB_TOKEN }}
|
||||
- name: ci/publish-webhook
|
||||
if: inputs.enable_reporting && env.REPORT_WEBHOOK_URL != ''
|
||||
env:
|
||||
REPORT_WEBHOOK_URL: ${{ secrets.REPORT_WEBHOOK_URL }}
|
||||
COMMIT_STATUS_MESSAGE: ${{ steps.final-results.outputs.commit_status_message }}
|
||||
COLOR: ${{ steps.final-results.outputs.color }}
|
||||
REPORT_URL: ${{ steps.upload-to-s3.outputs.report_url }}
|
||||
TEST_TYPE: ${{ inputs.test_type }}
|
||||
REPORT_TYPE: ${{ inputs.report_type }}
|
||||
COMMIT_SHA: ${{ inputs.commit_sha }}
|
||||
REF_BRANCH: ${{ inputs.ref_branch }}
|
||||
PR_NUMBER: ${{ inputs.pr_number }}
|
||||
DURATION_DISPLAY: ${{ steps.duration.outputs.duration_display }}
|
||||
RETEST_DISPLAY: ${{ steps.duration.outputs.retest_display }}
|
||||
PAYLOAD: ${{ steps.summary.outputs.webhook_payload }}
|
||||
run: |
|
||||
# Capitalize test type
|
||||
TEST_TYPE_CAP=$(echo "$TEST_TYPE" | sed 's/.*/\u&/')
|
||||
|
||||
# Build source line based on report type
|
||||
COMMIT_SHORT="${COMMIT_SHA::7}"
|
||||
COMMIT_URL="https://github.com/${{ github.repository }}/commit/${COMMIT_SHA}"
|
||||
if [ "$REPORT_TYPE" = "RELEASE_CUT" ]; then
|
||||
SOURCE_LINE=":github_round: [${COMMIT_SHORT}](${COMMIT_URL}) on \`${REF_BRANCH}\`"
|
||||
elif [ "$REPORT_TYPE" = "MASTER" ] || [ "$REPORT_TYPE" = "RELEASE" ]; then
|
||||
SOURCE_LINE=":git_merge: [${COMMIT_SHORT}](${COMMIT_URL}) on \`${REF_BRANCH}\`"
|
||||
else
|
||||
SOURCE_LINE=":open-pull-request: [mattermost-pr-${PR_NUMBER}](https://github.com/${{ github.repository }}/pull/${PR_NUMBER})"
|
||||
fi
|
||||
|
||||
# Build retest part for message
|
||||
RETEST_PART=""
|
||||
if [ -n "$RETEST_DISPLAY" ]; then
|
||||
RETEST_PART=" | ${RETEST_DISPLAY}"
|
||||
fi
|
||||
|
||||
# Build payload with attachments
|
||||
PAYLOAD=$(cat <<EOF
|
||||
{
|
||||
"username": "E2E Test",
|
||||
"icon_url": "https://mattermost.com/wp-content/uploads/2022/02/icon_WS.png",
|
||||
"attachments": [{
|
||||
"color": "${COLOR}",
|
||||
"text": "**Results - Playwright ${TEST_TYPE_CAP} Tests**\n\n${SOURCE_LINE}\n:docker: \`${{ env.SERVER_IMAGE }}\`\n${COMMIT_STATUS_MESSAGE}${RETEST_PART} | [full report](${REPORT_URL})\n${DURATION_DISPLAY}"
|
||||
}]
|
||||
}
|
||||
EOF
|
||||
)
|
||||
|
||||
# Send to webhook
|
||||
curl -X POST -H "Content-Type: application/json" -d "$PAYLOAD" "$REPORT_WEBHOOK_URL"
|
||||
- name: ci/write-job-summary
|
||||
if: always()
|
||||
env:
|
||||
REPORT_URL: ${{ steps.upload-to-s3.outputs.report_url }}
|
||||
TEST_TYPE: ${{ inputs.test_type }}
|
||||
PASSED: ${{ steps.final-results.outputs.passed }}
|
||||
FAILED: ${{ steps.final-results.outputs.failed }}
|
||||
FLAKY: ${{ steps.final-results.outputs.flaky }}
|
||||
SKIPPED: ${{ steps.final-results.outputs.skipped }}
|
||||
TOTAL_SPECS: ${{ steps.final-results.outputs.total_specs }}
|
||||
FAILED_SPECS_COUNT: ${{ steps.final-results.outputs.failed_specs_count }}
|
||||
FAILED_SPECS: ${{ steps.final-results.outputs.failed_specs }}
|
||||
COMMIT_STATUS_MESSAGE: ${{ steps.final-results.outputs.commit_status_message }}
|
||||
FAILED_TESTS: ${{ steps.final-results.outputs.failed_tests }}
|
||||
DURATION_DISPLAY: ${{ steps.duration.outputs.duration_display }}
|
||||
run: |
|
||||
{
|
||||
echo "## E2E Test Results - Playwright ${TEST_TYPE}"
|
||||
echo ""
|
||||
|
||||
if [ "$FAILED" = "0" ]; then
|
||||
echo "All tests passed: **${PASSED} passed**"
|
||||
else
|
||||
echo "<details>"
|
||||
echo "<summary>${FAILED} failed, ${PASSED} passed</summary>"
|
||||
echo ""
|
||||
echo "| Test | File |"
|
||||
echo "|------|------|"
|
||||
echo "${FAILED_TESTS}"
|
||||
echo "</details>"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "### Calculation Outputs"
|
||||
echo ""
|
||||
echo "| Output | Value |"
|
||||
echo "|--------|-------|"
|
||||
echo "| passed | ${PASSED} |"
|
||||
echo "| failed | ${FAILED} |"
|
||||
echo "| flaky | ${FLAKY} |"
|
||||
echo "| skipped | ${SKIPPED} |"
|
||||
echo "| total_specs | ${TOTAL_SPECS} |"
|
||||
echo "| failed_specs_count | ${FAILED_SPECS_COUNT} |"
|
||||
echo "| commit_status_message | ${COMMIT_STATUS_MESSAGE} |"
|
||||
echo "| failed_specs | ${FAILED_SPECS:-none} |"
|
||||
echo "| duration | ${DURATION_DISPLAY} |"
|
||||
# Flaky > 0 means some tests needed the inline retry to pass.
|
||||
if [ -n "$FLAKY" ] && [ "$FLAKY" -gt 0 ] 2>/dev/null; then
|
||||
echo "| retried (flaky) | ${FLAKY} |"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "---"
|
||||
echo "[View Full Report](${REPORT_URL})"
|
||||
} >> $GITHUB_STEP_SUMMARY
|
||||
- name: ci/assert-results
|
||||
env:
|
||||
SUMMARY_OUTCOME: ${{ steps.summary.outcome }}
|
||||
run: |
|
||||
[ "${{ steps.final-results.outputs.failed }}" = "0" ]
|
||||
|
||||
update-success-status:
|
||||
runs-on: ubuntu-24.04
|
||||
if: always() && needs.report.result == 'success' && needs.calculate-results.result == 'success'
|
||||
needs:
|
||||
- calculate-results
|
||||
- report
|
||||
steps:
|
||||
- uses: mattermost/actions/delivery/update-commit-status@f324ac89b05cc3511cb06e60642ac2fb829f0a63
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ github.token }}
|
||||
with:
|
||||
repository_full_name: ${{ github.repository }}
|
||||
commit_sha: ${{ inputs.commit_sha }}
|
||||
context: ${{ inputs.context_name }}
|
||||
description: "${{ needs.report.outputs.commit_status_message }}, ${{ needs.report.outputs.duration }}, image_tag:${{ inputs.server_image_tag }}${{ inputs.server_image_aliases && format(' ({0})', inputs.server_image_aliases) || '' }}"
|
||||
status: success
|
||||
target_url: ${{ needs.report.outputs.report_url }}
|
||||
|
||||
update-failure-status:
|
||||
runs-on: ubuntu-24.04
|
||||
if: always() && (needs.report.result != 'success' || needs.calculate-results.result != 'success')
|
||||
needs:
|
||||
- calculate-results
|
||||
- report
|
||||
steps:
|
||||
- uses: mattermost/actions/delivery/update-commit-status@f324ac89b05cc3511cb06e60642ac2fb829f0a63
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ github.token }}
|
||||
with:
|
||||
repository_full_name: ${{ github.repository }}
|
||||
commit_sha: ${{ inputs.commit_sha }}
|
||||
context: ${{ inputs.context_name }}
|
||||
description: "${{ needs.report.outputs.commit_status_message }}, ${{ needs.report.outputs.duration }}, image_tag:${{ inputs.server_image_tag }}${{ inputs.server_image_aliases && format(' ({0})', inputs.server_image_aliases) || '' }}"
|
||||
status: failure
|
||||
target_url: ${{ needs.report.outputs.report_url }}
|
||||
[ "$SUMMARY_OUTCOME" = "success" ]
|
||||
|
||||
@@ -51,14 +51,8 @@ on:
|
||||
required: false
|
||||
REPORT_WEBHOOK_URL:
|
||||
required: false
|
||||
AWS_ACCESS_KEY_ID:
|
||||
required: true
|
||||
AWS_SECRET_ACCESS_KEY:
|
||||
required: true
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
statuses: write
|
||||
permissions: {}
|
||||
|
||||
jobs:
|
||||
generate-build-variables:
|
||||
@@ -74,10 +68,15 @@ jobs:
|
||||
id: build-vars
|
||||
env:
|
||||
COMMIT_SHA: ${{ inputs.commit_sha }}
|
||||
PR_NUMBER: ${{ inputs.pr_number }}
|
||||
INPUT_REF_BRANCH: ${{ inputs.ref_branch }}
|
||||
INPUT_REPORT_TYPE: ${{ inputs.report_type }}
|
||||
INPUT_SERVER_EDITION: ${{ inputs.server_edition }}
|
||||
INPUT_SERVER_IMAGE_ALIASES: ${{ inputs.server_image_aliases }}
|
||||
INPUT_SERVER_IMAGE_REPO: ${{ inputs.server_image_repo }}
|
||||
INPUT_SERVER_IMAGE_TAG: ${{ inputs.server_image_tag }}
|
||||
RUN_ID: ${{ github.run_id }}
|
||||
PR_NUMBER: ${{ inputs.pr_number }}
|
||||
RUN_ATTEMPT: ${{ github.run_attempt }}
|
||||
RUN_ID: ${{ github.run_id }}
|
||||
run: |
|
||||
# Use provided server_image_tag or derive from commit SHA
|
||||
if [ -n "$INPUT_SERVER_IMAGE_TAG" ]; then
|
||||
@@ -99,7 +98,7 @@ jobs:
|
||||
# build on that branch instead of treating each image tag as its
|
||||
# own "branch". PR and commit-only fallback paths keep their
|
||||
# synthetic prefix because there's no real branch to use.
|
||||
REF_BRANCH="${{ inputs.ref_branch }}"
|
||||
REF_BRANCH="${INPUT_REF_BRANCH}"
|
||||
if [ -n "$PR_NUMBER" ]; then
|
||||
echo "branch=pr-${PR_NUMBER}" >> $GITHUB_OUTPUT
|
||||
elif [ -n "$REF_BRANCH" ]; then
|
||||
@@ -109,8 +108,8 @@ jobs:
|
||||
fi
|
||||
|
||||
# Determine server image name
|
||||
EDITION="${{ inputs.server_edition }}"
|
||||
REPO="${{ inputs.server_image_repo }}"
|
||||
EDITION="${INPUT_SERVER_EDITION}"
|
||||
REPO="${INPUT_SERVER_IMAGE_REPO}"
|
||||
REPO="${REPO:-mattermostdevelopment}"
|
||||
case "$EDITION" in
|
||||
fips) IMAGE_NAME="mattermost-enterprise-fips-edition" ;;
|
||||
@@ -121,7 +120,7 @@ jobs:
|
||||
echo "server_image=${SERVER_IMAGE}" >> $GITHUB_OUTPUT
|
||||
|
||||
# Validate server_image_aliases format if provided
|
||||
ALIASES="${{ inputs.server_image_aliases }}"
|
||||
ALIASES="${INPUT_SERVER_IMAGE_ALIASES}"
|
||||
if [ -n "$ALIASES" ] && ! [[ "$ALIASES" =~ ^[a-zA-Z0-9._,\ -]+$ ]]; then
|
||||
echo "::error::Invalid server_image_aliases format: ${ALIASES}"
|
||||
exit 1
|
||||
@@ -135,7 +134,7 @@ jobs:
|
||||
fi
|
||||
|
||||
# Generate context name suffix based on report type
|
||||
REPORT_TYPE="${{ inputs.report_type }}"
|
||||
REPORT_TYPE="${INPUT_REPORT_TYPE}"
|
||||
case "$REPORT_TYPE" in
|
||||
MASTER) echo "context_suffix=/master" >> $GITHUB_OUTPUT ;;
|
||||
RELEASE) echo "context_suffix=/release" >> $GITHUB_OUTPUT ;;
|
||||
@@ -154,64 +153,29 @@ jobs:
|
||||
steps:
|
||||
- name: ci/post-skip-status
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
COMMIT_SHA: ${{ inputs.commit_sha }}
|
||||
CONTEXT_NAME: "e2e-test/playwright-full/${{ inputs.server_edition || 'enterprise' }}${{ needs.generate-build-variables.outputs.context_suffix }}"
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
GITHUB_RUN_ID: ${{ github.run_id }}
|
||||
run: |
|
||||
gh api repos/${{ github.repository }}/statuses/${COMMIT_SHA} \
|
||||
gh api "repos/${GITHUB_REPOSITORY}/statuses/${COMMIT_SHA}" \
|
||||
-f state=success \
|
||||
-f context="${CONTEXT_NAME}" \
|
||||
-f description="No E2E-relevant changes - skipped" \
|
||||
-f target_url="https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}"
|
||||
-f target_url="https://github.com/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}"
|
||||
echo "Posted success for ${CONTEXT_NAME}"
|
||||
|
||||
# ── Routing fork ─────────────────────────────────────────────────────
|
||||
# vars.E2E_USE_TEST_IO_DISPATCH selects between v1 (legacy) and v2.
|
||||
# vars.E2E_USE_STAGING_TEST_IO_URL toggles v2's staging vs production
|
||||
# endpoint (default: staging).
|
||||
|
||||
playwright-full-v1:
|
||||
playwright-full:
|
||||
needs:
|
||||
- generate-build-variables
|
||||
if: inputs.should_run != 'false' && vars.E2E_USE_TEST_IO_DISPATCH != 'true'
|
||||
permissions:
|
||||
contents: read
|
||||
statuses: write
|
||||
uses: ./.github/workflows/e2e-tests-playwright-template.yml
|
||||
with:
|
||||
test_type: full
|
||||
test_filter: "--grep-invert @visual"
|
||||
workers: 8
|
||||
enabled_docker_services: "postgres inbucket minio openldap elasticsearch keycloak"
|
||||
commit_sha: ${{ inputs.commit_sha }}
|
||||
branch: ${{ needs.generate-build-variables.outputs.branch }}
|
||||
build_id: ${{ needs.generate-build-variables.outputs.build_id }}
|
||||
server_image_tag: ${{ needs.generate-build-variables.outputs.server_image_tag }}
|
||||
server_edition: ${{ inputs.server_edition }}
|
||||
server_image_repo: ${{ inputs.server_image_repo }}
|
||||
server_image_aliases: ${{ inputs.server_image_aliases }}
|
||||
server: ${{ inputs.server }}
|
||||
enable_reporting: ${{ inputs.enable_reporting }}
|
||||
report_type: ${{ inputs.report_type }}
|
||||
ref_branch: ${{ inputs.ref_branch }}
|
||||
pr_number: ${{ inputs.pr_number }}
|
||||
context_name: "e2e-test/playwright-full/${{ inputs.server_edition || 'enterprise' }}${{ needs.generate-build-variables.outputs.context_suffix }}"
|
||||
secrets:
|
||||
MM_LICENSE: ${{ secrets.MM_LICENSE }}
|
||||
REPORT_WEBHOOK_URL: ${{ secrets.REPORT_WEBHOOK_URL }}
|
||||
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
||||
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
||||
|
||||
playwright-full-v2:
|
||||
needs:
|
||||
- generate-build-variables
|
||||
if: inputs.should_run != 'false' && vars.E2E_USE_TEST_IO_DISPATCH == 'true'
|
||||
if: inputs.should_run != 'false'
|
||||
permissions:
|
||||
contents: read
|
||||
statuses: write
|
||||
id-token: write
|
||||
pull-requests: write
|
||||
uses: ./.github/workflows/e2e-tests-playwright-template-v2.yml
|
||||
uses: ./.github/workflows/e2e-tests-playwright-template.yml
|
||||
with:
|
||||
workers: 10
|
||||
enabled_docker_services: "postgres inbucket"
|
||||
|
||||
@@ -93,12 +93,6 @@ if [ -n "${DIAGNOSTIC_WEBHOOK_URL:-}" ]; then
|
||||
mme2e_log "Diagnostic report upload enabled."
|
||||
fi
|
||||
|
||||
if [ -n "${AWS_S3_BUCKET:-}" ]; then
|
||||
: ${AWS_ACCESS_KEY_ID:?}
|
||||
: ${AWS_SECRET_ACCESS_KEY:?}
|
||||
mme2e_log "S3 report upload enabled."
|
||||
fi
|
||||
|
||||
# Double check that the "results/" subdirectory to collect report informations from exists
|
||||
cd "../${TEST}/"
|
||||
if [ ! -d "results/" ]; then
|
||||
|
||||
+5
-3
@@ -8,6 +8,8 @@ This directory contains the E2E testing code for the Mattermost web client.
|
||||
|
||||
Please refer to the [dedicated developer documentation](https://developers.mattermost.com/contribute/more-info/webapp/e2e-testing/) for instructions.
|
||||
|
||||
> **Playwright note:** the instructions below describe the Docker Compose flow Cypress uses (and that Playwright previously used too). Playwright's CI and recommended local setup have since moved to [Testcontainers](https://node.testcontainers.org/) — see `playwright/README.md`'s Server Setup section for details. This Compose flow remains the way to run Cypress, and to run a plain server instance (`TEST=none make`).
|
||||
|
||||
##### For pipeline debugging
|
||||
|
||||
The E2E testing pipeline's scripts depend on the following tools being installed on your system: `docker`, `docker-compose`, `make`, `git`, `jq`, `node`, and some common utilities (`coreutils`, `findutils`, `bash`, `awk`, `sed`, `grep`)
|
||||
@@ -22,7 +24,7 @@ Instructions, detailed:
|
||||
* `ENABLED_DOCKER_SERVICES`: a space-separated list of services to start alongside the server. Default to `postgres inbucket`, for smoke test purposes and for lightweight and faster start-up time. Depending on the test requirement being worked on, you may want to override as needed, as such:
|
||||
- Cypress full tests require all services to be running: `postgres inbucket minio openldap elasticsearch keycloak`.
|
||||
- Cypress smoke tests require only the following: `postgres inbucket`.
|
||||
- Playwright full tests require only the following: `postgres inbucket`.
|
||||
- Playwright no longer runs against this Compose flow in CI (see the note above) — this only matters if you're using it to spin up a plain server instance (`TEST=none make`) for Playwright's `external` mode.
|
||||
* The following variables, will be passed over to the server container: `MM_LICENSE` (no enterprise features will be available if this is unset; required when `SERVER=cloud`), and the exploded `MM_ENV` (a comma-separated list of env var specifications)
|
||||
* The following variables, which will be passed over to the cypress container: `BRANCH`, `BUILD_ID`, `CI_BASE_URL`, `BROWSER`, `AUTOMATION_DASHBOARD_URL` and `AUTOMATION_DASHBOARD_TOKEN`
|
||||
* The `SERVER_IMAGE` variable can also be set if you want to select a custom mattermost-server image. If not specified, the value of the `SERVER_IMAGE_DEFAULT` variable defined in file `.ci/.e2erc` is used.
|
||||
@@ -39,7 +41,7 @@ Instructions, detailed:
|
||||
3. `make`: start and prepare the server, then run the Cypress smoke tests
|
||||
* You can track the progress of the run in the `http://localhost:4000/cycles` dashboard if you launched it locally
|
||||
* For `SERVER=cloud` runs, you'll need to first create a cloud customer against the specified `CWS_URL` service by running `make cloud-init`. The user isn't automatically removed, and may be reused across multiple runs until you run `make cloud-teardown` to delete it.
|
||||
* If you want to run the Playwright tests instead of the Cypress ones, you can run `TEST=playwright make`
|
||||
* If you want to run the Playwright tests instead of the Cypress ones, you can run `TEST=playwright make` — though `playwright/README.md`'s Testcontainers option is now the recommended way to run Playwright locally/in CI
|
||||
* If you just want to run a local server instance, without any further testing, you can run `TEST=none make`
|
||||
* If you're using the automation dashboard, you have the option of sharding the E2E test run: you can launch the `make` command in parallel on different machines (NB: you must use the same `BUILD_ID` and `BRANCH` values that you used for `make generate-test-cycle`) to distribute running the test cases across them. When doing this, you should also set on each machine the `CI_BASE_URL` variable to a value that uniquely identifies the instance where `make` is running.
|
||||
* This script will also parse the local test results, and write a `e2e-tests/${TEST}/results/summary.json` file containing the following keys: `passed`, `failed` and `failed_expected` (the total number of testcases that were run is the sum of these three numbers)
|
||||
@@ -71,4 +73,4 @@ For Cypress:
|
||||
* Your system needs to be setup for Cypress usage, to be able to run this command. Refer to the [E2E testing developer documentation](https://developers.mattermost.com/contribute/more-info/webapp/e2e-testing/) for this.
|
||||
4. The `cypress/results/testPasses.json` file will count, for each of the testfiles, how many times it was run, and how many times each of the testcases contained in it passed. If the attempts and passes numbers do not match, that specific testcase may be flaky.
|
||||
|
||||
For Playwright: WIP
|
||||
For Playwright: not currently supported — Playwright tests are run and re-run individually via `npm run test -- <spec>`, see `playwright/README.md`.
|
||||
|
||||
Generated
+37
-2026
File diff suppressed because it is too large
Load Diff
@@ -1,8 +1,6 @@
|
||||
{
|
||||
"name": "cypress",
|
||||
"devDependencies": {
|
||||
"@aws-sdk/client-s3": "3.1030.0",
|
||||
"@aws-sdk/lib-storage": "3.1030.0",
|
||||
"@babel/eslint-parser": "7.28.6",
|
||||
"@babel/eslint-plugin": "7.27.1",
|
||||
"@cypress/request": "3.0.10",
|
||||
@@ -21,11 +19,9 @@
|
||||
"@types/lodash.without": "4.4.9",
|
||||
"@types/mime-types": "3.0.1",
|
||||
"@types/mochawesome": "6.2.5",
|
||||
"@types/recursive-readdir": "2.2.4",
|
||||
"@types/shelljs": "0.10.0",
|
||||
"@typescript-eslint/eslint-plugin": "8.58.2",
|
||||
"@typescript-eslint/parser": "8.58.2",
|
||||
"async": "3.2.6",
|
||||
"authenticator": "1.1.5",
|
||||
"axios": "1.15.0",
|
||||
"chai": "6.2.2",
|
||||
@@ -71,7 +67,6 @@
|
||||
"node-polyfill-webpack-plugin": "4.1.0",
|
||||
"pdf-parse": "2.4.5",
|
||||
"pg": "8.20.0",
|
||||
"recursive-readdir": "2.2.3",
|
||||
"shelljs": "0.10.0",
|
||||
"timezones.json": "1.7.2",
|
||||
"ts-loader": "9.5.7",
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
/* eslint-disable no-console */
|
||||
|
||||
/*
|
||||
* This is used for saving artifacts to AWS S3, sending data to automation dashboard and
|
||||
* This is used for sending data to automation dashboard and
|
||||
* publishing quick summary to community channels.
|
||||
*
|
||||
* Usage: [ENV] node save_report.js
|
||||
@@ -14,8 +14,6 @@
|
||||
* BUILD_ID=[build_id] : Build identifier from CI
|
||||
* BUILD_TAG=[build_tag] : Docker image used to run the test
|
||||
*
|
||||
* For saving artifacts to AWS S3
|
||||
* - AWS_S3_BUCKET, AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
|
||||
* For saving test cases to Test Management
|
||||
* - TM4J_ENABLE=true|false
|
||||
* - TM4J_API_KEY=[api_key]
|
||||
@@ -42,7 +40,6 @@ const {
|
||||
readJsonFromFile,
|
||||
writeJsonToFile,
|
||||
} = require('./utils/report');
|
||||
const {saveArtifacts} = require('./utils/artifacts');
|
||||
const {MOCHAWESOME_REPORT_DIR, RESULTS_DIR} = require('./utils/constants');
|
||||
const {createTestCycle, createTestExecutions} = require('./utils/test_cases');
|
||||
|
||||
@@ -82,11 +79,6 @@ const saveReport = async () => {
|
||||
console.log(summary);
|
||||
writeJsonToFile(summary, 'summary.json', MOCHAWESOME_REPORT_DIR);
|
||||
|
||||
const result = await saveArtifacts();
|
||||
if (result && result.success) {
|
||||
console.log('Successfully uploaded artifacts to S3:', result.reportLink);
|
||||
}
|
||||
|
||||
// Create or use an existing test cycle
|
||||
let testCycle = {};
|
||||
if (TM4J_ENABLE === 'true') {
|
||||
@@ -97,7 +89,7 @@ const saveReport = async () => {
|
||||
// Send test report to "QA: UI Test Automation" channel via webhook
|
||||
if (TYPE && TYPE !== 'NONE' && WEBHOOK_URL) {
|
||||
const environment = readJsonFromFile(`${RESULTS_DIR}/environment.json`);
|
||||
const data = generateTestReport(summary, result && result.success, result && result.reportLink, environment, testCycle.key);
|
||||
const data = generateTestReport(summary, false, undefined, environment, testCycle.key);
|
||||
await sendReport('summary report to Community channel', WEBHOOK_URL, data);
|
||||
}
|
||||
|
||||
|
||||
@@ -132,11 +132,7 @@ Cypress.Commands.add('uiClickSidebarItem', (name) => {
|
||||
cy.uiGetSidebarItem(name).click({force: true});
|
||||
|
||||
if (name === 'threads') {
|
||||
cy.get('body').then((body) => {
|
||||
if (body.find('#genericModalLabel').length > 0) {
|
||||
cy.uiCloseModal('A new way to view and follow threads');
|
||||
}
|
||||
});
|
||||
// CRT intro modal was removed in MM-66470; only wait for the threads view to settle.
|
||||
cy.get('#tutorial-threads-mobile-header span.Button_label').contains('Followed threads');
|
||||
} else {
|
||||
cy.findAllByTestId('postView').last().scrollIntoView().should('be.visible');
|
||||
|
||||
@@ -1,88 +0,0 @@
|
||||
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
||||
// See LICENSE.txt for license information.
|
||||
|
||||
/* eslint-disable no-console */
|
||||
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
|
||||
const async = require('async');
|
||||
const {S3} = require('@aws-sdk/client-s3');
|
||||
const {Upload} = require('@aws-sdk/lib-storage');
|
||||
const mime = require('mime-types');
|
||||
const readdir = require('recursive-readdir');
|
||||
|
||||
const {MOCHAWESOME_REPORT_DIR} = require('./constants');
|
||||
|
||||
require('dotenv').config();
|
||||
|
||||
const {
|
||||
AWS_S3_BUCKET,
|
||||
AWS_ACCESS_KEY_ID,
|
||||
AWS_SECRET_ACCESS_KEY,
|
||||
BUILD_ID,
|
||||
BRANCH,
|
||||
BUILD_TAG,
|
||||
} = process.env;
|
||||
|
||||
const s3 = new S3({
|
||||
credentials: {
|
||||
accessKeyId: AWS_ACCESS_KEY_ID,
|
||||
secretAccessKey: AWS_SECRET_ACCESS_KEY,
|
||||
},
|
||||
});
|
||||
|
||||
function getFiles(dirPath) {
|
||||
return fs.existsSync(dirPath) ? readdir(dirPath) : [];
|
||||
}
|
||||
|
||||
async function saveArtifacts() {
|
||||
if (!AWS_S3_BUCKET || !AWS_ACCESS_KEY_ID || !AWS_SECRET_ACCESS_KEY) {
|
||||
console.log('No AWS credentials found. Test artifacts not uploaded to S3.');
|
||||
|
||||
return;
|
||||
}
|
||||
|
||||
const s3Folder = `${BUILD_ID}-${BRANCH}-${BUILD_TAG}`.replace(/\./g, '-');
|
||||
const uploadPath = path.resolve(__dirname, `../${MOCHAWESOME_REPORT_DIR}`);
|
||||
const filesToUpload = await getFiles(uploadPath);
|
||||
|
||||
return new Promise((resolve, reject) => {
|
||||
async.eachOfLimit(
|
||||
filesToUpload,
|
||||
10,
|
||||
async.asyncify(async (file) => {
|
||||
const Key = file.replace(uploadPath, s3Folder);
|
||||
const contentType = mime.lookup(file);
|
||||
const charset = mime.charset(contentType);
|
||||
|
||||
try {
|
||||
await new Upload({
|
||||
client: s3,
|
||||
params: {
|
||||
Key,
|
||||
Bucket: AWS_S3_BUCKET,
|
||||
Body: fs.readFileSync(file),
|
||||
ContentType: `${contentType}${charset ? '; charset=' + charset : ''}`,
|
||||
},
|
||||
}).done();
|
||||
return {success: true};
|
||||
} catch (e) {
|
||||
console.log('Failed to upload artifact:', file);
|
||||
throw new Error(e);
|
||||
}
|
||||
}),
|
||||
(err) => {
|
||||
if (err) {
|
||||
console.log('Failed to upload artifacts');
|
||||
return reject(new Error(err));
|
||||
}
|
||||
|
||||
const reportLink = `https://${AWS_S3_BUCKET}.s3.amazonaws.com/${s3Folder}/mochawesome.html`;
|
||||
resolve({success: true, reportLink});
|
||||
},
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
module.exports = {saveArtifacts};
|
||||
@@ -14,3 +14,6 @@ test/.eslintcache
|
||||
# build
|
||||
dist
|
||||
*.tsbuildinfo
|
||||
|
||||
# testcontainers ("full" mode) — generated per-run, see docs/testcontainers/testcontainers_plan.md
|
||||
.env.testcontainers
|
||||
|
||||
@@ -91,6 +91,7 @@ npm run show-report
|
||||
- Component abstractions for UI elements
|
||||
- Test utilities and fixtures
|
||||
- Server setup and management functions
|
||||
- Testcontainers orchestration (`lib/src/containers/`) for `testcontainers` mode (see Server Setup below)
|
||||
- Visual testing support
|
||||
|
||||
2. **`specs/` Directory**: Contains the actual test files organized by type:
|
||||
@@ -135,6 +136,12 @@ Tests can be configured through environment variables:
|
||||
- `PW_SLOWMO` - Add delay between actions in ms (default: 0)
|
||||
- `PW_WORKERS` - Number of parallel workers (default: 1)
|
||||
- `PERCY_TOKEN` - Authentication token for Percy visual testing service (required for Percy tests)
|
||||
- `PW_USE_TESTCONTAINERS` - Selects `testcontainers` mode, see Server Setup below (default: false)
|
||||
- `PW_TESTCONTAINERS_SERVICES` - Comma-separated additional services to start, e.g. `minio,openldap` (default: none; core stack is Postgres, Inbucket, Mattermost)
|
||||
- `PW_TESTCONTAINERS_WEBHOOK` - Start the webhook sidecar (`true` to enable; default: off on this release)
|
||||
- `PW_TESTCONTAINERS_REUSE` - Reuse containers across repeated local runs instead of recreating them; tear down explicitly with `npm run testcontainers:down` (default: true)
|
||||
- `SERVER_IMAGE` - Mattermost server image `testcontainers` mode starts (default: `mattermostdevelopment/mattermost-enterprise-edition:master`)
|
||||
- `MM_ENV` - Comma-separated `KEY=VALUE` server config overrides for `testcontainers` mode (default: none)
|
||||
|
||||
## Server Setup
|
||||
|
||||
@@ -146,10 +153,12 @@ Before running tests, a Mattermost server must be available. Two options:
|
||||
cd server && make run
|
||||
```
|
||||
|
||||
2. **Run using Docker** (recommended for testing):
|
||||
2. **Testcontainers** (recommended for testing, and what CI uses) — no separate step needed, Playwright starts and tears down the server + dependencies itself:
|
||||
```bash
|
||||
# Configure environment in e2e-tests/.ci/env
|
||||
cd e2e-tests && TEST=playwright make
|
||||
PW_USE_TESTCONTAINERS=true npm run test
|
||||
# Starts Postgres, Inbucket, Mattermost only by default; set PW_TESTCONTAINERS_SERVICES=... for sidecars and PW_TESTCONTAINERS_WEBHOOK=true for webhook
|
||||
# Containers are reused across runs by default; tear down with `npm run testcontainers:down` (or set PW_TESTCONTAINERS_REUSE=false for a self-cleaning one-off run)
|
||||
# `npm run testcontainers:up` brings the stack up (or confirms an existing one) without running any tests
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
@@ -17,28 +17,28 @@ cd webapp && make run
|
||||
cd server && make run-server
|
||||
```
|
||||
|
||||
**Option 2: Run using Docker (recommended for testing)**
|
||||
**Option 2: Testcontainers (recommended for testing, and what CI uses)**
|
||||
|
||||
No separate terminal or setup step needed — Playwright brings up Postgres, Inbucket, and the Mattermost server itself via [Testcontainers](https://node.testcontainers.org/), then tears them down after the run.
|
||||
|
||||
```bash
|
||||
# 1. Configure environment variables in e2e-tests/.ci/env
|
||||
# Create this file if it doesn't exist
|
||||
# Run with defaults (Postgres, Inbucket, Mattermost server — no optional sidecars)
|
||||
PW_USE_TESTCONTAINERS=true npm run test -- login
|
||||
|
||||
# 2. Set the server image (optional)
|
||||
# To use the latest master image:
|
||||
SERVER_IMAGE="mattermostdevelopment/mattermost-enterprise-edition:master"
|
||||
# If not set, it will use the current commit: mattermostdevelopment/mattermost-enterprise-edition:$(git rev-parse --short=7 HEAD)
|
||||
# Note: The image must exist in Docker Hub at https://hub.docker.com/r/mattermostdevelopment/mattermost-enterprise-edition/tags
|
||||
# Opt in to additional services (comma-separated); webhook is separate
|
||||
PW_USE_TESTCONTAINERS=true PW_TESTCONTAINERS_SERVICES=minio,openldap npm run test
|
||||
PW_USE_TESTCONTAINERS=true PW_TESTCONTAINERS_WEBHOOK=true npm run test
|
||||
|
||||
# 3. Add your license if needed
|
||||
MM_LICENSE=<your-license-key>
|
||||
# Pin a specific server image (defaults to mattermostdevelopment/mattermost-enterprise-edition:master)
|
||||
PW_USE_TESTCONTAINERS=true SERVER_IMAGE=mattermostdevelopment/mattermost-enterprise-edition:<tag> npm run test
|
||||
|
||||
# 4. For additional configuration options, see e2e-tests/README.md
|
||||
|
||||
# 5. Run the server and Playwright's smoke tests from the e2e-tests directory
|
||||
cd e2e-tests && TEST=playwright make
|
||||
# Pass arbitrary MM_* config overrides as comma-separated KEY=VALUE pairs
|
||||
PW_USE_TESTCONTAINERS=true MM_ENV=MM_LICENSE=<your-license-key> npm run test
|
||||
```
|
||||
|
||||
This approach uses the server's Docker image to create a consistent testing environment. It automatically configures the server with the necessary settings for Playwright tests and handles dependencies.
|
||||
Containers are reused across invocations by default (`PW_TESTCONTAINERS_REUSE=true`) instead of being recreated every run — tear the stack down explicitly when you're done with `npm run testcontainers:down`. Set `PW_TESTCONTAINERS_REUSE=false` for a one-off run that tears itself down when it finishes. Use `npm run testcontainers:up` to just bring the stack up (or confirm an existing one's still reachable) without running any tests.
|
||||
|
||||
See `lib/README.md` for every available environment variable.
|
||||
|
||||
#### 2. Install dependencies and run the test.
|
||||
|
||||
|
||||
@@ -19,7 +19,15 @@ eslintPluginHeader.rules.header.meta.schema = false;
|
||||
|
||||
export default [
|
||||
{
|
||||
ignores: ['**/node_modules', '**/dist', '**/playwright-report', '**/test-results', '**/results'],
|
||||
ignores: [
|
||||
'**/node_modules',
|
||||
'**/dist',
|
||||
'**/playwright-report',
|
||||
'**/test-results',
|
||||
'**/results',
|
||||
'lib/src/containers/assets/webhook/tests/**',
|
||||
'lib/src/containers/assets/webhook/**',
|
||||
],
|
||||
},
|
||||
...compat
|
||||
.extends('eslint:recommended', 'plugin:@typescript-eslint/recommended', 'plugin:import/recommended')
|
||||
@@ -51,6 +59,7 @@ export default [
|
||||
'@typescript-eslint/explicit-module-boundary-types': 'off',
|
||||
'@typescript-eslint/no-explicit-any': 'off',
|
||||
'@typescript-eslint/no-var-requires': 'off',
|
||||
'@typescript-eslint/no-require-imports': 'off',
|
||||
'no-console': 'error',
|
||||
'header/header': [
|
||||
'error',
|
||||
|
||||
@@ -1,21 +1,31 @@
|
||||
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
||||
// See LICENSE.txt for license information.
|
||||
|
||||
import {baseGlobalSetup, testConfig} from '@mattermost/playwright-lib';
|
||||
import chalk from 'chalk';
|
||||
|
||||
import {baseGlobalSetup, startStack, stopStack, testConfig} from '@mattermost/playwright-lib';
|
||||
|
||||
async function globalSetup() {
|
||||
try {
|
||||
// With PW_USE_TESTCONTAINERS=true, bring up the server + dependencies via Testcontainers
|
||||
// before pinging it. No-op otherwise, when a server is expected to already be running.
|
||||
await startStack();
|
||||
await baseGlobalSetup();
|
||||
} catch (error: unknown) {
|
||||
// eslint-disable-next-line no-console
|
||||
console.error(error);
|
||||
throw new Error(
|
||||
`Global setup failed.\n\tEnsure the server at ${testConfig.baseURL} is running and accessible.\n\tPlease check the logs for more details.`,
|
||||
);
|
||||
console.error(chalk.cyan('[testcontainers]'), error);
|
||||
// Whatever startStack() managed to bring up (e.g. baseGlobalSetup() failed after the
|
||||
// stack itself came up fine) shouldn't linger just because we're about to throw.
|
||||
await stopStack();
|
||||
const message = error instanceof Error ? error.message : String(error);
|
||||
const hint = testConfig.useTestContainers
|
||||
? 'Check the container named above and its logs under logs/.'
|
||||
: `Ensure the server at ${testConfig.baseURL} is running and accessible.`;
|
||||
throw new Error(chalk.red(`[testcontainers] Global setup failed: ${message}\n${hint}`));
|
||||
}
|
||||
|
||||
return function () {
|
||||
// placeholder for teardown setup
|
||||
return async function () {
|
||||
await stopStack();
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
@@ -138,6 +138,28 @@ All environment variables are optional with sensible defaults.
|
||||
| -------- | ------------------------------------ | ------- |
|
||||
| `CI` | Set automatically in CI environments | N/A |
|
||||
|
||||
#### Testcontainers
|
||||
|
||||
Selects `testcontainers` mode — Playwright brings up the server + dependencies itself via [Testcontainers](https://node.testcontainers.org/) — instead of the `external` mode default, which expects a server already running at `PW_BASE_URL`.
|
||||
|
||||
| Variable | Description | Default |
|
||||
| ------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------ |
|
||||
| `PW_USE_TESTCONTAINERS` | Selects `testcontainers` mode | `false` |
|
||||
| `PW_TESTCONTAINERS_SERVICES` | Comma-separated additional services to start (`openldap`, `keycloak`, `elasticsearch`, `opensearch`, `minio`, `azurite`); unset/empty starts none (core: Postgres, Inbucket, Mattermost) | none (empty) |
|
||||
| `PW_TESTCONTAINERS_WEBHOOK` | Start the webhook sidecar (`true` to enable) | `false` |
|
||||
| `PW_TESTCONTAINERS_REUSE` | Reuse containers across repeated local runs instead of recreating them; tear down with `npm run testcontainers:down` (requires `testcontainers.reuse.enable=true` in `~/.testcontainers.properties`) | `true` |
|
||||
| `PW_TESTCONTAINERS_CONTAINER_RUNNER` | Set when the Playwright process itself runs inside a Docker container (e.g. CI), to join it to the Testcontainers network | `false` |
|
||||
| `SERVER_IMAGE` | Prebuilt Mattermost server image `testcontainers` mode starts | `mattermostdevelopment/mattermost-enterprise-edition:master` |
|
||||
| `MM_ENV` | Comma-separated `KEY=VALUE` server config overrides, merged over the test baseline | none |
|
||||
| `PW_LDAP_HOST` / `PW_LDAP_PORT` | OpenLDAP host/port (only used when `openldap` is started) | `localhost` / `389` |
|
||||
| `PW_KEYCLOAK_URL` | Keycloak URL (only used when `keycloak` is started) | `http://localhost:8484` |
|
||||
| `PW_ELASTICSEARCH_URL` | Elasticsearch URL (only used when `elasticsearch` is started) | `http://localhost:9200` |
|
||||
| `PW_OPENSEARCH_URL` | OpenSearch URL (only used when `opensearch` is started) | `http://localhost:9201` |
|
||||
| `PW_MINIO_URL` | Minio URL (only used when `minio` is started) | `http://localhost:9000` |
|
||||
| `PW_AZURITE_URL` | Azurite URL (only used when `azurite` is started) | `http://localhost:10000` |
|
||||
|
||||
In `testcontainers` mode, these host/port defaults are never actually used — Testcontainers always assigns its own dynamic port or network alias per run, so a `testcontainers`-mode run can never collide with a same-machine `external`-mode session using the fixed defaults above. Tear a reused stack down explicitly with `npm run testcontainers:down` (from the `playwright/` package).
|
||||
|
||||
## Accessibility Testing
|
||||
|
||||
The library includes built-in accessibility testing using [axe-core](https://github.com/dequelabs/axe-core):
|
||||
|
||||
@@ -44,17 +44,22 @@
|
||||
},
|
||||
"dependencies": {
|
||||
"@axe-core/playwright": "4.11.1",
|
||||
"@azure/storage-blob": "12.33.0",
|
||||
"@mattermost/client": "file:../../../webapp/platform/client",
|
||||
"@mattermost/types": "file:../../../webapp/platform/types",
|
||||
"@percy/cli": "1.31.11",
|
||||
"@percy/playwright": "1.1.0",
|
||||
"@testcontainers/postgresql": "12.0.4",
|
||||
"async-wait-until": "2.0.31",
|
||||
"axe-core": "4.11.2",
|
||||
"chalk": "5.6.2",
|
||||
"deepmerge": "4.3.1",
|
||||
"dotenv": "17.4.2",
|
||||
"ldapts": "9.0.0",
|
||||
"luxon": "3.7.2",
|
||||
"mime-types": "3.0.2",
|
||||
"minio": "8.0.7",
|
||||
"testcontainers": "12.0.4",
|
||||
"uuid": "13.0.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
|
||||
@@ -18,7 +18,10 @@ export default {
|
||||
plugins: [
|
||||
typescript(),
|
||||
copy({
|
||||
targets: [{src: 'src/asset/**/*', dest: 'dist/asset'}], // Copy assets to dist/
|
||||
targets: [
|
||||
{src: 'src/asset/**/*', dest: 'dist/asset'}, // Copy assets to dist/
|
||||
{src: 'src/containers/assets/**/*', dest: 'dist/containers/assets'},
|
||||
],
|
||||
}),
|
||||
],
|
||||
external: [
|
||||
@@ -26,14 +29,22 @@ export default {
|
||||
'@mattermost/client',
|
||||
'@mattermost/types/config',
|
||||
'@axe-core/playwright',
|
||||
'@azure/storage-blob',
|
||||
'@percy/playwright',
|
||||
'@testcontainers/postgresql',
|
||||
'dotenv',
|
||||
'ldapts',
|
||||
'luxon',
|
||||
'node:fs/promises',
|
||||
'minio',
|
||||
'node:child_process',
|
||||
'node:path',
|
||||
'node:fs',
|
||||
'node:fs/promises',
|
||||
'node:os',
|
||||
'node:url',
|
||||
'node:util',
|
||||
'mime-types',
|
||||
'testcontainers',
|
||||
'uuid',
|
||||
'async-wait-until',
|
||||
'chalk',
|
||||
|
||||
@@ -5,13 +5,25 @@ import {writeFile} from 'node:fs/promises';
|
||||
import path from 'node:path';
|
||||
import fs from 'node:fs';
|
||||
|
||||
import {Browser, BrowserContext, request} from '@playwright/test';
|
||||
import {UserProfile} from '@mattermost/types/users';
|
||||
import type {Browser, BrowserContext, Page} from '@playwright/test';
|
||||
import {request} from '@playwright/test';
|
||||
import type {UserProfile} from '@mattermost/types/users';
|
||||
|
||||
import {testConfig} from './test_config';
|
||||
import {resolveAppUrl, testConfig} from './test_config';
|
||||
import {pages} from './ui/pages';
|
||||
import {resolvePlaywrightPath} from './util';
|
||||
|
||||
/** Keep page.goto pointed at testConfig.baseURL after testcontainers remaps the host port. */
|
||||
export function bindPageToLiveBaseURL(page: Page): void {
|
||||
const originalGoto = page.goto.bind(page);
|
||||
page.goto = ((url, options) => {
|
||||
if (typeof url === 'string') {
|
||||
return originalGoto(resolveAppUrl(url), options);
|
||||
}
|
||||
return originalGoto(url, options);
|
||||
}) as typeof page.goto;
|
||||
}
|
||||
|
||||
export class TestBrowser {
|
||||
readonly browser: Browser;
|
||||
private contexts: BrowserContext[] = [];
|
||||
@@ -21,7 +33,11 @@ export class TestBrowser {
|
||||
}
|
||||
|
||||
async login(user: UserProfile) {
|
||||
const options = {storageState: ''};
|
||||
const options: {storageState: string; baseURL: string} = {
|
||||
storageState: '',
|
||||
// Capture the current mapped URL at context creation (updated after server restarts).
|
||||
baseURL: testConfig.baseURL,
|
||||
};
|
||||
if (user) {
|
||||
// Log in via API request and save user storage
|
||||
const storagePath = await loginByAPI(user.username, user.password);
|
||||
@@ -31,6 +47,7 @@ export class TestBrowser {
|
||||
// Sign in a user in new browser context
|
||||
const context = await this.browser.newContext(options);
|
||||
const page = await context.newPage();
|
||||
bindPageToLiveBaseURL(page);
|
||||
|
||||
const channelsPage = new pages.ChannelsPage(page);
|
||||
const systemConsolePage = new pages.SystemConsolePage(page);
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
# Copied from server/build/Dockerfile.elasticsearch on 2026-07-19.
|
||||
ARG ELASTICSEARCH_VERSION=9.0.0
|
||||
FROM docker.elastic.co/elasticsearch/elasticsearch:${ELASTICSEARCH_VERSION}
|
||||
|
||||
RUN /usr/share/elasticsearch/bin/elasticsearch-plugin install --batch analysis-icu analysis-nori analysis-kuromoji analysis-smartcn
|
||||
@@ -0,0 +1,5 @@
|
||||
# Copied from server/build/Dockerfile.opensearch on 2026-07-19.
|
||||
ARG OPENSEARCH_VERSION=3.0.0
|
||||
FROM opensearchproject/opensearch:$OPENSEARCH_VERSION
|
||||
|
||||
RUN /usr/share/opensearch/bin/opensearch-plugin install analysis-icu analysis-nori analysis-kuromoji analysis-smartcn
|
||||
@@ -0,0 +1,13 @@
|
||||
# Vendored assets
|
||||
|
||||
These files are copied from `server/build/` in the Mattermost monorepo so that
|
||||
`@mattermost/playwright-lib`'s Testcontainers support is self-contained once published to npm
|
||||
(it can't reach outside its own package at runtime). If the source files change, these copies
|
||||
need to be refreshed manually — they are not symlinked or build-generated.
|
||||
|
||||
| File | Copied from | Date |
|
||||
| ---------------------------- | ------------------------------------------------ | ---------- |
|
||||
| `postgres.conf` | `server/build/docker/postgres.conf` | 2026-07-19 |
|
||||
| `keycloak-realm-export.json` | `server/build/docker/keycloak/realm-export.json` | 2026-07-19 |
|
||||
| `Dockerfile.elasticsearch` | `server/build/Dockerfile.elasticsearch` | 2026-07-19 |
|
||||
| `Dockerfile.opensearch` | `server/build/Dockerfile.opensearch` | 2026-07-19 |
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,9 @@
|
||||
# Copied from server/build/docker/postgres.conf on 2026-07-19.
|
||||
max_connections = 500
|
||||
listen_addresses = '*'
|
||||
fsync = off
|
||||
full_page_writes = off
|
||||
default_text_search_config = 'pg_catalog.english'
|
||||
commit_delay=1000
|
||||
logging_collector=off
|
||||
password_encryption = 'scram-sha-256'
|
||||
@@ -0,0 +1,20 @@
|
||||
# webhook_serve.js, utils/webhook_utils.js, and tests/plugins/post_message_as.js are vendored
|
||||
# copies of e2e-tests/cypress's files of the same name (shared sidecar, also used by Cypress) —
|
||||
# copied on 2026-07-20. Dependencies match e2e-tests/.ci/server.generate.sh's "playwright" Compose
|
||||
# service, just installed at build time instead of on every container start.
|
||||
FROM node:24-alpine
|
||||
|
||||
WORKDIR /webhook
|
||||
|
||||
RUN npm init -y >/dev/null && \
|
||||
npm install express@5.1.0 axios@1.11.0 client-oauth2@github:larkox/js-client-oauth2#e24e2eb5dfcbbbb3a59d095e831dbe0012b0ac49
|
||||
|
||||
COPY webhook_serve.js ./webhook_serve.js
|
||||
COPY utils/ ./utils/
|
||||
COPY tests/plugins/post_message_as.js ./tests/plugins/post_message_as.js
|
||||
|
||||
RUN chown -R node:node /webhook
|
||||
USER node
|
||||
|
||||
EXPOSE 3000
|
||||
CMD ["node", "webhook_serve.js"]
|
||||
+44
@@ -0,0 +1,44 @@
|
||||
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
||||
// See LICENSE.txt for license information.
|
||||
|
||||
const axios = require('axios');
|
||||
|
||||
module.exports = async ({sender, message, channelId, rootId, createAt = 0, baseUrl}) => {
|
||||
const loginResponse = await axios({
|
||||
url: `${baseUrl}/api/v4/users/login`,
|
||||
headers: {'X-Requested-With': 'XMLHttpRequest'},
|
||||
method: 'post',
|
||||
data: {login_id: sender.username, password: sender.password},
|
||||
});
|
||||
|
||||
const setCookie = loginResponse.headers['set-cookie'];
|
||||
let cookieString = '';
|
||||
setCookie.forEach((cookie) => {
|
||||
const nameAndValue = cookie.split(';')[0];
|
||||
cookieString += nameAndValue + ';';
|
||||
});
|
||||
|
||||
let response;
|
||||
try {
|
||||
response = await axios({
|
||||
url: `${baseUrl}/api/v4/posts`,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'X-Requested-With': 'XMLHttpRequest',
|
||||
Cookie: cookieString,
|
||||
},
|
||||
method: 'post',
|
||||
data: {
|
||||
channel_id: channelId,
|
||||
message,
|
||||
type: '',
|
||||
create_at: createAt,
|
||||
root_id: rootId,
|
||||
},
|
||||
});
|
||||
} catch (err) {
|
||||
expect(Boolean(err)).to.equal(false);
|
||||
}
|
||||
|
||||
return {status: response.status, data: response.data};
|
||||
};
|
||||
@@ -0,0 +1,837 @@
|
||||
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
||||
// See LICENSE.txt for license information.
|
||||
|
||||
// Helper function to create dialog base structure
|
||||
function createDialog(triggerId, webhookBaseUrl, dialogConfig) {
|
||||
const baseDialog = {
|
||||
trigger_id: triggerId,
|
||||
url: `${webhookBaseUrl}/dialog_submit`,
|
||||
dialog: {
|
||||
callback_id: dialogConfig.callback_id,
|
||||
title: dialogConfig.title,
|
||||
submit_label: dialogConfig.submit_label || 'Submit',
|
||||
notify_on_cancel: true,
|
||||
...dialogConfig.dialog_props,
|
||||
elements: dialogConfig.elements || [],
|
||||
},
|
||||
};
|
||||
|
||||
if (dialogConfig.icon_url) {
|
||||
baseDialog.dialog.icon_url = dialogConfig.icon_url;
|
||||
}
|
||||
|
||||
if (dialogConfig.introduction_text) {
|
||||
baseDialog.dialog.introduction_text = dialogConfig.introduction_text;
|
||||
}
|
||||
|
||||
if (dialogConfig.state) {
|
||||
baseDialog.dialog.state = dialogConfig.state;
|
||||
}
|
||||
|
||||
if (dialogConfig.source_url) {
|
||||
baseDialog.dialog.source_url = dialogConfig.source_url;
|
||||
}
|
||||
|
||||
return baseDialog;
|
||||
}
|
||||
|
||||
// Helper function to create form response structure
|
||||
function createFormResponse(formConfig) {
|
||||
return {
|
||||
callback_id: formConfig.callback_id,
|
||||
title: formConfig.title,
|
||||
submit_label: formConfig.submit_label || 'Submit',
|
||||
notify_on_cancel: true,
|
||||
elements: formConfig.elements || [],
|
||||
...formConfig.form_props,
|
||||
};
|
||||
}
|
||||
|
||||
// Helper function to create common form elements
|
||||
function createElement(type, config) {
|
||||
const baseElement = {
|
||||
display_name: config.display_name,
|
||||
name: config.name,
|
||||
type,
|
||||
optional: config.optional || false,
|
||||
};
|
||||
|
||||
if (config.placeholder) {
|
||||
baseElement.placeholder = config.placeholder;
|
||||
}
|
||||
if (config.help_text) {
|
||||
baseElement.help_text = config.help_text;
|
||||
}
|
||||
if (config.default) {
|
||||
baseElement.default = config.default;
|
||||
}
|
||||
if (config.subtype) {
|
||||
baseElement.subtype = config.subtype;
|
||||
}
|
||||
if (config.min_length) {
|
||||
baseElement.min_length = config.min_length;
|
||||
}
|
||||
if (config.max_length) {
|
||||
baseElement.max_length = config.max_length;
|
||||
}
|
||||
if (config.data_source) {
|
||||
baseElement.data_source = config.data_source;
|
||||
}
|
||||
if (config.options) {
|
||||
baseElement.options = config.options;
|
||||
}
|
||||
if (config.refresh) {
|
||||
baseElement.refresh = config.refresh;
|
||||
}
|
||||
if (config.action_button) {
|
||||
baseElement.action_button = config.action_button;
|
||||
}
|
||||
|
||||
return baseElement;
|
||||
}
|
||||
|
||||
// Standard icon URL
|
||||
const STANDARD_ICON = 'https://mattermost.com/wp-content/uploads/2022/02/icon_WS.png';
|
||||
|
||||
// Dialog configurations
|
||||
const DIALOG_CONFIGS = {
|
||||
full: {
|
||||
callback_id: 'somecallbackid',
|
||||
title: 'Title for Full Dialog Test',
|
||||
icon_url: STANDARD_ICON,
|
||||
elements: [
|
||||
createElement('text', {
|
||||
display_name: 'Display Name',
|
||||
name: 'realname',
|
||||
default: 'default text',
|
||||
placeholder: 'placeholder',
|
||||
help_text: 'This a regular input in an interactive dialog triggered by a test integration.',
|
||||
}),
|
||||
createElement('text', {
|
||||
display_name: 'Email',
|
||||
name: 'someemail',
|
||||
subtype: 'email',
|
||||
placeholder: 'placeholder@bladekick.com',
|
||||
help_text: 'This a regular email input in an interactive dialog triggered by a test integration.',
|
||||
}),
|
||||
createElement('text', {display_name: 'Number', name: 'somenumber', subtype: 'number'}),
|
||||
createElement('text', {
|
||||
display_name: 'Password',
|
||||
name: 'somepassword',
|
||||
subtype: 'password',
|
||||
default: 'p@ssW0rd',
|
||||
placeholder: 'placeholder',
|
||||
help_text: 'This a password input in an interactive dialog triggered by a test integration.',
|
||||
optional: true,
|
||||
}),
|
||||
createElement('textarea', {
|
||||
display_name: 'Display Name Long Text Area',
|
||||
name: 'realnametextarea',
|
||||
placeholder: 'placeholder',
|
||||
optional: true,
|
||||
min_length: 5,
|
||||
max_length: 100,
|
||||
}),
|
||||
createElement('select', {
|
||||
display_name: 'User Selector',
|
||||
name: 'someuserselector',
|
||||
placeholder: 'Select a user...',
|
||||
data_source: 'users',
|
||||
}),
|
||||
createElement('select', {
|
||||
display_name: 'Channel Selector',
|
||||
name: 'somechannelselector',
|
||||
placeholder: 'Select a channel...',
|
||||
help_text: 'Choose a channel from the list.',
|
||||
data_source: 'channels',
|
||||
optional: true,
|
||||
}),
|
||||
createElement('select', {
|
||||
display_name: 'Option Selector',
|
||||
name: 'someoptionselector',
|
||||
placeholder: 'Select an option...',
|
||||
options: [
|
||||
{text: 'Option1', value: 'opt1'},
|
||||
{text: 'Option2', value: 'opt2'},
|
||||
{text: 'Option3', value: 'opt3'},
|
||||
],
|
||||
}),
|
||||
createElement('radio', {
|
||||
display_name: 'Radio Option Selector',
|
||||
name: 'someradiooptions',
|
||||
help_text: '',
|
||||
options: [
|
||||
{text: 'Engineering', value: 'engineering'},
|
||||
{text: 'Sales', value: 'sales'},
|
||||
],
|
||||
}),
|
||||
createElement('bool', {
|
||||
display_name: 'Boolean Selector',
|
||||
name: 'boolean_input',
|
||||
placeholder: 'Was this modal helpful?',
|
||||
default: 'True',
|
||||
optional: true,
|
||||
help_text: 'This is the help text',
|
||||
}),
|
||||
],
|
||||
dialog_props: {state: 'somestate'},
|
||||
},
|
||||
|
||||
simple: {
|
||||
callback_id: 'somecallbackid',
|
||||
title: 'Title for Dialog Test without elements',
|
||||
icon_url: STANDARD_ICON,
|
||||
submit_label: 'Submit Test',
|
||||
dialog_props: {state: 'somestate'},
|
||||
},
|
||||
|
||||
userAndChannel: {
|
||||
callback_id: 'somecallbackid',
|
||||
title: 'Title for Dialog Test with user and channel element',
|
||||
icon_url: STANDARD_ICON,
|
||||
submit_label: 'Submit Test',
|
||||
elements: [
|
||||
createElement('select', {
|
||||
display_name: 'User Selector',
|
||||
name: 'someuserselector',
|
||||
placeholder: 'Select a user...',
|
||||
data_source: 'users',
|
||||
}),
|
||||
createElement('select', {
|
||||
display_name: 'Channel Selector',
|
||||
name: 'somechannelselector',
|
||||
placeholder: 'Select a channel...',
|
||||
help_text: 'Choose a channel from the list.',
|
||||
data_source: 'channels',
|
||||
optional: true,
|
||||
}),
|
||||
],
|
||||
dialog_props: {state: 'somestate'},
|
||||
},
|
||||
|
||||
boolean: {
|
||||
callback_id: 'somecallbackid',
|
||||
title: 'Title for Dialog Test with boolean element',
|
||||
icon_url: STANDARD_ICON,
|
||||
submit_label: 'Submit Test',
|
||||
elements: [
|
||||
createElement('bool', {
|
||||
display_name: 'Boolean Selector',
|
||||
name: 'boolean_input',
|
||||
placeholder: 'Was this modal helpful?',
|
||||
default: 'True',
|
||||
optional: true,
|
||||
help_text: 'This is the help text',
|
||||
}),
|
||||
],
|
||||
dialog_props: {state: 'somestate'},
|
||||
},
|
||||
|
||||
fieldRefresh: {
|
||||
callback_id: 'field_refresh_callback',
|
||||
title: 'Field Refresh Demo',
|
||||
introduction_text: 'Enter project name then select type to see different fields',
|
||||
elements: [
|
||||
createElement('text', {
|
||||
display_name: 'Project Name',
|
||||
name: 'project_name',
|
||||
placeholder: 'Enter project name',
|
||||
}),
|
||||
createElement('select', {
|
||||
display_name: 'Project Type',
|
||||
name: 'project_type',
|
||||
refresh: true,
|
||||
placeholder: 'Select project type...',
|
||||
options: [
|
||||
{text: 'Web Application', value: 'web'},
|
||||
{text: 'Mobile App', value: 'mobile'},
|
||||
{text: 'API Service', value: 'api'},
|
||||
],
|
||||
}),
|
||||
],
|
||||
},
|
||||
|
||||
multistepStep1: {
|
||||
callback_id: 'multistep_callback',
|
||||
title: 'Step 1 - Personal Info',
|
||||
introduction_text: 'Multi-step registration - Step 1 of 3',
|
||||
submit_label: 'Next Step',
|
||||
elements: [
|
||||
createElement('text', {
|
||||
display_name: 'First Name',
|
||||
name: 'first_name',
|
||||
placeholder: 'Enter your first name',
|
||||
}),
|
||||
createElement('text', {
|
||||
display_name: 'Email',
|
||||
name: 'email',
|
||||
subtype: 'email',
|
||||
placeholder: 'Enter your email address',
|
||||
}),
|
||||
],
|
||||
dialog_props: {state: 'step1'},
|
||||
},
|
||||
|
||||
multistepStep2: {
|
||||
callback_id: 'multistep_callback',
|
||||
title: 'Step 2 - Work Info',
|
||||
introduction_text: 'Multi-step registration - Step 2 of 3',
|
||||
submit_label: 'Next Step',
|
||||
elements: [
|
||||
createElement('select', {
|
||||
display_name: 'Department',
|
||||
name: 'department',
|
||||
placeholder: 'Select department...',
|
||||
options: [
|
||||
{text: 'Engineering', value: 'engineering'},
|
||||
{text: 'Marketing', value: 'marketing'},
|
||||
{text: 'Sales', value: 'sales'},
|
||||
],
|
||||
}),
|
||||
createElement('radio', {
|
||||
display_name: 'Experience Level',
|
||||
name: 'experience_level',
|
||||
options: [
|
||||
{text: 'Junior', value: 'junior'},
|
||||
{text: 'Mid-level', value: 'mid'},
|
||||
{text: 'Senior', value: 'senior'},
|
||||
],
|
||||
}),
|
||||
],
|
||||
form_props: {state: 'step2'},
|
||||
},
|
||||
|
||||
multistepStep3: {
|
||||
callback_id: 'multistep_callback',
|
||||
title: 'Step 3 - Final Details',
|
||||
introduction_text: 'Multi-step registration - Step 3 of 3',
|
||||
submit_label: 'Complete Registration',
|
||||
elements: [
|
||||
createElement('textarea', {
|
||||
display_name: 'Comments',
|
||||
name: 'comments',
|
||||
placeholder: 'Any additional comments...',
|
||||
optional: true,
|
||||
}),
|
||||
createElement('bool', {display_name: 'Terms & Conditions', name: 'terms_accepted'}),
|
||||
],
|
||||
form_props: {state: 'step3'},
|
||||
},
|
||||
|
||||
actionButtonParent: {
|
||||
callback_id: 'action_button_parent_callback',
|
||||
title: 'Parent Dialog with Action Button',
|
||||
elements: [],
|
||||
},
|
||||
|
||||
actionButtonChild: {
|
||||
callback_id: 'child_callback',
|
||||
title: 'Child Dialog',
|
||||
elements: [
|
||||
createElement('text', {
|
||||
display_name: 'Child Input',
|
||||
name: 'child_input',
|
||||
placeholder: 'Enter value',
|
||||
optional: true,
|
||||
}),
|
||||
],
|
||||
},
|
||||
};
|
||||
|
||||
// Public API functions
|
||||
function getFullDialog(triggerId, webhookBaseUrl) {
|
||||
return createDialog(triggerId, webhookBaseUrl, DIALOG_CONFIGS.full);
|
||||
}
|
||||
|
||||
function getSimpleDialog(triggerId, webhookBaseUrl) {
|
||||
return createDialog(triggerId, webhookBaseUrl, DIALOG_CONFIGS.simple);
|
||||
}
|
||||
|
||||
function getUserAndChannelDialog(triggerId, webhookBaseUrl) {
|
||||
return createDialog(triggerId, webhookBaseUrl, DIALOG_CONFIGS.userAndChannel);
|
||||
}
|
||||
|
||||
function getBooleanDialog(triggerId, webhookBaseUrl) {
|
||||
return createDialog(triggerId, webhookBaseUrl, DIALOG_CONFIGS.boolean);
|
||||
}
|
||||
|
||||
function getFieldRefreshDialog(triggerId, webhookBaseUrl) {
|
||||
const config = {...DIALOG_CONFIGS.fieldRefresh};
|
||||
config.source_url = `${webhookBaseUrl}/field_refresh_source`;
|
||||
return createDialog(triggerId, webhookBaseUrl, config);
|
||||
}
|
||||
|
||||
function getMultistepStep1Dialog(triggerId, webhookBaseUrl) {
|
||||
return createDialog(triggerId, webhookBaseUrl, DIALOG_CONFIGS.multistepStep1);
|
||||
}
|
||||
|
||||
function getMultistepStep2Dialog(triggerId, webhookBaseUrl) {
|
||||
const config = {...DIALOG_CONFIGS.multistepStep2};
|
||||
config.dialog_props = {url: `${webhookBaseUrl}/dialog_submit`, ...config.form_props};
|
||||
return createFormResponse(config);
|
||||
}
|
||||
|
||||
function getMultistepStep3Dialog(triggerId, webhookBaseUrl) {
|
||||
const config = {...DIALOG_CONFIGS.multistepStep3};
|
||||
config.dialog_props = {url: `${webhookBaseUrl}/dialog_submit`, ...config.form_props};
|
||||
return createFormResponse(config);
|
||||
}
|
||||
|
||||
function getMultiSelectDialog(triggerId, webhookBaseUrl, includeDefaults = false) {
|
||||
return {
|
||||
trigger_id: triggerId,
|
||||
url: `${webhookBaseUrl}/dialog_submit`,
|
||||
dialog: {
|
||||
callback_id: 'somecallbackid',
|
||||
title: 'Title for Dialog Test with multiselect elements',
|
||||
icon_url: 'https://mattermost.com/wp-content/uploads/2022/02/icon_WS.png',
|
||||
submit_label: 'Submit Multiselect Test',
|
||||
notify_on_cancel: true,
|
||||
state: 'somestate',
|
||||
elements: [
|
||||
{
|
||||
display_name: 'Multi Option Selector',
|
||||
name: 'multiselect_options',
|
||||
type: 'select',
|
||||
multiselect: true,
|
||||
default: includeDefaults ? 'opt1,opt3' : '',
|
||||
placeholder: 'Select multiple options...',
|
||||
help_text: 'You can select multiple options from this list.',
|
||||
optional: false,
|
||||
min_length: 0,
|
||||
max_length: 0,
|
||||
data_source: '',
|
||||
options: [
|
||||
{
|
||||
text: 'Engineering',
|
||||
value: 'opt1',
|
||||
},
|
||||
{
|
||||
text: 'Sales',
|
||||
value: 'opt2',
|
||||
},
|
||||
{
|
||||
text: 'Marketing',
|
||||
value: 'opt3',
|
||||
},
|
||||
{
|
||||
text: 'Support',
|
||||
value: 'opt4',
|
||||
},
|
||||
{
|
||||
text: 'Product',
|
||||
value: 'opt5',
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
display_name: 'Multi User Selector',
|
||||
name: 'multiselect_users',
|
||||
type: 'select',
|
||||
multiselect: true,
|
||||
default: '',
|
||||
placeholder: 'Select multiple users...',
|
||||
help_text: 'Choose multiple users from the team.',
|
||||
optional: false,
|
||||
min_length: 0,
|
||||
max_length: 0,
|
||||
data_source: 'users',
|
||||
options: null,
|
||||
},
|
||||
{
|
||||
display_name: 'Single Option Selector',
|
||||
name: 'single_select_options',
|
||||
type: 'select',
|
||||
multiselect: false,
|
||||
default: includeDefaults ? 'single2' : '',
|
||||
placeholder: 'Select one option...',
|
||||
help_text: 'This is a regular single-select for comparison.',
|
||||
optional: false,
|
||||
min_length: 0,
|
||||
max_length: 0,
|
||||
data_source: '',
|
||||
options: [
|
||||
{
|
||||
text: 'Single Option 1',
|
||||
value: 'single1',
|
||||
},
|
||||
{
|
||||
text: 'Single Option 2',
|
||||
value: 'single2',
|
||||
},
|
||||
{
|
||||
text: 'Single Option 3',
|
||||
value: 'single3',
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
function getDynamicSelectDialog(triggerId, webhookBaseUrl) {
|
||||
return {
|
||||
trigger_id: triggerId,
|
||||
url: `${webhookBaseUrl}/dialog_submit`,
|
||||
dialog: {
|
||||
callback_id: 'somecallbackid',
|
||||
title: 'Title for Dialog Test with dynamic select element',
|
||||
icon_url: 'https://mattermost.com/wp-content/uploads/2022/02/icon_WS.png',
|
||||
submit_label: 'Submit Dynamic Select Test',
|
||||
notify_on_cancel: true,
|
||||
state: 'somestate',
|
||||
elements: [
|
||||
{
|
||||
display_name: 'Dynamic Role Selector',
|
||||
name: 'dynamic_role_selector',
|
||||
type: 'select',
|
||||
data_source: 'dynamic',
|
||||
data_source_url: `${webhookBaseUrl}/dynamic_select_source`,
|
||||
default: '',
|
||||
placeholder: 'Search for a role...',
|
||||
help_text: 'Start typing to search for available roles. Options are loaded dynamically.',
|
||||
optional: false,
|
||||
min_length: 0,
|
||||
max_length: 0,
|
||||
},
|
||||
{
|
||||
display_name: 'Optional Dynamic Selector',
|
||||
name: 'optional_dynamic_selector',
|
||||
type: 'select',
|
||||
data_source: 'dynamic',
|
||||
data_source_url: `${webhookBaseUrl}/dynamic_select_source`,
|
||||
default: 'backend_eng',
|
||||
placeholder: 'Search for another role...',
|
||||
help_text: 'This field is optional and has a default value.',
|
||||
optional: true,
|
||||
min_length: 0,
|
||||
max_length: 0,
|
||||
},
|
||||
],
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
// Basic date field test - MM-T2530A
|
||||
function getBasicDateDialog(triggerId, webhookBaseUrl) {
|
||||
return {
|
||||
trigger_id: triggerId,
|
||||
url: `${webhookBaseUrl}/datetime_dialog_submit`,
|
||||
dialog: {
|
||||
callback_id: 'basic_date_callback',
|
||||
title: 'DateTime Fields Test',
|
||||
icon_url: 'https://mattermost.com/wp-content/uploads/2022/02/icon_WS.png',
|
||||
elements: [
|
||||
{
|
||||
display_name: 'Event Date',
|
||||
name: 'event_date',
|
||||
type: 'date',
|
||||
default: '',
|
||||
placeholder: 'Select a date',
|
||||
help_text: 'Select the date for your event',
|
||||
optional: false,
|
||||
},
|
||||
],
|
||||
submit_label: 'Submit',
|
||||
notify_on_cancel: true,
|
||||
state: 'datetime_state',
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
// Basic datetime field test - MM-T2530B
|
||||
function getBasicDateTimeDialog(triggerId, webhookBaseUrl) {
|
||||
return {
|
||||
trigger_id: triggerId,
|
||||
url: `${webhookBaseUrl}/datetime_dialog_submit`,
|
||||
dialog: {
|
||||
callback_id: 'basic_datetime_callback',
|
||||
title: 'DateTime Fields Test',
|
||||
icon_url: 'https://mattermost.com/wp-content/uploads/2022/02/icon_WS.png',
|
||||
elements: [
|
||||
{
|
||||
display_name: 'Event Date',
|
||||
name: 'event_date',
|
||||
type: 'date',
|
||||
default: '',
|
||||
placeholder: 'Select a date',
|
||||
help_text: 'Select the date for your event',
|
||||
optional: false,
|
||||
},
|
||||
{
|
||||
display_name: 'Meeting Time',
|
||||
name: 'meeting_time',
|
||||
type: 'datetime',
|
||||
default: '',
|
||||
placeholder: 'Select date and time',
|
||||
help_text: 'Select the date and time for your meeting',
|
||||
optional: false,
|
||||
time_interval: 60,
|
||||
},
|
||||
],
|
||||
submit_label: 'Submit',
|
||||
notify_on_cancel: true,
|
||||
state: 'datetime_state',
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
// Date field with min_date constraint - MM-T2530C
|
||||
function getMinDateConstraintDialog(triggerId, webhookBaseUrl) {
|
||||
return {
|
||||
trigger_id: triggerId,
|
||||
url: `${webhookBaseUrl}/datetime_dialog_submit`,
|
||||
dialog: {
|
||||
callback_id: 'mindate_callback',
|
||||
title: 'DateTime Fields Test',
|
||||
icon_url: 'https://mattermost.com/wp-content/uploads/2022/02/icon_WS.png',
|
||||
elements: [
|
||||
{
|
||||
display_name: 'Future Date Only',
|
||||
name: 'future_date',
|
||||
type: 'date',
|
||||
default: '',
|
||||
placeholder: 'Select a future date',
|
||||
help_text: 'Must be today or later',
|
||||
optional: true,
|
||||
min_date: 'today',
|
||||
},
|
||||
],
|
||||
submit_label: 'Submit',
|
||||
notify_on_cancel: true,
|
||||
state: 'datetime_state',
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
// DateTime field with custom time interval - MM-T2530D
|
||||
function getCustomIntervalDialog(triggerId, webhookBaseUrl) {
|
||||
return {
|
||||
trigger_id: triggerId,
|
||||
url: `${webhookBaseUrl}/datetime_dialog_submit`,
|
||||
dialog: {
|
||||
callback_id: 'interval_callback',
|
||||
title: 'DateTime Fields Test',
|
||||
icon_url: 'https://mattermost.com/wp-content/uploads/2022/02/icon_WS.png',
|
||||
elements: [
|
||||
{
|
||||
display_name: 'Custom Interval Time',
|
||||
name: 'interval_time',
|
||||
type: 'datetime',
|
||||
default: '',
|
||||
placeholder: 'Select time (30min intervals)',
|
||||
help_text: 'Time picker with 30-minute intervals',
|
||||
optional: true,
|
||||
time_interval: 30,
|
||||
},
|
||||
],
|
||||
submit_label: 'Submit',
|
||||
notify_on_cancel: true,
|
||||
state: 'datetime_state',
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
// Relative date values test - MM-T2530F
|
||||
function getRelativeDateDialog(triggerId, webhookBaseUrl) {
|
||||
return {
|
||||
trigger_id: triggerId,
|
||||
url: `${webhookBaseUrl}/datetime_dialog_submit`,
|
||||
dialog: {
|
||||
callback_id: 'relative_callback',
|
||||
title: 'DateTime Fields Test',
|
||||
icon_url: 'https://mattermost.com/wp-content/uploads/2022/02/icon_WS.png',
|
||||
elements: [
|
||||
{
|
||||
display_name: 'Relative Date Example',
|
||||
name: 'relative_date',
|
||||
type: 'date',
|
||||
default: 'today',
|
||||
placeholder: 'Today by default',
|
||||
help_text: 'Defaults to today using relative date',
|
||||
optional: true,
|
||||
},
|
||||
{
|
||||
display_name: 'Relative DateTime Example',
|
||||
name: 'relative_datetime',
|
||||
type: 'datetime',
|
||||
default: '+1d',
|
||||
placeholder: 'Tomorrow by default',
|
||||
help_text: 'Defaults to tomorrow using relative date',
|
||||
optional: true,
|
||||
},
|
||||
],
|
||||
submit_label: 'Submit',
|
||||
notify_on_cancel: true,
|
||||
state: 'datetime_state',
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
// Legacy function for backward compatibility - returns basic datetime dialog
|
||||
function getDateTimeDialog(triggerId, webhookBaseUrl) {
|
||||
return getBasicDateTimeDialog(triggerId, webhookBaseUrl);
|
||||
}
|
||||
|
||||
function getTimezoneManualDialog(triggerId, webhookBaseUrl) {
|
||||
return createDialog(triggerId, webhookBaseUrl, {
|
||||
callback_id: 'timezone_manual',
|
||||
title: 'Timezone & Manual Entry Demo',
|
||||
introduction_text:
|
||||
'**Timezone & Manual Entry Demo**\n\n' +
|
||||
'This dialog demonstrates timezone support and manual time entry features.',
|
||||
elements: [
|
||||
{
|
||||
display_name: 'Your Local Time (Manual Entry)',
|
||||
name: 'local_manual',
|
||||
type: 'datetime',
|
||||
help_text: 'Type any time: 9am, 14:30, 3:45pm - no rounding',
|
||||
datetime_config: {
|
||||
manual_time_entry: true,
|
||||
},
|
||||
optional: true,
|
||||
},
|
||||
{
|
||||
display_name: 'London Office Hours (Dropdown)',
|
||||
name: 'london_dropdown',
|
||||
type: 'datetime',
|
||||
help_text: 'Times shown in GMT - select from 60 min intervals',
|
||||
datetime_config: {
|
||||
location_timezone: 'Europe/London',
|
||||
time_interval: 60,
|
||||
},
|
||||
optional: true,
|
||||
},
|
||||
{
|
||||
display_name: 'London Office Hours (Manual Entry)',
|
||||
name: 'london_manual',
|
||||
type: 'datetime',
|
||||
help_text: 'Type time in GMT: 9am, 14:30, 3:45pm - no rounding',
|
||||
datetime_config: {
|
||||
location_timezone: 'Europe/London',
|
||||
manual_time_entry: true,
|
||||
},
|
||||
optional: true,
|
||||
},
|
||||
],
|
||||
});
|
||||
}
|
||||
|
||||
function getFileUploadDialog(triggerId, webhookBaseUrl) {
|
||||
return {
|
||||
trigger_id: triggerId,
|
||||
url: `${webhookBaseUrl}/dialog_submit`,
|
||||
dialog: {
|
||||
callback_id: 'somecallbackid',
|
||||
title: 'Title for Dialog Test with file upload element',
|
||||
icon_url: 'https://mattermost.com/wp-content/uploads/2022/02/icon_WS.png',
|
||||
submit_label: 'Submit File Upload Test',
|
||||
notify_on_cancel: true,
|
||||
state: 'somestate',
|
||||
elements: [
|
||||
{
|
||||
display_name: 'Upload Single Document',
|
||||
name: 'single_document',
|
||||
type: 'file',
|
||||
placeholder: 'Select one document...',
|
||||
help_text: 'Upload a single document (replaces previous selection).',
|
||||
optional: false,
|
||||
},
|
||||
{
|
||||
display_name: 'Upload Multiple Files',
|
||||
name: 'multiple_files',
|
||||
type: 'file',
|
||||
allow_multiple: true,
|
||||
placeholder: 'Select multiple files...',
|
||||
help_text: 'Upload multiple files (can select and add more).',
|
||||
optional: false,
|
||||
},
|
||||
{
|
||||
display_name: 'Description',
|
||||
name: 'description',
|
||||
type: 'textarea',
|
||||
subtype: '',
|
||||
default: '',
|
||||
placeholder: 'Describe the uploaded files...',
|
||||
help_text: 'Provide a description for the uploaded files.',
|
||||
optional: true,
|
||||
min_length: 0,
|
||||
max_length: 500,
|
||||
data_source: '',
|
||||
options: null,
|
||||
},
|
||||
],
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
function getActionButtonParentDialog(triggerId, webhookBaseUrl) {
|
||||
const config = {
|
||||
...DIALOG_CONFIGS.actionButtonParent,
|
||||
elements: [
|
||||
createElement('text', {
|
||||
display_name: 'Your Name',
|
||||
name: 'your_name',
|
||||
placeholder: 'Enter your name',
|
||||
optional: true,
|
||||
}),
|
||||
|
||||
// Two action buttons on the same dialog. Each carries a distinct
|
||||
// context.source so the child dialog can reflect which one was pressed.
|
||||
createElement('action_button', {
|
||||
display_name: 'Open Details',
|
||||
name: 'open_details',
|
||||
action_button: {
|
||||
url: `${webhookBaseUrl}/dialog/open_child`,
|
||||
context: {source: 'Details'},
|
||||
},
|
||||
}),
|
||||
createElement('action_button', {
|
||||
display_name: 'Open Summary',
|
||||
name: 'open_summary',
|
||||
action_button: {
|
||||
url: `${webhookBaseUrl}/dialog/open_child`,
|
||||
context: {source: 'Summary'},
|
||||
},
|
||||
}),
|
||||
],
|
||||
};
|
||||
return createDialog(triggerId, webhookBaseUrl, config);
|
||||
}
|
||||
|
||||
// `source` comes from the pressed action button's context.source and is reflected
|
||||
// in the child dialog's title and introduction text, so a test can verify which
|
||||
// button opened it.
|
||||
function getActionButtonChildDialog(triggerId, webhookBaseUrl, source) {
|
||||
const label = source || 'Unknown';
|
||||
const config = {
|
||||
...DIALOG_CONFIGS.actionButtonChild,
|
||||
title: `${label} Dialog`,
|
||||
introduction_text: `This child dialog was opened from the "${label}" action button.`,
|
||||
};
|
||||
return createDialog(triggerId, webhookBaseUrl, config);
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
getFullDialog,
|
||||
getSimpleDialog,
|
||||
getUserAndChannelDialog,
|
||||
getBooleanDialog,
|
||||
getFieldRefreshDialog,
|
||||
getMultistepStep1Dialog,
|
||||
getMultistepStep2Dialog,
|
||||
getMultistepStep3Dialog,
|
||||
getMultiSelectDialog,
|
||||
getDynamicSelectDialog,
|
||||
getDateTimeDialog,
|
||||
getBasicDateDialog,
|
||||
getBasicDateTimeDialog,
|
||||
getMinDateConstraintDialog,
|
||||
getCustomIntervalDialog,
|
||||
getRelativeDateDialog,
|
||||
getTimezoneManualDialog,
|
||||
getFileUploadDialog,
|
||||
getActionButtonParentDialog,
|
||||
getActionButtonChildDialog,
|
||||
};
|
||||
@@ -0,0 +1,770 @@
|
||||
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
||||
// See LICENSE.txt for license information.
|
||||
|
||||
/* eslint-disable no-console */
|
||||
|
||||
const express = require('express');
|
||||
const axios = require('axios');
|
||||
const ClientOAuth2 = require('client-oauth2');
|
||||
|
||||
const webhookUtils = require('./utils/webhook_utils');
|
||||
const postMessageAs = require('./tests/plugins/post_message_as');
|
||||
|
||||
const port = 3000;
|
||||
|
||||
const server = express();
|
||||
server.use(express.json());
|
||||
server.use(express.urlencoded({extended: true}));
|
||||
|
||||
process.title = process.argv[2];
|
||||
|
||||
server.get('/', ping);
|
||||
server.post('/setup', doSetup);
|
||||
server.post('/message_menus', postMessageMenus);
|
||||
server.post('/dialog_request', onDialogRequest);
|
||||
server.post('/simple_dialog_request', onSimpleDialogRequest);
|
||||
server.post('/user_and_channel_dialog_request', onUserAndChannelDialogRequest);
|
||||
server.post('/dialog_submit', onDialogSubmit);
|
||||
server.post('/boolean_dialog_request', onBooleanDialogRequest);
|
||||
server.post('/multiselect_dialog_request', onMultiSelectDialogRequest);
|
||||
server.post('/dynamic_select_dialog_request', onDynamicSelectDialogRequest);
|
||||
server.post('/file_upload_dialog_request', onFileUploadDialogRequest);
|
||||
server.post('/dynamic_select_source', onDynamicSelectSource);
|
||||
server.post('/dialog/field-refresh', onFieldRefreshDialogRequest);
|
||||
server.post('/dialog/multistep', onMultistepDialogRequest);
|
||||
server.post('/dialog/action_button_request', onActionButtonDialogRequest);
|
||||
server.post('/dialog/open_child', onOpenChildDialog);
|
||||
server.post('/field_refresh_source', onFieldRefreshSource);
|
||||
server.post('/datetime_dialog_request', onDateTimeDialogRequest);
|
||||
server.post('/datetime_dialog_submit', onDateTimeDialogSubmit);
|
||||
server.post('/slack_compatible_message_response', postSlackCompatibleMessageResponse);
|
||||
server.post('/mm_blocks_integration', postMmBlocksIntegration);
|
||||
server.post('/mm_blocks_integration_update', postMmBlocksIntegrationUpdate);
|
||||
server.post('/mm_blocks_integration_static_select', postMmBlocksIntegrationStaticSelect);
|
||||
server.post('/mm_blocks_integration_echo_query', postMmBlocksIntegrationEchoQuery);
|
||||
server.post('/mm_blocks_integration_echo_context', postMmBlocksIntegrationEchoContext);
|
||||
server.post('/send_message_to_channel', postSendMessageToChannel);
|
||||
server.post('/post_outgoing_webhook', postOutgoingWebhook);
|
||||
server.post('/send_oauth_credentials', postSendOauthCredentials);
|
||||
server.get('/start_oauth', getStartOAuth);
|
||||
server.get('/complete_oauth', getCompleteOauth);
|
||||
server.post('/post_oauth_message', postOAuthMessage);
|
||||
|
||||
server.listen(port, (err) => {
|
||||
if (err) {
|
||||
console.error(err);
|
||||
throw err;
|
||||
}
|
||||
console.log(`Webhook test server listening on port ${port}!`);
|
||||
});
|
||||
|
||||
function ping(req, res) {
|
||||
return res.json({
|
||||
message: "I'm alive!",
|
||||
endpoints: [
|
||||
'GET /',
|
||||
'POST /setup',
|
||||
'POST /message_menus',
|
||||
'POST /dialog_request',
|
||||
'POST /simple_dialog_request',
|
||||
'POST /user_and_channel_dialog_request',
|
||||
'POST /dialog_submit',
|
||||
'POST /boolean_dialog_request',
|
||||
'POST /multiselect_dialog_request',
|
||||
'POST /dynamic_select_dialog_request',
|
||||
'POST /file_upload_dialog_request',
|
||||
'POST /dynamic_select_source',
|
||||
'POST /dialog/field-refresh',
|
||||
'POST /dialog/multistep',
|
||||
'POST /dialog/action_button_request',
|
||||
'POST /dialog/open_child',
|
||||
'POST /field_refresh_source',
|
||||
'POST /datetime_dialog_request',
|
||||
'POST /datetime_dialog_submit',
|
||||
'POST /slack_compatible_message_response',
|
||||
'POST /mm_blocks_integration',
|
||||
'POST /mm_blocks_integration_update',
|
||||
'POST /mm_blocks_integration_static_select',
|
||||
'POST /mm_blocks_integration_echo_query',
|
||||
'POST /mm_blocks_integration_echo_context',
|
||||
'POST /send_message_to_channel',
|
||||
'POST /post_outgoing_webhook',
|
||||
'POST /send_oauth_credentials',
|
||||
'GET /start_oauth',
|
||||
'GET /complete_oauth',
|
||||
'POST /post_oauth_message',
|
||||
],
|
||||
});
|
||||
}
|
||||
|
||||
// Set base URLs and credential to be accessible by any endpoint
|
||||
let baseUrl;
|
||||
let webhookBaseUrl;
|
||||
let adminUsername;
|
||||
let adminPassword;
|
||||
function doSetup(req, res) {
|
||||
baseUrl = req.body.baseUrl;
|
||||
webhookBaseUrl = req.body.webhookBaseUrl;
|
||||
adminUsername = req.body.adminUsername;
|
||||
adminPassword = req.body.adminPassword;
|
||||
|
||||
return res.status(201).send('Successfully setup the new base URLs and credential.');
|
||||
}
|
||||
|
||||
let client;
|
||||
let authedUser;
|
||||
function postSendOauthCredentials(req, res) {
|
||||
const {appID, appSecret} = req.body;
|
||||
client = new ClientOAuth2({
|
||||
clientId: appID,
|
||||
clientSecret: appSecret,
|
||||
authorizationUri: `${baseUrl}/oauth/authorize`,
|
||||
accessTokenUri: `${baseUrl}/oauth/access_token`,
|
||||
redirectUri: `${webhookBaseUrl}/complete_oauth`,
|
||||
});
|
||||
return res.status(200).send('OK');
|
||||
}
|
||||
|
||||
function getStartOAuth(req, res) {
|
||||
return res.redirect(client.code.getUri());
|
||||
}
|
||||
|
||||
function getCompleteOauth(req, res) {
|
||||
client.code
|
||||
.getToken(req.originalUrl)
|
||||
.then((user) => {
|
||||
authedUser = user;
|
||||
return res.status(200).send('OK');
|
||||
})
|
||||
.catch((reason) => {
|
||||
return res.status(reason.status).send(reason);
|
||||
});
|
||||
}
|
||||
|
||||
async function postOAuthMessage(req, res) {
|
||||
const {channelId, message, rootId, createAt} = req.body;
|
||||
const apiUrl = `${baseUrl}/api/v4/posts`;
|
||||
authedUser.sign({
|
||||
method: 'post',
|
||||
url: apiUrl,
|
||||
});
|
||||
try {
|
||||
await axios({
|
||||
url: apiUrl,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'X-Requested-With': 'XMLHttpRequest',
|
||||
Authorization: 'Bearer ' + authedUser.accessToken,
|
||||
},
|
||||
method: 'post',
|
||||
data: {
|
||||
channel_id: channelId,
|
||||
message,
|
||||
type: '',
|
||||
create_at: createAt,
|
||||
root_id: rootId,
|
||||
},
|
||||
});
|
||||
} catch {
|
||||
// Do nothing
|
||||
}
|
||||
return res.status(200).send('OK');
|
||||
}
|
||||
|
||||
function postSlackCompatibleMessageResponse(req, res) {
|
||||
const {spoiler, skipSlackParsing} = req.body.context;
|
||||
|
||||
res.setHeader('Content-Type', 'application/json');
|
||||
return res.json({
|
||||
ephemeral_text: spoiler,
|
||||
skip_slack_parsing: skipSlackParsing,
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Mattermost mm_blocks external actions POST the same integration envelope as legacy message buttons.
|
||||
* @see model.PostActionIntegrationResponse
|
||||
*/
|
||||
function postMmBlocksIntegration(req, res) {
|
||||
const userName = req.body && req.body.user_name ? req.body.user_name : 'unknown';
|
||||
|
||||
res.setHeader('Content-Type', 'application/json');
|
||||
return res.status(200).json({
|
||||
ephemeral_text: `Playwright mm_blocks integration OK (user: ${userName}).`,
|
||||
skip_slack_parsing: true,
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns a PostActionIntegrationResponse update so the interactive post is edited in-place
|
||||
* (persisted webhook post or ephemeral mm_blocks post).
|
||||
*/
|
||||
function postMmBlocksIntegrationUpdate(req, res) {
|
||||
res.setHeader('Content-Type', 'application/json');
|
||||
return res.status(200).json({
|
||||
update: {
|
||||
message: 'E2E mm_blocks post updated (message field).',
|
||||
props: {
|
||||
mm_blocks: [
|
||||
{
|
||||
type: 'text',
|
||||
text: 'PLAYWRIGHT_MM_BLOCKS_UPDATED',
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
skip_slack_parsing: true,
|
||||
});
|
||||
}
|
||||
|
||||
/** Echoes URL query parameters Mattermost merged onto the integration request (action query + block query). */
|
||||
function postMmBlocksIntegrationEchoQuery(req, res) {
|
||||
const entries = Object.keys(req.query || {})
|
||||
.sort()
|
||||
.map((k) => `${k}=${String(req.query[k])}`);
|
||||
const summary = entries.join('&');
|
||||
|
||||
res.setHeader('Content-Type', 'application/json');
|
||||
return res.status(200).json({
|
||||
ephemeral_text: `Playwright mm_blocks query OK (${summary})`,
|
||||
skip_slack_parsing: true,
|
||||
});
|
||||
}
|
||||
|
||||
/** Echoes `context.test_marker` from the Mattermost integration POST body for mm_blocks external actions. */
|
||||
function postMmBlocksIntegrationEchoContext(req, res) {
|
||||
const ctx = (req.body && req.body.context) || {};
|
||||
const marker = typeof ctx.test_marker === 'string' ? ctx.test_marker : JSON.stringify(ctx.test_marker ?? null);
|
||||
|
||||
res.setHeader('Content-Type', 'application/json');
|
||||
return res.status(200).json({
|
||||
ephemeral_text: `Playwright mm_blocks context OK (test_marker: ${marker}).`,
|
||||
skip_slack_parsing: true,
|
||||
});
|
||||
}
|
||||
|
||||
/** Echoes `context.selected_option` from the Mattermost integration POST for mm_blocks static_select. */
|
||||
function postMmBlocksIntegrationStaticSelect(req, res) {
|
||||
const selected = req.body && req.body.context && req.body.context.selected_option;
|
||||
const label = typeof selected === 'string' ? selected : JSON.stringify(selected ?? null);
|
||||
|
||||
res.setHeader('Content-Type', 'application/json');
|
||||
return res.status(200).json({
|
||||
ephemeral_text: `Playwright mm_blocks static_select OK (selected_option: ${label}).`,
|
||||
skip_slack_parsing: true,
|
||||
});
|
||||
}
|
||||
|
||||
function postMessageMenus(req, res) {
|
||||
let responseData = {};
|
||||
const {body} = req;
|
||||
if (body && body.context.action === 'do_something') {
|
||||
responseData = {
|
||||
ephemeral_text: `Ephemeral | ${body.type} ${body.data_source} option: ${body.context.selected_option}`,
|
||||
};
|
||||
}
|
||||
|
||||
res.setHeader('Content-Type', 'application/json');
|
||||
return res.json(responseData);
|
||||
}
|
||||
|
||||
async function openDialog(dialog) {
|
||||
// Callers invoke this fire-and-forget (no await/catch), so any rejection here
|
||||
// would become an unhandled rejection and crash the whole webhook process.
|
||||
// Guard against a missing baseUrl (set by /setup) and swallow request errors.
|
||||
if (!baseUrl) {
|
||||
console.error('openDialog called before /setup ran — baseUrl is not set; skipping dialog open');
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
await axios({
|
||||
method: 'post',
|
||||
url: `${baseUrl}/api/v4/actions/dialogs/open`,
|
||||
data: dialog,
|
||||
});
|
||||
} catch (err) {
|
||||
const status = err.response && err.response.status;
|
||||
const body = err.response && err.response.data;
|
||||
console.error(
|
||||
'openDialog request failed:',
|
||||
status || err.code || err.message,
|
||||
body ? JSON.stringify(body) : '',
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
function onDialogRequest(req, res) {
|
||||
const {body} = req;
|
||||
if (body.trigger_id) {
|
||||
const dialog = webhookUtils.getFullDialog(body.trigger_id, webhookBaseUrl);
|
||||
openDialog(dialog);
|
||||
}
|
||||
|
||||
res.setHeader('Content-Type', 'application/json');
|
||||
return res.json({text: 'Full dialog triggered via slash command!'});
|
||||
}
|
||||
|
||||
function onSimpleDialogRequest(req, res) {
|
||||
const {body} = req;
|
||||
if (body.trigger_id) {
|
||||
const dialog = webhookUtils.getSimpleDialog(body.trigger_id, webhookBaseUrl);
|
||||
openDialog(dialog);
|
||||
}
|
||||
|
||||
res.setHeader('Content-Type', 'application/json');
|
||||
return res.json({text: 'Simple dialog triggered via slash command!'});
|
||||
}
|
||||
|
||||
function onUserAndChannelDialogRequest(req, res) {
|
||||
const {body} = req;
|
||||
if (body.trigger_id) {
|
||||
const dialog = webhookUtils.getUserAndChannelDialog(body.trigger_id, webhookBaseUrl);
|
||||
openDialog(dialog);
|
||||
}
|
||||
|
||||
res.setHeader('Content-Type', 'application/json');
|
||||
return res.json({text: 'Simple dialog triggered via slash command!'});
|
||||
}
|
||||
|
||||
function onBooleanDialogRequest(req, res) {
|
||||
const {body} = req;
|
||||
if (body.trigger_id) {
|
||||
const dialog = webhookUtils.getBooleanDialog(body.trigger_id, webhookBaseUrl);
|
||||
openDialog(dialog);
|
||||
}
|
||||
|
||||
res.setHeader('Content-Type', 'application/json');
|
||||
return res.json({text: 'Simple dialog triggered via slash command!'});
|
||||
}
|
||||
|
||||
function onMultiSelectDialogRequest(req, res) {
|
||||
const {body} = req;
|
||||
if (body.trigger_id) {
|
||||
// Check URL parameters or body for includeDefaults flag
|
||||
const includeDefaults = req.query.includeDefaults === 'true' || req.query.includeDefaults === true;
|
||||
const dialog = webhookUtils.getMultiSelectDialog(body.trigger_id, webhookBaseUrl, includeDefaults);
|
||||
openDialog(dialog);
|
||||
}
|
||||
|
||||
res.setHeader('Content-Type', 'application/json');
|
||||
return res.json({text: 'Multiselect dialog triggered via slash command!'});
|
||||
}
|
||||
|
||||
function onDynamicSelectDialogRequest(req, res) {
|
||||
const {body} = req;
|
||||
if (body.trigger_id) {
|
||||
const dialog = webhookUtils.getDynamicSelectDialog(body.trigger_id, webhookBaseUrl);
|
||||
openDialog(dialog);
|
||||
}
|
||||
|
||||
res.setHeader('Content-Type', 'application/json');
|
||||
return res.json({text: 'Dynamic select dialog triggered via slash command!'});
|
||||
}
|
||||
|
||||
function onFileUploadDialogRequest(req, res) {
|
||||
const {body} = req;
|
||||
if (body.trigger_id) {
|
||||
const dialog = webhookUtils.getFileUploadDialog(body.trigger_id, webhookBaseUrl);
|
||||
openDialog(dialog);
|
||||
}
|
||||
|
||||
res.setHeader('Content-Type', 'application/json');
|
||||
return res.json({text: 'File upload dialog triggered via slash command!'});
|
||||
}
|
||||
|
||||
function onDynamicSelectSource(req, res) {
|
||||
const {body} = req;
|
||||
|
||||
// Simulate dynamic options based on search text
|
||||
const searchText = (body.submission.query || '').toLowerCase();
|
||||
|
||||
const allOptions = [
|
||||
{text: 'Backend Engineer', value: 'backend_eng'},
|
||||
{text: 'Frontend Engineer', value: 'frontend_eng'},
|
||||
{text: 'Full Stack Engineer', value: 'fullstack_eng'},
|
||||
{text: 'DevOps Engineer', value: 'devops_eng'},
|
||||
{text: 'QA Engineer', value: 'qa_eng'},
|
||||
{text: 'Product Manager', value: 'product_mgr'},
|
||||
{text: 'Engineering Manager', value: 'eng_mgr'},
|
||||
{text: 'Senior Backend Engineer', value: 'sr_backend_eng'},
|
||||
{text: 'Senior Frontend Engineer', value: 'sr_frontend_eng'},
|
||||
{text: 'Principal Engineer', value: 'principal_eng'},
|
||||
{text: 'Staff Engineer', value: 'staff_eng'},
|
||||
{text: 'Technical Lead', value: 'tech_lead'},
|
||||
];
|
||||
|
||||
// Filter options based on search text
|
||||
const filteredOptions = searchText
|
||||
? allOptions.filter(
|
||||
(option) =>
|
||||
option.text.toLowerCase().includes(searchText) || option.value.toLowerCase().includes(searchText),
|
||||
)
|
||||
: allOptions.slice(0, 6); // Limit to first 6 if no search
|
||||
|
||||
res.setHeader('Content-Type', 'application/json');
|
||||
return res.json({
|
||||
items: filteredOptions,
|
||||
});
|
||||
}
|
||||
|
||||
function onDateTimeDialogRequest(req, res) {
|
||||
const {body} = req;
|
||||
if (body.trigger_id) {
|
||||
let dialog;
|
||||
const command = body.text ? body.text.trim() : '';
|
||||
|
||||
// Use focused dialog functions based on command parameter
|
||||
switch (command) {
|
||||
case 'basic':
|
||||
dialog = webhookUtils.getBasicDateDialog(body.trigger_id, webhookBaseUrl);
|
||||
break;
|
||||
case 'mindate':
|
||||
dialog = webhookUtils.getMinDateConstraintDialog(body.trigger_id, webhookBaseUrl);
|
||||
break;
|
||||
case 'interval':
|
||||
dialog = webhookUtils.getCustomIntervalDialog(body.trigger_id, webhookBaseUrl);
|
||||
break;
|
||||
case 'relative':
|
||||
dialog = webhookUtils.getRelativeDateDialog(body.trigger_id, webhookBaseUrl);
|
||||
break;
|
||||
case 'timezone-manual':
|
||||
dialog = webhookUtils.getTimezoneManualDialog(body.trigger_id, webhookBaseUrl);
|
||||
break;
|
||||
default:
|
||||
// Default to basic datetime dialog for backward compatibility
|
||||
dialog = webhookUtils.getBasicDateTimeDialog(body.trigger_id, webhookBaseUrl);
|
||||
break;
|
||||
}
|
||||
console.log('Opening DateTime dialog', dialog.dialog.title);
|
||||
openDialog(dialog);
|
||||
}
|
||||
|
||||
res.setHeader('Content-Type', 'application/json');
|
||||
return res.json({text: 'DateTime dialog triggered via slash command!'});
|
||||
}
|
||||
|
||||
function onDateTimeDialogSubmit(req, res) {
|
||||
console.log('DateTime dialog submit handler called!');
|
||||
const {body} = req;
|
||||
|
||||
res.setHeader('Content-Type', 'application/json');
|
||||
|
||||
// Log the submitted datetime values for debugging
|
||||
console.log('DateTime dialog submission:', JSON.stringify(body, null, 2));
|
||||
|
||||
// Extract datetime values from submission
|
||||
const submission = body.submission || {};
|
||||
const eventDate = submission.event_date;
|
||||
const meetingTime = submission.meeting_time;
|
||||
const relativeDate = submission.relative_date;
|
||||
const relativeDateTime = submission.relative_datetime;
|
||||
|
||||
// Create a success message with the submitted values
|
||||
let message = 'Form submitted successfully! ';
|
||||
if (eventDate || meetingTime || relativeDate || relativeDateTime) {
|
||||
message += 'Submitted values: ';
|
||||
if (eventDate) {
|
||||
message += `Event Date: ${eventDate}, `;
|
||||
}
|
||||
if (meetingTime) {
|
||||
message += `Meeting Time: ${meetingTime}, `;
|
||||
}
|
||||
if (relativeDate) {
|
||||
message += `Relative Date: ${relativeDate}, `;
|
||||
}
|
||||
if (relativeDateTime) {
|
||||
message += `Relative DateTime: ${relativeDateTime}, `;
|
||||
}
|
||||
message = message.slice(0, -2); // Remove trailing comma and space
|
||||
}
|
||||
|
||||
// Send success response that will appear as a post in the channel
|
||||
sendSysadminResponse(message, body.channel_id);
|
||||
return res.json({text: message});
|
||||
}
|
||||
|
||||
function onDialogSubmit(req, res) {
|
||||
const {body} = req;
|
||||
|
||||
res.setHeader('Content-Type', 'application/json');
|
||||
|
||||
let message;
|
||||
if (body.cancelled) {
|
||||
message = 'Dialog cancelled';
|
||||
console.log('[WEBHOOK] Dialog cancelled');
|
||||
sendSysadminResponse(message, body.channel_id);
|
||||
return res.json({text: message});
|
||||
}
|
||||
|
||||
// Check if this is a multistep submission
|
||||
if (body.callback_id === 'multistep_callback') {
|
||||
const currentState = body.state || '';
|
||||
|
||||
// Determine next step based on current state
|
||||
if (currentState === 'step1') {
|
||||
// Move to step 2
|
||||
const nextForm = webhookUtils.getMultistepStep2Dialog(null, webhookBaseUrl);
|
||||
return res.json({
|
||||
type: 'form',
|
||||
form: nextForm,
|
||||
});
|
||||
} else if (currentState === 'step2') {
|
||||
// Move to step 3
|
||||
const nextForm = webhookUtils.getMultistepStep3Dialog(null, webhookBaseUrl);
|
||||
return res.json({
|
||||
type: 'form',
|
||||
form: nextForm,
|
||||
});
|
||||
}
|
||||
|
||||
// Final step - complete the multistep
|
||||
const submission = body.submission || {};
|
||||
message = `Multistep completed successfully! Final step values: ${JSON.stringify(submission, null, 2)}`;
|
||||
sendSysadminResponse(message, body.channel_id);
|
||||
return res.json({text: message});
|
||||
}
|
||||
|
||||
// Check if this is a field refresh dialog submission
|
||||
if (body.callback_id === 'field_refresh_callback') {
|
||||
const submission = body.submission || {};
|
||||
message = `Field refresh dialog submitted successfully! Values: ${JSON.stringify(submission, null, 2)}`;
|
||||
sendSysadminResponse(message, body.channel_id);
|
||||
return res.json({text: message});
|
||||
}
|
||||
|
||||
// Regular dialog submission
|
||||
// Format submission data for the channel message
|
||||
const sanitize = (str) => String(str).replace(/[<>&"']/g, (ch) => `&#${ch.charCodeAt(0)};`);
|
||||
const submissionData = Object.entries(body.submission || {})
|
||||
.map(([key, value]) => `**${sanitize(key)}**: ${sanitize(value)}`)
|
||||
.join('\n');
|
||||
|
||||
message = `Dialog submitted successfully!\n\n**Submission Data:**\n${submissionData}`;
|
||||
|
||||
sendSysadminResponse(message, body.channel_id);
|
||||
return res.json({text: message});
|
||||
}
|
||||
|
||||
/**
|
||||
* @route "POST /send_message_to_channel?type={messageType}&channel_id={channelId}"
|
||||
* @query type - message type of empty string for regular message if not provided (default), "system_message", etc
|
||||
* @query channel_id - channel where to send the message
|
||||
*/
|
||||
function postSendMessageToChannel(req, res) {
|
||||
const channelId = req.query.channel_id;
|
||||
const response = {
|
||||
response_type: 'in_channel',
|
||||
text: 'Extra response 2',
|
||||
channel_id: channelId,
|
||||
extra_responses: [
|
||||
{
|
||||
response_type: 'in_channel',
|
||||
text: 'Hello World',
|
||||
channel_id: channelId,
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
if (req.query.type) {
|
||||
response.type = req.query.type;
|
||||
}
|
||||
|
||||
res.json(response);
|
||||
}
|
||||
|
||||
// Convenient way to send response in a channel by using sysadmin account
|
||||
function sendSysadminResponse(message, channelId) {
|
||||
postMessageAs({
|
||||
sender: {
|
||||
username: adminUsername,
|
||||
password: adminPassword,
|
||||
},
|
||||
message,
|
||||
channelId,
|
||||
baseUrl,
|
||||
});
|
||||
}
|
||||
|
||||
const responseTypes = ['in_channel', 'comment'];
|
||||
|
||||
function getWebhookResponse(body, {responseType, username, iconUrl}) {
|
||||
const payload = Object.entries(body)
|
||||
.map(([key, value]) => `- ${key}: "${value}"`)
|
||||
.join('\n');
|
||||
|
||||
return `
|
||||
\`\`\`
|
||||
#### Outgoing Webhook Payload
|
||||
${payload}
|
||||
#### Webhook override to Mattermost instance
|
||||
- response_type: "${responseType}"
|
||||
- type: ""
|
||||
- username: "${username}"
|
||||
- icon_url: "${iconUrl}"
|
||||
\`\`\`
|
||||
`;
|
||||
}
|
||||
|
||||
/**
|
||||
* @route "POST /post_outgoing_webhook?override_username={username}&override_icon_url={iconUrl}&response_type={comment}"
|
||||
* @query override_username - the user name that overrides the user name defined by the outgoing webhook
|
||||
* @query override_icon_url - the user icon url that overrides the user icon url defined by the outgoing webhook
|
||||
* @query response_type - "in_channel" (default) or "comment"
|
||||
*/
|
||||
function postOutgoingWebhook(req, res) {
|
||||
const {body, query} = req;
|
||||
if (!body) {
|
||||
res.status(404).send({error: 'Invalid data'});
|
||||
}
|
||||
|
||||
const responseType = query.response_type || responseTypes[0];
|
||||
const username = query.override_username || '';
|
||||
const iconUrl = query.override_icon_url || '';
|
||||
|
||||
const response = {
|
||||
text: getWebhookResponse(body, {responseType, username, iconUrl}),
|
||||
username,
|
||||
icon_url: iconUrl,
|
||||
type: '',
|
||||
response_type: responseType,
|
||||
};
|
||||
res.status(200).send(response);
|
||||
}
|
||||
|
||||
function onFieldRefreshDialogRequest(req, res) {
|
||||
const {body} = req;
|
||||
if (body.trigger_id) {
|
||||
const dialog = webhookUtils.getFieldRefreshDialog(body.trigger_id, webhookBaseUrl);
|
||||
openDialog(dialog);
|
||||
}
|
||||
|
||||
res.setHeader('Content-Type', 'application/json');
|
||||
return res.json({text: 'Field refresh dialog triggered via slash command!'});
|
||||
}
|
||||
|
||||
function onMultistepDialogRequest(req, res) {
|
||||
const {body} = req;
|
||||
if (body.trigger_id) {
|
||||
const dialog = webhookUtils.getMultistepStep1Dialog(body.trigger_id, webhookBaseUrl);
|
||||
openDialog(dialog);
|
||||
}
|
||||
|
||||
res.setHeader('Content-Type', 'application/json');
|
||||
return res.json({text: 'Multistep dialog triggered via slash command!'});
|
||||
}
|
||||
|
||||
function onActionButtonDialogRequest(req, res) {
|
||||
const {body} = req;
|
||||
if (body.trigger_id) {
|
||||
const dialog = webhookUtils.getActionButtonParentDialog(body.trigger_id, webhookBaseUrl);
|
||||
openDialog(dialog);
|
||||
}
|
||||
|
||||
res.setHeader('Content-Type', 'application/json');
|
||||
return res.json({text: 'Action button dialog triggered!'});
|
||||
}
|
||||
|
||||
async function onOpenChildDialog(req, res) {
|
||||
const {body} = req;
|
||||
|
||||
// context.source identifies which action button on the parent dialog was
|
||||
// pressed; it is forwarded by the server in the PostActionIntegrationRequest.
|
||||
const source = (body.context && body.context.source) || 'Unknown';
|
||||
console.log('onOpenChildDialog called with trigger_id:', body.trigger_id, 'source:', source);
|
||||
if (body.trigger_id) {
|
||||
const childDialog = webhookUtils.getActionButtonChildDialog(body.trigger_id, webhookBaseUrl, source);
|
||||
|
||||
// Await the dialog open before responding. The server's /execute call
|
||||
// (DoActionRequest) waits for this response, so awaiting here ensures the
|
||||
// child's WS open_dialog event is published before the browser's
|
||||
// executeDialogAction promise resolves — removing the render race in tests.
|
||||
await openDialog(childDialog);
|
||||
}
|
||||
|
||||
res.setHeader('Content-Type', 'application/json');
|
||||
return res.json({});
|
||||
}
|
||||
|
||||
function onFieldRefreshSource(req, res) {
|
||||
const {body} = req;
|
||||
const submission = body.submission || {};
|
||||
const projectType = submission.project_type;
|
||||
const projectName = submission.project_name || '';
|
||||
|
||||
res.setHeader('Content-Type', 'application/json');
|
||||
|
||||
// Return updated form based on project type selection
|
||||
const elements = [
|
||||
{
|
||||
display_name: 'Project Name',
|
||||
name: 'project_name',
|
||||
type: 'text',
|
||||
placeholder: 'Enter project name',
|
||||
default: projectName,
|
||||
optional: false,
|
||||
},
|
||||
{
|
||||
display_name: 'Project Type',
|
||||
name: 'project_type',
|
||||
type: 'select',
|
||||
refresh: true,
|
||||
placeholder: 'Select project type...',
|
||||
default: projectType,
|
||||
options: [
|
||||
{text: 'Web Application', value: 'web'},
|
||||
{text: 'Mobile App', value: 'mobile'},
|
||||
{text: 'API Service', value: 'api'},
|
||||
],
|
||||
},
|
||||
];
|
||||
|
||||
// Add different fields based on project type
|
||||
if (projectType === 'web') {
|
||||
elements.push({
|
||||
display_name: 'Framework',
|
||||
name: 'framework',
|
||||
type: 'select',
|
||||
placeholder: 'Select framework...',
|
||||
options: [
|
||||
{text: 'React', value: 'react'},
|
||||
{text: 'Vue', value: 'vue'},
|
||||
{text: 'Angular', value: 'angular'},
|
||||
],
|
||||
});
|
||||
} else if (projectType === 'mobile') {
|
||||
elements.push({
|
||||
display_name: 'Platform',
|
||||
name: 'platform',
|
||||
type: 'select',
|
||||
placeholder: 'Select platform...',
|
||||
options: [
|
||||
{text: 'iOS', value: 'ios'},
|
||||
{text: 'Android', value: 'android'},
|
||||
{text: 'React Native', value: 'react-native'},
|
||||
],
|
||||
});
|
||||
} else if (projectType === 'api') {
|
||||
elements.push({
|
||||
display_name: 'Language',
|
||||
name: 'language',
|
||||
type: 'select',
|
||||
placeholder: 'Select language...',
|
||||
options: [
|
||||
{text: 'Go', value: 'go'},
|
||||
{text: 'Node.js', value: 'nodejs'},
|
||||
{text: 'Python', value: 'python'},
|
||||
],
|
||||
});
|
||||
}
|
||||
|
||||
return res.json({
|
||||
type: 'form',
|
||||
form: {
|
||||
title: 'Field Refresh Demo',
|
||||
introduction_text: 'Enter project name then select type to see different fields',
|
||||
submit_label: 'Submit',
|
||||
elements,
|
||||
},
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,37 @@
|
||||
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
||||
// See LICENSE.txt for license information.
|
||||
|
||||
import {GenericContainer, Wait} from 'testcontainers';
|
||||
import type {StartedNetwork, StartedTestContainer} from 'testcontainers';
|
||||
|
||||
import {AZURITE_ALIAS, AZURITE_BLOB_PORT, TESTCONTAINERS_LABELS} from './constants';
|
||||
import {AZURITE_IMAGE} from './default_images';
|
||||
import {startWithRetry} from './retry';
|
||||
|
||||
import {testConfig} from '@/test_config';
|
||||
|
||||
// An alternative to Minio for blob storage.
|
||||
export async function startAzuriteContainer(network: StartedNetwork): Promise<StartedTestContainer> {
|
||||
return startWithRetry('azurite', async () => {
|
||||
let builder = new GenericContainer(AZURITE_IMAGE)
|
||||
.withCommand([
|
||||
'azurite-blob',
|
||||
'--blobHost',
|
||||
'0.0.0.0',
|
||||
'--blobPort',
|
||||
String(AZURITE_BLOB_PORT),
|
||||
'--skipApiVersionCheck',
|
||||
])
|
||||
.withExposedPorts(AZURITE_BLOB_PORT)
|
||||
.withNetwork(network)
|
||||
.withNetworkAliases(AZURITE_ALIAS)
|
||||
.withLabels(TESTCONTAINERS_LABELS)
|
||||
.withWaitStrategy(Wait.forListeningPorts());
|
||||
|
||||
if (testConfig.testcontainersReuse) {
|
||||
builder = builder.withReuse();
|
||||
}
|
||||
|
||||
return builder.start();
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,68 @@
|
||||
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
||||
// See LICENSE.txt for license information.
|
||||
|
||||
// Single source of truth for the fixed values every container/helper needs to agree on.
|
||||
// Kept separate from test_config.ts because these are not overridable — they're either
|
||||
// Testcontainers network aliases (only meaningful inside the Testcontainers network) or
|
||||
// fixed test-only credentials for a throwaway local service.
|
||||
|
||||
export const POSTGRES_ALIAS = 'postgres';
|
||||
export const POSTGRES_PORT = 5432;
|
||||
export const POSTGRES_DB = 'mattermost_test';
|
||||
export const POSTGRES_USER = 'mmuser';
|
||||
export const POSTGRES_PASSWORD = 'mostest';
|
||||
|
||||
export const INBUCKET_ALIAS = 'inbucket';
|
||||
export const INBUCKET_WEB_PORT = 9001;
|
||||
export const INBUCKET_SMTP_PORT = 10025;
|
||||
export const INBUCKET_POP3_PORT = 10110;
|
||||
|
||||
export const MATTERMOST_ALIAS = 'server';
|
||||
export const MATTERMOST_PORT = 8065;
|
||||
|
||||
// Interactive-message/dialog callback sidecar shared with Cypress. Always started, like
|
||||
// postgres/inbucket — not gated behind testcontainersServices.
|
||||
export const WEBHOOK_ALIAS = 'webhook';
|
||||
export const WEBHOOK_PORT = 3000;
|
||||
|
||||
export const OPENLDAP_ALIAS = 'openldap';
|
||||
export const OPENLDAP_PORT = 389;
|
||||
export const OPENLDAP_ADMIN_DN = 'cn=admin,dc=mm,dc=test,dc=com';
|
||||
export const OPENLDAP_ADMIN_PASSWORD = 'mostest';
|
||||
export const OPENLDAP_BASE_DN = 'dc=mm,dc=test,dc=com';
|
||||
|
||||
export const KEYCLOAK_ALIAS = 'keycloak';
|
||||
export const KEYCLOAK_PORT = 8080;
|
||||
export const KEYCLOAK_REALM = 'mattermost';
|
||||
export const KEYCLOAK_ADMIN_USER = 'admin';
|
||||
export const KEYCLOAK_ADMIN_PASSWORD = 'admin';
|
||||
|
||||
export const ELASTICSEARCH_ALIAS = 'elasticsearch';
|
||||
export const ELASTICSEARCH_PORT = 9200;
|
||||
|
||||
export const MINIO_ALIAS = 'minio';
|
||||
export const MINIO_PORT = 9000;
|
||||
export const MINIO_ACCESS_KEY = 'minioaccesskey';
|
||||
export const MINIO_SECRET_KEY = 'miniosecretkey';
|
||||
export const MINIO_BUCKET = 'mattermost-test';
|
||||
|
||||
// Alternative to Minio for blob storage.
|
||||
export const AZURITE_ALIAS = 'azurite';
|
||||
export const AZURITE_BLOB_PORT = 10000;
|
||||
// Azurite's well-known default emulator account — published by Microsoft's own docs, not a secret.
|
||||
export const AZURITE_ACCOUNT_NAME = 'devstoreaccount1';
|
||||
export const AZURITE_ACCOUNT_KEY =
|
||||
'Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==';
|
||||
export const AZURITE_CONTAINER = 'mattermost-test';
|
||||
|
||||
// Alternative to Elasticsearch for search.
|
||||
export const OPENSEARCH_ALIAS = 'opensearch';
|
||||
export const OPENSEARCH_PORT = 9201;
|
||||
export const OPENSEARCH_ADMIN_PASSWORD = 'Test@dmin_123';
|
||||
|
||||
// Applied to every container this module starts, so `npm run testcontainers:down` can find and
|
||||
// remove them from a fresh process — the in-memory `started` state in stack.ts only exists in
|
||||
// the process that created it.
|
||||
export const TESTCONTAINERS_LABEL_KEY = 'mm-playwright-testcontainers';
|
||||
export const TESTCONTAINERS_LABEL_VALUE = 'true';
|
||||
export const TESTCONTAINERS_LABELS = {[TESTCONTAINERS_LABEL_KEY]: TESTCONTAINERS_LABEL_VALUE};
|
||||
@@ -0,0 +1,18 @@
|
||||
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
||||
// See LICENSE.txt for license information.
|
||||
|
||||
// Single place to check and bump every image.
|
||||
// The Mattermost server's default is overridable via the SERVER_IMAGE env var (testConfig.serverImage).
|
||||
export const MATTERMOST_SERVER_IMAGE = 'mattermostdevelopment/mattermost-enterprise-edition:master';
|
||||
export const POSTGRES_IMAGE = 'postgres:14';
|
||||
export const INBUCKET_IMAGE = 'inbucket/inbucket:3.1.1';
|
||||
export const OPENLDAP_IMAGE = 'osixia/openldap:1.4.0';
|
||||
export const KEYCLOAK_IMAGE = 'quay.io/keycloak/keycloak:23.0.7';
|
||||
export const MINIO_IMAGE = 'minio/minio:RELEASE.2024-06-22T05-26-45Z';
|
||||
export const AZURITE_IMAGE = 'mcr.microsoft.com/azure-storage/azurite:3.34.0';
|
||||
// Built from a Dockerfile on top of docker.elastic.co/elasticsearch/elasticsearch, rather than
|
||||
// pulled as a fixed image — so only the version is fixed here.
|
||||
export const ELASTICSEARCH_VERSION = '9.0.0';
|
||||
// Built from a Dockerfile on top of opensearchproject/opensearch, rather than pulled as a fixed
|
||||
// image — so only the version is fixed here.
|
||||
export const OPENSEARCH_VERSION = '3.0.0';
|
||||
@@ -0,0 +1,48 @@
|
||||
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
||||
// See LICENSE.txt for license information.
|
||||
|
||||
import {GenericContainer, Wait} from 'testcontainers';
|
||||
import type {StartedNetwork, StartedTestContainer} from 'testcontainers';
|
||||
|
||||
import {ELASTICSEARCH_ALIAS, ELASTICSEARCH_PORT, TESTCONTAINERS_LABELS} from './constants';
|
||||
import {ELASTICSEARCH_VERSION as DEFAULT_ELASTICSEARCH_VERSION} from './default_images';
|
||||
import {containerAssetPath} from './paths';
|
||||
import {startWithRetry} from './retry';
|
||||
|
||||
import {testConfig} from '@/test_config';
|
||||
|
||||
// Built from a vendored Dockerfile, not the generic @testcontainers/elasticsearch wrapper — it
|
||||
// installs the CJK analysis plugins (analysis-icu/nori/kuromoji/smartcn) real search tests rely on.
|
||||
export async function startElasticsearchContainer(network: StartedNetwork): Promise<StartedTestContainer> {
|
||||
return startWithRetry('elasticsearch', async () => {
|
||||
// deleteOnExit: false — otherwise the built image bakes in this session's Ryuk id, so Ryuk
|
||||
// reaps the "reused" container the moment this session ends, defeating withReuse() below.
|
||||
const image = await GenericContainer.fromDockerfile(containerAssetPath(), 'Dockerfile.elasticsearch')
|
||||
.withBuildArgs({
|
||||
ELASTICSEARCH_VERSION: process.env.ELASTICSEARCH_VERSION || DEFAULT_ELASTICSEARCH_VERSION,
|
||||
})
|
||||
.build(undefined, {deleteOnExit: false});
|
||||
|
||||
let builder = image
|
||||
.withExposedPorts(ELASTICSEARCH_PORT)
|
||||
.withNetwork(network)
|
||||
.withNetworkAliases(ELASTICSEARCH_ALIAS)
|
||||
.withLabels(TESTCONTAINERS_LABELS)
|
||||
.withEnvironment({
|
||||
'http.host': '0.0.0.0',
|
||||
'http.port': String(ELASTICSEARCH_PORT),
|
||||
'xpack.security.enabled': 'false',
|
||||
'action.destructive_requires_name': 'false',
|
||||
'transport.host': '127.0.0.1',
|
||||
ES_JAVA_OPTS: '-Xms512m -Xmx512m',
|
||||
})
|
||||
.withStartupTimeout(3 * 60_000)
|
||||
.withWaitStrategy(Wait.forHttp('/_cluster/health', ELASTICSEARCH_PORT).forStatusCode(200));
|
||||
|
||||
if (testConfig.testcontainersReuse) {
|
||||
builder = builder.withReuse();
|
||||
}
|
||||
|
||||
return builder.start();
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
||||
// See LICENSE.txt for license information.
|
||||
|
||||
// Test-oriented MM_* config the Mattermost server container starts with by default,
|
||||
// merged under testConfig.serverEnv (MM_ENV) so callers can still override any of it.
|
||||
export const SERVER_ENV_BASELINE: Record<string, string> = {
|
||||
MM_SERVICEENVIRONMENT: 'test',
|
||||
MM_CLUSTERSETTINGS_READONLYCONFIG: 'false',
|
||||
MM_CONNECTEDWORKSPACESSETTINGS_ENABLEREMOTECLUSTERSERVICE: 'true',
|
||||
MM_CONNECTEDWORKSPACESSETTINGS_ENABLESHAREDWORKSPACES: 'true',
|
||||
MM_LOGSETTINGS_CONSOLELEVEL: 'DEBUG',
|
||||
MM_LOGSETTINGS_ENABLEDIAGNOSTICS: 'false',
|
||||
MM_PLUGINSETTINGS_ENABLEUPLOADS: 'true',
|
||||
MM_SERVICESETTINGS_ALLOWCORSFROM: '*',
|
||||
MM_SERVICESETTINGS_ALLOWEDUNTRUSTEDINTERNALCONNECTIONS: 'keycloak elasticsearch opensearch minio azurite webhook',
|
||||
MM_SERVICESETTINGS_ENABLELOCALMODE: 'true',
|
||||
MM_SERVICESETTINGS_ENABLESECURITYFIXALERT: 'false',
|
||||
MM_SERVICESETTINGS_ENABLETESTING: 'true',
|
||||
// Feature flags this test suite needs on, off by default in the server
|
||||
// Kept in sync with e2e-tests/.ci/server.generate.sh for this release
|
||||
MM_FEATUREFLAGS_ENABLEREMOTECLUSTERSERVICE: 'true',
|
||||
MM_FEATUREFLAGS_MOVETHREADSENABLED: 'true',
|
||||
MM_FEATUREFLAGS_CUSTOMPROFILEATTRIBUTES: 'true',
|
||||
MM_FEATUREFLAGS_PERMISSIONPOLICIES: 'true',
|
||||
MM_FEATUREFLAGS_CLASSIFICATIONMARKINGS: 'true',
|
||||
};
|
||||
@@ -0,0 +1,39 @@
|
||||
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
||||
// See LICENSE.txt for license information.
|
||||
|
||||
import {GenericContainer, Wait} from 'testcontainers';
|
||||
import type {StartedNetwork, StartedTestContainer} from 'testcontainers';
|
||||
|
||||
import {
|
||||
INBUCKET_ALIAS,
|
||||
INBUCKET_POP3_PORT,
|
||||
INBUCKET_SMTP_PORT,
|
||||
INBUCKET_WEB_PORT,
|
||||
TESTCONTAINERS_LABELS,
|
||||
} from './constants';
|
||||
import {INBUCKET_IMAGE} from './default_images';
|
||||
import {startWithRetry} from './retry';
|
||||
|
||||
import {testConfig} from '@/test_config';
|
||||
|
||||
export async function startInbucketContainer(network: StartedNetwork): Promise<StartedTestContainer> {
|
||||
return startWithRetry('inbucket', async () => {
|
||||
let builder = new GenericContainer(INBUCKET_IMAGE)
|
||||
.withExposedPorts(INBUCKET_WEB_PORT, INBUCKET_SMTP_PORT, INBUCKET_POP3_PORT)
|
||||
.withNetwork(network)
|
||||
.withNetworkAliases(INBUCKET_ALIAS)
|
||||
.withLabels(TESTCONTAINERS_LABELS)
|
||||
.withEnvironment({
|
||||
INBUCKET_WEB_ADDR: `0.0.0.0:${INBUCKET_WEB_PORT}`,
|
||||
INBUCKET_POP3_ADDR: `0.0.0.0:${INBUCKET_POP3_PORT}`,
|
||||
INBUCKET_SMTP_ADDR: `0.0.0.0:${INBUCKET_SMTP_PORT}`,
|
||||
})
|
||||
.withWaitStrategy(Wait.forListeningPorts());
|
||||
|
||||
if (testConfig.testcontainersReuse) {
|
||||
builder = builder.withReuse();
|
||||
}
|
||||
|
||||
return builder.start();
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,4 @@
|
||||
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
||||
// See LICENSE.txt for license information.
|
||||
|
||||
export {startStack, stopStack} from './stack';
|
||||
@@ -0,0 +1,51 @@
|
||||
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
||||
// See LICENSE.txt for license information.
|
||||
|
||||
import {GenericContainer, Wait} from 'testcontainers';
|
||||
import type {StartedNetwork, StartedTestContainer} from 'testcontainers';
|
||||
|
||||
import {
|
||||
KEYCLOAK_ADMIN_PASSWORD,
|
||||
KEYCLOAK_ADMIN_USER,
|
||||
KEYCLOAK_ALIAS,
|
||||
KEYCLOAK_PORT,
|
||||
TESTCONTAINERS_LABELS,
|
||||
} from './constants';
|
||||
import {KEYCLOAK_IMAGE} from './default_images';
|
||||
import {containerAssetPath} from './paths';
|
||||
import {startWithRetry} from './retry';
|
||||
|
||||
import {testConfig} from '@/test_config';
|
||||
|
||||
export async function startKeycloakContainer(network: StartedNetwork): Promise<StartedTestContainer> {
|
||||
return startWithRetry('keycloak', async () => {
|
||||
let builder = new GenericContainer(KEYCLOAK_IMAGE)
|
||||
.withEntrypoint(['/opt/keycloak/bin/kc.sh'])
|
||||
.withCommand(['start', '--import-realm'])
|
||||
.withExposedPorts(KEYCLOAK_PORT)
|
||||
.withNetwork(network)
|
||||
.withNetworkAliases(KEYCLOAK_ALIAS)
|
||||
.withLabels(TESTCONTAINERS_LABELS)
|
||||
.withEnvironment({
|
||||
KEYCLOAK_ADMIN: KEYCLOAK_ADMIN_USER,
|
||||
KEYCLOAK_ADMIN_PASSWORD,
|
||||
KC_HOSTNAME_STRICT: 'false',
|
||||
KC_HOSTNAME_STRICT_HTTPS: 'false',
|
||||
KC_HTTP_ENABLED: 'true',
|
||||
})
|
||||
.withCopyFilesToContainer([
|
||||
{
|
||||
source: containerAssetPath('keycloak-realm-export.json'),
|
||||
target: '/opt/keycloak/data/import/realm-export.json',
|
||||
},
|
||||
])
|
||||
.withStartupTimeout(3 * 60_000)
|
||||
.withWaitStrategy(Wait.forHttp('/realms/master', KEYCLOAK_PORT).forStatusCode(200));
|
||||
|
||||
if (testConfig.testcontainersReuse) {
|
||||
builder = builder.withReuse();
|
||||
}
|
||||
|
||||
return builder.start();
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,21 @@
|
||||
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
||||
// See LICENSE.txt for license information.
|
||||
|
||||
import chalk from 'chalk';
|
||||
|
||||
const PREFIX = chalk.cyan('[testcontainers]');
|
||||
|
||||
export function logTestcontainers(message: string): void {
|
||||
// eslint-disable-next-line no-console
|
||||
console.log(`${PREFIX} ${message}`);
|
||||
}
|
||||
|
||||
export function warnTestcontainers(message: string): void {
|
||||
// eslint-disable-next-line no-console
|
||||
console.warn(`${PREFIX} ${message}`);
|
||||
}
|
||||
|
||||
export function errorTestcontainers(message: string): void {
|
||||
// eslint-disable-next-line no-console
|
||||
console.error(`${PREFIX} ${message}`);
|
||||
}
|
||||
@@ -0,0 +1,100 @@
|
||||
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
||||
// See LICENSE.txt for license information.
|
||||
|
||||
import {GenericContainer, Wait} from 'testcontainers';
|
||||
import type {StartedTestContainer} from 'testcontainers';
|
||||
|
||||
import {
|
||||
INBUCKET_ALIAS,
|
||||
INBUCKET_SMTP_PORT,
|
||||
MATTERMOST_ALIAS,
|
||||
MATTERMOST_PORT,
|
||||
POSTGRES_ALIAS,
|
||||
POSTGRES_DB,
|
||||
POSTGRES_PASSWORD,
|
||||
POSTGRES_PORT,
|
||||
POSTGRES_USER,
|
||||
TESTCONTAINERS_LABELS,
|
||||
} from './constants';
|
||||
import {SERVER_ENV_BASELINE} from './env_baseline';
|
||||
import {startWithRetry} from './retry';
|
||||
|
||||
import {testConfig} from '@/test_config';
|
||||
|
||||
// Env this container computes itself from the stack Testcontainers just built. Must win over any
|
||||
// stray testConfig.serverEnv (MM_ENV) entry and over testConfig.bootEnvOverrides (passed in as
|
||||
// `extraEnv` by restartMattermostContainer()), or a stray key collision there could break the
|
||||
// server's own connectivity. Deliberately does NOT know about any additional service (LDAP/
|
||||
// Keycloak/Elasticsearch/OpenSearch/Minio/Azurite) — those are each spec's own responsibility via
|
||||
// pw.ensure<Service>(), which points the already-running server at them through patchConfig.
|
||||
//
|
||||
// MM_LICENSE (if set) is passed straight through: the server reads it directly at startup
|
||||
// (platform.LoadLicense), so it boots already licensed instead of needing an authenticated upload
|
||||
// call after the fact.
|
||||
function structuralEnv(): Record<string, string> {
|
||||
return {
|
||||
MM_SQLSETTINGS_DRIVERNAME: 'postgres',
|
||||
MM_SQLSETTINGS_DATASOURCE: `postgres://${POSTGRES_USER}:${POSTGRES_PASSWORD}@${POSTGRES_ALIAS}:${POSTGRES_PORT}/${POSTGRES_DB}?sslmode=disable&connect_timeout=10&binary_parameters=yes`,
|
||||
MM_EMAILSETTINGS_SMTPSERVER: INBUCKET_ALIAS,
|
||||
MM_EMAILSETTINGS_SMTPPORT: String(INBUCKET_SMTP_PORT),
|
||||
...(process.env.MM_LICENSE ? {MM_LICENSE: process.env.MM_LICENSE} : {}),
|
||||
// Overrides (not merges) SERVER_ENV_BASELINE's own value for this same key — appends the
|
||||
// network's gateway IP so the SSRF guard also allows fetching from file_server.ts's mock
|
||||
// file server, reachable at that address (see test_config.ts). Only known once the
|
||||
// network is up (testConfig.testcontainersNetworkGatewayIp is set by stack.ts's
|
||||
// startStack() before this container ever starts), so falls back to the baseline's own
|
||||
// value verbatim on the off chance this ever runs without it.
|
||||
MM_SERVICESETTINGS_ALLOWEDUNTRUSTEDINTERNALCONNECTIONS: testConfig.testcontainersNetworkGatewayIp
|
||||
? `${SERVER_ENV_BASELINE.MM_SERVICESETTINGS_ALLOWEDUNTRUSTEDINTERNALCONNECTIONS} ${testConfig.testcontainersNetworkGatewayIp}`
|
||||
: SERVER_ENV_BASELINE.MM_SERVICESETTINGS_ALLOWEDUNTRUSTEDINTERNALCONNECTIONS,
|
||||
};
|
||||
}
|
||||
|
||||
// Readiness requires both the /api/v4/system/ping health check AND the permissions-migration job
|
||||
// scheduler's "All migrations are complete." log line (scheduler.go, jobs/migrations package).
|
||||
// Ping alone isn't enough: MigrationKeyAdvancedPermissionsPhase2 runs as an async job whose
|
||||
// scheduler deliberately delays its first tick 60s after startup — a real window a spec's very
|
||||
// first API call can otherwise land inside, tripping IsPhase2MigrationCompleted() gates with
|
||||
// "required migrations have not yet completed" (confirmed in practice: a permissions-page spec
|
||||
// hit exactly this, with the "Edit Scheme" link stuck disabled, when this wait was dropped).
|
||||
// Requires MM_LOGSETTINGS_CONSOLELEVEL=DEBUG (env_baseline.ts) since the scheduler logs that line
|
||||
// at Debug. Only paid on a genuinely fresh boot — a reused/adopted stack skips this entirely.
|
||||
//
|
||||
// Joins the network by name (withNetworkMode) rather than a StartedNetwork object: also called
|
||||
// from restartMattermostContainer(), which runs in a worker process that never holds the actual
|
||||
// StartedNetwork handle — only the network's name (threaded through testConfig) is available
|
||||
// there.
|
||||
export async function startMattermostContainer(
|
||||
networkName: string,
|
||||
extraEnv: Record<string, string> = {},
|
||||
): Promise<StartedTestContainer> {
|
||||
const env: Record<string, string> = {
|
||||
...SERVER_ENV_BASELINE,
|
||||
...testConfig.serverEnv,
|
||||
...extraEnv,
|
||||
...structuralEnv(),
|
||||
};
|
||||
|
||||
return startWithRetry('server', async () => {
|
||||
let builder = new GenericContainer(testConfig.serverImage)
|
||||
.withPlatform('linux/amd64') // The published server images are amd64-only.
|
||||
.withNetworkMode(networkName)
|
||||
.withNetworkAliases(MATTERMOST_ALIAS)
|
||||
.withLabels(TESTCONTAINERS_LABELS)
|
||||
.withExposedPorts(MATTERMOST_PORT)
|
||||
.withEnvironment(env)
|
||||
.withStartupTimeout(5 * 60_000)
|
||||
.withWaitStrategy(
|
||||
Wait.forAll([
|
||||
Wait.forHttp('/api/v4/system/ping', MATTERMOST_PORT).forStatusCode(200),
|
||||
Wait.forLogMessage(/All migrations are complete\./, 1),
|
||||
]),
|
||||
);
|
||||
|
||||
if (testConfig.testcontainersReuse) {
|
||||
builder = builder.withReuse();
|
||||
}
|
||||
|
||||
return builder.start();
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,33 @@
|
||||
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
||||
// See LICENSE.txt for license information.
|
||||
|
||||
import {GenericContainer, Wait} from 'testcontainers';
|
||||
import type {StartedNetwork, StartedTestContainer} from 'testcontainers';
|
||||
|
||||
import {MINIO_ACCESS_KEY, MINIO_ALIAS, MINIO_PORT, MINIO_SECRET_KEY, TESTCONTAINERS_LABELS} from './constants';
|
||||
import {MINIO_IMAGE} from './default_images';
|
||||
import {startWithRetry} from './retry';
|
||||
|
||||
import {testConfig} from '@/test_config';
|
||||
|
||||
export async function startMinioContainer(network: StartedNetwork): Promise<StartedTestContainer> {
|
||||
return startWithRetry('minio', async () => {
|
||||
let builder = new GenericContainer(MINIO_IMAGE)
|
||||
.withCommand(['server', '/data', '--console-address', ':9002'])
|
||||
.withExposedPorts(MINIO_PORT)
|
||||
.withNetwork(network)
|
||||
.withNetworkAliases(MINIO_ALIAS)
|
||||
.withLabels(TESTCONTAINERS_LABELS)
|
||||
.withEnvironment({
|
||||
MINIO_ROOT_USER: MINIO_ACCESS_KEY,
|
||||
MINIO_ROOT_PASSWORD: MINIO_SECRET_KEY,
|
||||
})
|
||||
.withWaitStrategy(Wait.forHttp('/minio/health/live', MINIO_PORT).forStatusCode(200));
|
||||
|
||||
if (testConfig.testcontainersReuse) {
|
||||
builder = builder.withReuse();
|
||||
}
|
||||
|
||||
return builder.start();
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,120 @@
|
||||
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
||||
// See LICENSE.txt for license information.
|
||||
|
||||
import {execFile} from 'node:child_process';
|
||||
import {promisify} from 'node:util';
|
||||
|
||||
import {GenericContainer} from 'testcontainers';
|
||||
import type {WaitStrategy} from 'testcontainers';
|
||||
|
||||
import {MATTERMOST_ALIAS, MATTERMOST_PORT, TESTCONTAINERS_LABELS} from './constants';
|
||||
|
||||
import {testConfig} from '@/test_config';
|
||||
|
||||
const execFileAsync = promisify(execFile);
|
||||
|
||||
const MMCTL_ENTRYPOINT = '/mattermost/bin/mmctl';
|
||||
const MMCTL_CONFIG_DIR = '/tmp/mmctl-xdg';
|
||||
const MMCTL_CREDENTIALS_NAME = 'e2e';
|
||||
|
||||
export type MmctlResult = {
|
||||
exitCode: number;
|
||||
output: string;
|
||||
};
|
||||
|
||||
// Matches the Credentials/CredentialsList shape mmctl reads (server/cmd/mmctl/commands/auth_utils.go).
|
||||
// Written directly instead of via `mmctl auth login`, since that needs a password file and this
|
||||
// image has no shell to create one. AuthMethod "T" treats authToken as a plain bearer token, which
|
||||
// mmctl reads into Client4.AuthToken.
|
||||
function buildCredentialsFileContent(username: string, authToken: string): string {
|
||||
const credentialsList = {
|
||||
[MMCTL_CREDENTIALS_NAME]: {
|
||||
name: MMCTL_CREDENTIALS_NAME,
|
||||
username,
|
||||
authToken,
|
||||
authMethod: 'T',
|
||||
instanceUrl: `http://${MATTERMOST_ALIAS}:${MATTERMOST_PORT}`,
|
||||
active: true,
|
||||
},
|
||||
};
|
||||
return JSON.stringify(credentialsList);
|
||||
}
|
||||
|
||||
async function streamToString(stream: NodeJS.ReadableStream): Promise<string> {
|
||||
const chunks: Buffer[] = [];
|
||||
for await (const chunk of stream) {
|
||||
chunks.push(Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk));
|
||||
}
|
||||
return Buffer.concat(chunks).toString('utf-8');
|
||||
}
|
||||
|
||||
// Wait.forOneShotStartup() throws on any non-zero exit code, but a non-zero exit from mmctl is a
|
||||
// meaningful result to inspect, not a startup failure. This strategy ignores the exit code;
|
||||
// completion is awaited afterward via `docker wait`.
|
||||
class NoOpWaitStrategy implements WaitStrategy {
|
||||
private startupTimeoutMs = 0;
|
||||
|
||||
async waitUntilReady(): Promise<void> {
|
||||
// No-op — completion is awaited by the caller via `docker wait`.
|
||||
}
|
||||
|
||||
withStartupTimeout(startupTimeoutMs: number): this {
|
||||
this.startupTimeoutMs = startupTimeoutMs;
|
||||
return this;
|
||||
}
|
||||
|
||||
isStartupTimeoutSet(): boolean {
|
||||
return true;
|
||||
}
|
||||
|
||||
getStartupTimeout(): number {
|
||||
return this.startupTimeoutMs;
|
||||
}
|
||||
}
|
||||
|
||||
async function waitForExitCode(containerId: string): Promise<number> {
|
||||
// Bounded so a hung mmctl command can't stall cleanup indefinitely.
|
||||
const {stdout} = await execFileAsync('docker', ['wait', containerId], {timeout: 60_000});
|
||||
return parseInt(stdout.trim(), 10);
|
||||
}
|
||||
|
||||
/**
|
||||
* Runs a single mmctl command in its own throwaway container built from the server image, acting
|
||||
* as a real remote client rather than the `--local` unix-socket mode used for the server's healthcheck.
|
||||
*
|
||||
* Joins the network by name (withNetworkMode), not via getNetwork() — this runs in the Playwright
|
||||
* worker process, a different OS process from the one that created the network, so getNetwork()'s
|
||||
* in-process cache would create a second, unrelated network instead of finding the real one.
|
||||
*/
|
||||
export async function runMmctl(args: string[], username: string, authToken: string): Promise<MmctlResult> {
|
||||
if (!testConfig.testcontainersNetworkName) {
|
||||
throw new Error(
|
||||
'No Testcontainers network name available (PW_TESTCONTAINERS_NETWORK_NAME) — is PW_USE_TESTCONTAINERS=true?',
|
||||
);
|
||||
}
|
||||
|
||||
const container = await new GenericContainer(testConfig.serverImage)
|
||||
.withPlatform('linux/amd64') // The published server images are amd64-only.
|
||||
.withNetworkMode(testConfig.testcontainersNetworkName)
|
||||
.withLabels(TESTCONTAINERS_LABELS)
|
||||
.withEnvironment({XDG_CONFIG_HOME: MMCTL_CONFIG_DIR})
|
||||
.withCopyContentToContainer([
|
||||
{
|
||||
content: buildCredentialsFileContent(username, authToken),
|
||||
target: `${MMCTL_CONFIG_DIR}/mmctl/config`,
|
||||
},
|
||||
])
|
||||
.withEntrypoint([MMCTL_ENTRYPOINT])
|
||||
.withCommand(args)
|
||||
.withWaitStrategy(new NoOpWaitStrategy())
|
||||
.withStartupTimeout(60_000)
|
||||
.start();
|
||||
|
||||
try {
|
||||
const exitCode = await waitForExitCode(container.getId());
|
||||
const output = await streamToString(await container.logs());
|
||||
return {exitCode, output};
|
||||
} finally {
|
||||
await container.stop({remove: true});
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,73 @@
|
||||
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
||||
// See LICENSE.txt for license information.
|
||||
|
||||
import {execFile} from 'node:child_process';
|
||||
import {promisify} from 'node:util';
|
||||
|
||||
import {Network} from 'testcontainers';
|
||||
import type {StartedNetwork} from 'testcontainers';
|
||||
|
||||
import {logTestcontainers, warnTestcontainers} from './log';
|
||||
|
||||
const execFileAsync = promisify(execFile);
|
||||
|
||||
// One bridge network per Playwright invocation, shared by every container it starts. When
|
||||
// Playwright itself runs inside a container, that container also joins this network so it can
|
||||
// reach everything by alias instead of a mapped port.
|
||||
let startedNetwork: StartedNetwork | undefined;
|
||||
|
||||
export async function getNetwork(): Promise<StartedNetwork> {
|
||||
if (!startedNetwork) {
|
||||
logTestcontainers('creating network...');
|
||||
startedNetwork = await new Network().start();
|
||||
logTestcontainers('network created.');
|
||||
}
|
||||
return startedNetwork;
|
||||
}
|
||||
|
||||
/**
|
||||
* The bridge network's gateway IP (e.g. 172.18.0.1) — a real address bound to an interface on the
|
||||
* Docker host itself, so a process listening on 0.0.0.0 on the host is reachable both from the
|
||||
* host directly and from any container on this network, without a network alias or mapped port.
|
||||
* Takes a network id/name rather than a StartedNetwork object so it also works from
|
||||
* reuseExistingStack(), which only has testConfig's network name.
|
||||
*/
|
||||
export async function getNetworkGatewayIp(network: string): Promise<string> {
|
||||
const {stdout} = await execFileAsync('docker', [
|
||||
'network',
|
||||
'inspect',
|
||||
network,
|
||||
'--format',
|
||||
'{{(index .IPAM.Config 0).Gateway}}',
|
||||
]);
|
||||
return stdout.trim();
|
||||
}
|
||||
|
||||
export async function stopNetwork(): Promise<void> {
|
||||
if (!startedNetwork) {
|
||||
return;
|
||||
}
|
||||
|
||||
const network = startedNetwork;
|
||||
startedNetwork = undefined;
|
||||
|
||||
// A container a worker process swapped in via restartMattermostContainer() (a different OS
|
||||
// process, invisible to this one) can still be mid-detach from the network at this exact
|
||||
// moment, which Docker reports as "has active endpoints". Ryuk removes the network anyway
|
||||
// once that settles, so retry briefly rather than surfacing a scary but harmless error.
|
||||
const attempts = 5;
|
||||
for (let attempt = 1; attempt <= attempts; attempt++) {
|
||||
try {
|
||||
await network.stop();
|
||||
return;
|
||||
} catch (error) {
|
||||
if (attempt === attempts) {
|
||||
warnTestcontainers(
|
||||
`could not remove network ${network.getId()} (Ryuk will remove it shortly): ${String(error)}`,
|
||||
);
|
||||
return;
|
||||
}
|
||||
await new Promise((resolve) => setTimeout(resolve, 1000));
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,37 @@
|
||||
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
||||
// See LICENSE.txt for license information.
|
||||
|
||||
import {GenericContainer, Wait} from 'testcontainers';
|
||||
import type {StartedNetwork, StartedTestContainer} from 'testcontainers';
|
||||
|
||||
import {OPENLDAP_ADMIN_PASSWORD, OPENLDAP_ALIAS, OPENLDAP_PORT, TESTCONTAINERS_LABELS} from './constants';
|
||||
import {OPENLDAP_IMAGE} from './default_images';
|
||||
import {startWithRetry} from './retry';
|
||||
|
||||
import {testConfig} from '@/test_config';
|
||||
|
||||
// osixia/openldap is known to occasionally fail its first boot under load — startWithRetry
|
||||
// retries a bounded number of times rather than letting a flaky first attempt fail the whole run.
|
||||
export async function startOpenldapContainer(network: StartedNetwork): Promise<StartedTestContainer> {
|
||||
return startWithRetry('openldap', async () => {
|
||||
let builder = new GenericContainer(OPENLDAP_IMAGE)
|
||||
.withExposedPorts(OPENLDAP_PORT)
|
||||
.withNetwork(network)
|
||||
.withNetworkAliases(OPENLDAP_ALIAS)
|
||||
.withLabels(TESTCONTAINERS_LABELS)
|
||||
.withEnvironment({
|
||||
LDAP_TLS_VERIFY_CLIENT: 'never',
|
||||
LDAP_ORGANISATION: 'Mattermost Test',
|
||||
LDAP_DOMAIN: 'mm.test.com',
|
||||
LDAP_ADMIN_PASSWORD: OPENLDAP_ADMIN_PASSWORD,
|
||||
})
|
||||
.withStartupTimeout(2 * 60_000)
|
||||
.withWaitStrategy(Wait.forListeningPorts());
|
||||
|
||||
if (testConfig.testcontainersReuse) {
|
||||
builder = builder.withReuse();
|
||||
}
|
||||
|
||||
return builder.start();
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,46 @@
|
||||
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
||||
// See LICENSE.txt for license information.
|
||||
|
||||
import {GenericContainer, Wait} from 'testcontainers';
|
||||
import type {StartedNetwork, StartedTestContainer} from 'testcontainers';
|
||||
|
||||
import {OPENSEARCH_ADMIN_PASSWORD, OPENSEARCH_ALIAS, OPENSEARCH_PORT, TESTCONTAINERS_LABELS} from './constants';
|
||||
import {OPENSEARCH_VERSION as DEFAULT_OPENSEARCH_VERSION} from './default_images';
|
||||
import {containerAssetPath} from './paths';
|
||||
import {startWithRetry} from './retry';
|
||||
|
||||
import {testConfig} from '@/test_config';
|
||||
|
||||
// Built from a vendored Dockerfile (installs the same CJK analysis plugins as the
|
||||
// Elasticsearch container) — an alternative to Elasticsearch for search.
|
||||
export async function startOpensearchContainer(network: StartedNetwork): Promise<StartedTestContainer> {
|
||||
return startWithRetry('opensearch', async () => {
|
||||
// deleteOnExit: false — otherwise the built image bakes in this session's Ryuk id, so Ryuk
|
||||
// reaps the "reused" container the moment this session ends, defeating withReuse() below.
|
||||
const image = await GenericContainer.fromDockerfile(containerAssetPath(), 'Dockerfile.opensearch')
|
||||
.withBuildArgs({OPENSEARCH_VERSION: process.env.OPENSEARCH_VERSION || DEFAULT_OPENSEARCH_VERSION})
|
||||
.build(undefined, {deleteOnExit: false});
|
||||
|
||||
let builder = image
|
||||
.withExposedPorts(OPENSEARCH_PORT)
|
||||
.withNetwork(network)
|
||||
.withNetworkAliases(OPENSEARCH_ALIAS)
|
||||
.withLabels(TESTCONTAINERS_LABELS)
|
||||
.withEnvironment({
|
||||
'http.port': String(OPENSEARCH_PORT),
|
||||
'discovery.type': 'single-node',
|
||||
'plugins.security.disabled': 'true',
|
||||
DISABLE_INSTALL_DEMO_CONFIG: 'true',
|
||||
OPENSEARCH_INITIAL_ADMIN_PASSWORD: OPENSEARCH_ADMIN_PASSWORD,
|
||||
OPENSEARCH_JAVA_OPTS: '-Xms512m -Xmx512m',
|
||||
})
|
||||
.withStartupTimeout(3 * 60_000)
|
||||
.withWaitStrategy(Wait.forHttp('/_cluster/health', OPENSEARCH_PORT).forStatusCode(200));
|
||||
|
||||
if (testConfig.testcontainersReuse) {
|
||||
builder = builder.withReuse();
|
||||
}
|
||||
|
||||
return builder.start();
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
||||
// See LICENSE.txt for license information.
|
||||
|
||||
import path from 'node:path';
|
||||
|
||||
// Resolved relative to this module's own location (not the caller's cwd or the monorepo), so it
|
||||
// keeps working once `@mattermost/playwright-lib` is installed as an npm package with no access
|
||||
// to the rest of the repo. `preserveModules` keeps `containers/assets` alongside this file's
|
||||
// compiled output in `dist`, same as `src`.
|
||||
//
|
||||
// `__dirname` rather than `import.meta.url`: despite `"type": "module"`, Playwright loads this
|
||||
// package via `require()`, not `import()` — and Node's require()-of-ESM interop disallows
|
||||
// `import.meta` (throws "Cannot use 'import.meta' outside a module"), while `__dirname` still
|
||||
// resolves since Node wraps the module as CommonJS to support that require() call.
|
||||
const assetsDir = path.join(__dirname, 'assets');
|
||||
|
||||
export function containerAssetPath(...segments: string[]): string {
|
||||
return path.join(assetsDir, ...segments);
|
||||
}
|
||||
@@ -0,0 +1,40 @@
|
||||
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
||||
// See LICENSE.txt for license information.
|
||||
|
||||
import {PostgreSqlContainer} from '@testcontainers/postgresql';
|
||||
import type {StartedPostgreSqlContainer} from '@testcontainers/postgresql';
|
||||
import {Wait} from 'testcontainers';
|
||||
import type {StartedNetwork} from 'testcontainers';
|
||||
|
||||
import {POSTGRES_ALIAS, POSTGRES_DB, POSTGRES_PASSWORD, POSTGRES_USER, TESTCONTAINERS_LABELS} from './constants';
|
||||
import {POSTGRES_IMAGE} from './default_images';
|
||||
import {containerAssetPath} from './paths';
|
||||
import {startWithRetry} from './retry';
|
||||
|
||||
import {testConfig} from '@/test_config';
|
||||
|
||||
export async function startPostgresContainer(network: StartedNetwork): Promise<StartedPostgreSqlContainer> {
|
||||
return startWithRetry('postgres', async () => {
|
||||
let builder = new PostgreSqlContainer(POSTGRES_IMAGE)
|
||||
.withDatabase(POSTGRES_DB)
|
||||
.withUsername(POSTGRES_USER)
|
||||
.withPassword(POSTGRES_PASSWORD)
|
||||
.withNetwork(network)
|
||||
.withNetworkAliases(POSTGRES_ALIAS)
|
||||
.withLabels(TESTCONTAINERS_LABELS)
|
||||
.withCopyFilesToContainer([
|
||||
{
|
||||
source: containerAssetPath('postgres.conf'),
|
||||
target: '/etc/postgresql/postgresql.conf',
|
||||
},
|
||||
])
|
||||
.withCommand(['postgres', '-c', 'config_file=/etc/postgresql/postgresql.conf'])
|
||||
.withWaitStrategy(Wait.forLogMessage(/database system is ready to accept connections/, 1));
|
||||
|
||||
if (testConfig.testcontainersReuse) {
|
||||
builder = builder.withReuse();
|
||||
}
|
||||
|
||||
return builder.start();
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
||||
// See LICENSE.txt for license information.
|
||||
|
||||
import type {StartedNetwork, StartedTestContainer} from 'testcontainers';
|
||||
|
||||
import {startAzuriteContainer} from './azurite_container';
|
||||
import {startElasticsearchContainer} from './elasticsearch_container';
|
||||
import {startKeycloakContainer} from './keycloak_container';
|
||||
import {startMinioContainer} from './minio_container';
|
||||
import {startOpenldapContainer} from './openldap_container';
|
||||
import {startOpensearchContainer} from './opensearch_container';
|
||||
|
||||
import type {TestContainersServiceName} from '@/test_config';
|
||||
|
||||
// The single place to extend when a new additional service is needed. `stack.ts` starts these
|
||||
// only for the names in testConfig.testcontainersServices.
|
||||
export const ADDITIONAL_SERVICE_STARTERS: Record<
|
||||
TestContainersServiceName,
|
||||
(network: StartedNetwork) => Promise<StartedTestContainer>
|
||||
> = {
|
||||
openldap: startOpenldapContainer,
|
||||
keycloak: startKeycloakContainer,
|
||||
elasticsearch: startElasticsearchContainer,
|
||||
opensearch: startOpensearchContainer,
|
||||
minio: startMinioContainer,
|
||||
azurite: startAzuriteContainer,
|
||||
};
|
||||
@@ -0,0 +1,38 @@
|
||||
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
||||
// See LICENSE.txt for license information.
|
||||
|
||||
import {errorTestcontainers} from './log';
|
||||
|
||||
import {duration, wait} from '@/util';
|
||||
|
||||
const DEFAULT_ATTEMPTS = 3;
|
||||
|
||||
// Wraps a container's build+start so transient failures (image pull/build over a flaky network,
|
||||
// occasional first-boot flakiness) get a bounded, backed-off retry instead of failing the whole
|
||||
// run — and so any failure, at any attempt, is unambiguous about which container/image caused it,
|
||||
// rather than surfacing as testcontainers' own generic "Failed to build image"/"Failed to start
|
||||
// container" with no name attached.
|
||||
export async function startWithRetry<T>(
|
||||
label: string,
|
||||
start: () => Promise<T>,
|
||||
attempts = DEFAULT_ATTEMPTS,
|
||||
): Promise<T> {
|
||||
let lastError: unknown;
|
||||
|
||||
for (let attempt = 1; attempt <= attempts; attempt++) {
|
||||
try {
|
||||
return await start();
|
||||
} catch (error) {
|
||||
lastError = error;
|
||||
const message = error instanceof Error ? error.message : String(error);
|
||||
errorTestcontainers(`"${label}" failed on attempt ${attempt}/${attempts}: ${message}`);
|
||||
if (attempt < attempts) {
|
||||
await wait(duration.two_sec * attempt);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
throw new Error(`Failed to start "${label}" container after ${attempts} attempts: ${String(lastError)}`, {
|
||||
cause: lastError,
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,677 @@
|
||||
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
||||
// See LICENSE.txt for license information.
|
||||
|
||||
import {execFile} from 'node:child_process';
|
||||
import fs from 'node:fs';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
import {promisify} from 'node:util';
|
||||
|
||||
import {test} from '@playwright/test';
|
||||
import type {StartedPostgreSqlContainer} from '@testcontainers/postgresql';
|
||||
import type {StartedNetwork, StartedTestContainer} from 'testcontainers';
|
||||
|
||||
import {
|
||||
AZURITE_ALIAS,
|
||||
AZURITE_BLOB_PORT,
|
||||
ELASTICSEARCH_ALIAS,
|
||||
ELASTICSEARCH_PORT,
|
||||
INBUCKET_ALIAS,
|
||||
INBUCKET_WEB_PORT,
|
||||
KEYCLOAK_ALIAS,
|
||||
KEYCLOAK_PORT,
|
||||
MATTERMOST_ALIAS,
|
||||
MATTERMOST_PORT,
|
||||
MINIO_ALIAS,
|
||||
MINIO_PORT,
|
||||
OPENLDAP_ALIAS,
|
||||
OPENLDAP_PORT,
|
||||
OPENSEARCH_ALIAS,
|
||||
OPENSEARCH_PORT,
|
||||
POSTGRES_ALIAS,
|
||||
POSTGRES_DB,
|
||||
POSTGRES_PASSWORD,
|
||||
POSTGRES_PORT,
|
||||
POSTGRES_USER,
|
||||
WEBHOOK_ALIAS,
|
||||
WEBHOOK_PORT,
|
||||
} from './constants';
|
||||
import {
|
||||
AZURITE_IMAGE,
|
||||
ELASTICSEARCH_VERSION,
|
||||
INBUCKET_IMAGE,
|
||||
KEYCLOAK_IMAGE,
|
||||
MINIO_IMAGE,
|
||||
OPENLDAP_IMAGE,
|
||||
OPENSEARCH_VERSION,
|
||||
POSTGRES_IMAGE,
|
||||
} from './default_images';
|
||||
import {startInbucketContainer} from './inbucket_container';
|
||||
import {logTestcontainers} from './log';
|
||||
import {startMattermostContainer} from './mattermost_container';
|
||||
import {getNetwork, getNetworkGatewayIp, stopNetwork} from './network';
|
||||
import {startPostgresContainer} from './postgres_container';
|
||||
import {ADDITIONAL_SERVICE_STARTERS} from './requirements';
|
||||
import {startWebhookContainer} from './webhook_container';
|
||||
|
||||
import {clearClientCache} from '@/server/client';
|
||||
import {defaultBootEnv, testConfig} from '@/test_config';
|
||||
import type {TestContainersServiceName} from '@/test_config';
|
||||
import {duration} from '@/util';
|
||||
|
||||
const execFileAsync = promisify(execFile);
|
||||
|
||||
const ENV_FILE_PATH = path.resolve(process.cwd(), '.env.testcontainers');
|
||||
const LOG_DIR = path.resolve(process.cwd(), 'logs');
|
||||
|
||||
type StartedStack = {
|
||||
network: StartedNetwork;
|
||||
postgres: StartedPostgreSqlContainer;
|
||||
inbucket: StartedTestContainer;
|
||||
// Opt-in on older release branches (PW_TESTCONTAINERS_WEBHOOK=true); unused specs are omitted.
|
||||
webhook?: StartedTestContainer;
|
||||
mattermost: StartedTestContainer;
|
||||
additional: Partial<Record<TestContainersServiceName, StartedTestContainer>>;
|
||||
};
|
||||
|
||||
let started: StartedStack | undefined;
|
||||
// True if this process is running against a stack an EARLIER process created (via
|
||||
// reuseExistingStack()) rather than one it started itself — there are no real Testcontainers
|
||||
// handles to hold in this case, only values .env.testcontainers resolved into testConfig via
|
||||
// dotenv. A reusing process never owns the stack's lifecycle, so stopStack() must leave both
|
||||
// the containers and the env file untouched for whatever other process still needs them.
|
||||
let reused = false;
|
||||
|
||||
/**
|
||||
* Brings up a bridge network, Postgres, Inbucket, the Mattermost server, and whichever
|
||||
* additional services testConfig.testcontainersServices names. No-op if `testcontainers` mode isn't
|
||||
* selected (PW_USE_TESTCONTAINERS unset), already started (repeated calls within the same
|
||||
* process, e.g. a stray double-invocation, are harmless), or already reused. Otherwise first
|
||||
* checks whether an earlier process's stack is still alive and reuses it instead — see
|
||||
* reuseExistingStack().
|
||||
*/
|
||||
export async function startStack(): Promise<void> {
|
||||
if (!testConfig.useTestContainers || started || reused) {
|
||||
return;
|
||||
}
|
||||
|
||||
if (await reuseExistingStack()) {
|
||||
return;
|
||||
}
|
||||
|
||||
// reuseExistingStack() found nothing live to reuse — any bootEnvOverrides read from a stale
|
||||
// .env.testcontainers (e.g. left behind by a manual `docker rm` or a crashed prior process)
|
||||
// no longer describes anything real. Reset to the genuine defaults the container about to be
|
||||
// created will actually boot with, or a later restart could wrongly believe some stale
|
||||
// setting is already active and skip a restart it actually needs.
|
||||
testConfig.bootEnvOverrides = defaultBootEnv();
|
||||
|
||||
const network = await getNetwork();
|
||||
|
||||
if (testConfig.containerRunner) {
|
||||
await joinSelfToNetwork(network.getId());
|
||||
}
|
||||
|
||||
// Stored so a host-side mock file server can be reached from both the browser and containers
|
||||
// on this network (see getNetworkGatewayIp).
|
||||
testConfig.testcontainersNetworkGatewayIp = await getNetworkGatewayIp(network.getId());
|
||||
|
||||
const additionalNames = testConfig.testcontainersServices;
|
||||
// Older release CI omits webhook-dependent specs; opt in with PW_TESTCONTAINERS_WEBHOOK=true.
|
||||
const startWebhook = process.env.PW_TESTCONTAINERS_WEBHOOK === 'true';
|
||||
|
||||
logTestcontainers(
|
||||
`pulling/starting images: server, postgres, inbucket${startWebhook ? ', webhook' : ''}${additionalNames.length ? `, ${additionalNames.join(', ')}` : ''}`,
|
||||
);
|
||||
await logServerImageAge(testConfig.serverImage);
|
||||
|
||||
// Tracks every container that actually comes up, independent of whether the group as a whole
|
||||
// (or the mattermost start after it) ultimately succeeds — so a failure partway through still
|
||||
// knows exactly what to tear down instead of leaking whatever already started.
|
||||
const startedContainers: StartedTestContainer[] = [];
|
||||
const trackAndLog = <T extends StartedTestContainer>(name: string, promise: Promise<T>): Promise<T> => {
|
||||
const startedAt = Date.now();
|
||||
|
||||
// The biggest blind spot is the server: its own wait strategy alone can take minutes
|
||||
// (see mattermost_container.ts), during which nothing else prints — so ping every 30s to
|
||||
// make clear the run hasn't stalled.
|
||||
const heartbeat = setInterval(() => {
|
||||
logTestcontainers(`still waiting on ${name} (${elapsedSeconds(startedAt)}s elapsed)...`);
|
||||
}, duration.half_min);
|
||||
|
||||
return promise.then(
|
||||
(container) => {
|
||||
clearInterval(heartbeat);
|
||||
startedContainers.push(container);
|
||||
logTestcontainers(`${name} ready in ${elapsedSeconds(startedAt)}s.`);
|
||||
return container;
|
||||
},
|
||||
(error) => {
|
||||
clearInterval(heartbeat);
|
||||
throw error;
|
||||
},
|
||||
);
|
||||
};
|
||||
|
||||
try {
|
||||
const [postgres, inbucket, webhook, ...additionalContainers] = await Promise.all([
|
||||
trackAndLog('postgres', startPostgresContainer(network)),
|
||||
trackAndLog('inbucket', startInbucketContainer(network)),
|
||||
startWebhook ? trackAndLog('webhook', startWebhookContainer(network)) : Promise.resolve(undefined),
|
||||
...additionalNames.map((name) => trackAndLog(name, ADDITIONAL_SERVICE_STARTERS[name](network))),
|
||||
]);
|
||||
|
||||
const additional: Partial<Record<TestContainersServiceName, StartedTestContainer>> = {};
|
||||
additionalNames.forEach((name, index) => {
|
||||
additional[name] = additionalContainers[index];
|
||||
});
|
||||
|
||||
const mattermost = await trackAndLog('server', startMattermostContainer(network.getName()));
|
||||
|
||||
started = {network, postgres, inbucket, webhook, mattermost, additional};
|
||||
} catch (error) {
|
||||
await Promise.allSettled(startedContainers.map((container) => container.stop()));
|
||||
await stopNetwork();
|
||||
throw error;
|
||||
}
|
||||
|
||||
applyResolvedConfig(started);
|
||||
resetEnvFile('initial boot');
|
||||
|
||||
logStackStarted(started);
|
||||
}
|
||||
|
||||
/**
|
||||
* True if .env.testcontainers points at a Mattermost container that's still running — i.e. some
|
||||
* OTHER process already brought up a stack this process should reuse instead of duplicating.
|
||||
* Always a different process (the only channel between them is the env file, not runtime IPC);
|
||||
* covers PW_TESTCONTAINERS_REUSE=true across separate local invocations, and a CI dispatcher that
|
||||
* starts the server once per worker job and runs one spec per process against it.
|
||||
*
|
||||
* Deliberately does not gate on testConfig.testcontainersReuse: that flag governs whether the
|
||||
* OWNING process leaves the stack running on its own exit — a different decision from whether
|
||||
* THIS process should reuse a stack it finds already alive. Reuse always applies once liveness
|
||||
* is confirmed.
|
||||
*/
|
||||
async function reuseExistingStack(): Promise<boolean> {
|
||||
if (!testConfig.mattermostContainerId || !(await isContainerRunning(testConfig.mattermostContainerId))) {
|
||||
return false;
|
||||
}
|
||||
|
||||
reused = true;
|
||||
|
||||
if (testConfig.containerRunner) {
|
||||
await joinSelfToNetwork(testConfig.testcontainersNetworkName);
|
||||
}
|
||||
|
||||
logTestcontainers(
|
||||
'reusing already-running server (with PW_TESTCONTAINERS_REUSE=true), see .env.testcontainers for stack information',
|
||||
);
|
||||
logStackReused();
|
||||
return true;
|
||||
}
|
||||
|
||||
// Same shape of summary as logStackStarted(), but built from testConfig's resolved fields
|
||||
// instead of live StartedTestContainer handles — reusing never gets those (see `reused`'s
|
||||
// declaration above), only whatever an EARLIER process's startStack() persisted to
|
||||
// .env.testcontainers and this process's dotenv.config() read back into testConfig.
|
||||
function logStackReused(): void {
|
||||
const lines: string[] = [
|
||||
` - ${'server'.padEnd(13)} = ${testConfig.baseURL}`,
|
||||
` - ${'postgres'.padEnd(13)} = ${testConfig.postgresUrl}`,
|
||||
` - ${'inbucket'.padEnd(13)} = ${testConfig.smtpURL}`,
|
||||
];
|
||||
if (testConfig.webhookBaseUrl) {
|
||||
lines.push(` - ${'webhook'.padEnd(13)} = ${testConfig.webhookBaseUrl}`);
|
||||
}
|
||||
|
||||
const additionalUrls: Record<TestContainersServiceName, string> = {
|
||||
openldap: `${testConfig.ldapHost}:${testConfig.ldapPort}`,
|
||||
keycloak: testConfig.keycloakUrl,
|
||||
elasticsearch: testConfig.elasticsearchUrl,
|
||||
opensearch: testConfig.opensearchUrl,
|
||||
minio: testConfig.minioUrl,
|
||||
azurite: testConfig.azuriteUrl,
|
||||
};
|
||||
testConfig.testcontainersServices.forEach((name) => {
|
||||
lines.push(` - ${name.padEnd(13)} = ${additionalUrls[name]}`);
|
||||
});
|
||||
|
||||
// eslint-disable-next-line no-console
|
||||
console.log(
|
||||
`Testcontainers (reused, network ${testConfig.testcontainersNetworkName}, tear down with: "npm run testcontainers:down"):\n${lines.join('\n')}\n`,
|
||||
);
|
||||
}
|
||||
|
||||
async function isContainerRunning(containerId: string): Promise<boolean> {
|
||||
try {
|
||||
const {stdout} = await execFileAsync('docker', ['inspect', '-f', '{{.State.Running}}', containerId]);
|
||||
return stdout.trim() === 'true';
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Tears down the stack: always collects logs and removes the generated env file; only actually
|
||||
* stops containers when reuse isn't enabled (PW_TESTCONTAINERS_REUSE=true leaves them running
|
||||
* for the next invocation — local or a CI dispatcher's next spec — to reuse).
|
||||
*
|
||||
* A no-op beyond clearing the local flag when this process reused rather than created the stack:
|
||||
* it never owned the containers or the env file, so it must leave both exactly as it found them
|
||||
* for whichever process (or later dispatch) still depends on them.
|
||||
*/
|
||||
export async function stopStack(options: {force?: boolean} = {}): Promise<void> {
|
||||
if (!testConfig.useTestContainers) {
|
||||
return;
|
||||
}
|
||||
|
||||
if (reused) {
|
||||
reused = false;
|
||||
logTestcontainers('this process reused an existing server — leaving it untouched.');
|
||||
return;
|
||||
}
|
||||
|
||||
if (!started) {
|
||||
return;
|
||||
}
|
||||
|
||||
const stack = started;
|
||||
await collectLogs(stack);
|
||||
|
||||
const shouldStop = options.force || !testConfig.testcontainersReuse;
|
||||
if (shouldStop) {
|
||||
await Promise.allSettled([
|
||||
stack.mattermost.stop(),
|
||||
stack.inbucket.stop(),
|
||||
stack.webhook?.stop(),
|
||||
stack.postgres.stop(),
|
||||
...Object.values(stack.additional).map((container) => container?.stop()),
|
||||
]);
|
||||
await stopNetwork();
|
||||
logStackStopped(stack);
|
||||
archiveEnvFile();
|
||||
removeEnvFile();
|
||||
} else {
|
||||
logStackLeftRunning(stack);
|
||||
}
|
||||
|
||||
started = undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* True if every key in `env` already has the given value in testConfig.bootEnvOverrides — i.e.
|
||||
* the currently-running Mattermost container was already booted this way, so a pw.ensure*() can
|
||||
* skip restartMattermostContainer() for these settings.
|
||||
*/
|
||||
export function bootEnvMatches(env: Record<string, string>): boolean {
|
||||
return Object.entries(env).every(([key, value]) => testConfig.bootEnvOverrides[key] === value);
|
||||
}
|
||||
|
||||
/**
|
||||
* Stops the current Mattermost container and starts a fresh one with additional env merged in —
|
||||
* used for settings like FileSettings.DriverName, ElasticsearchSettings.Backend, or
|
||||
* MM_FEATUREFLAGS_* which are never re-read from a running server, so patchConfig alone can't
|
||||
* change them.
|
||||
*
|
||||
* Safe to call from a Playwright worker process, unlike startStack()/stopStack(): it works from
|
||||
* testConfig's container id and network name rather than the in-process StartedStack, since
|
||||
* global setup (which owns that in-process state) and worker processes are different OS
|
||||
* processes.
|
||||
*
|
||||
* `env` is merged into testConfig.bootEnvOverrides (not replaced) so an earlier pw.ensure*()
|
||||
* call's settings survive a later, unrelated one restarting the same container again. The merged
|
||||
* result, new container id, and new baseURL are appended to .env.testcontainers so any other
|
||||
* process picks up the real current state instead of a stale or default one.
|
||||
*
|
||||
* Always performs the restart without checking whether `env` is already active — callers (e.g.
|
||||
* ensureMinio()/ensureAzurite()) own that decision via bootEnvMatches().
|
||||
*/
|
||||
export async function restartMattermostContainer(env: Record<string, string>): Promise<void> {
|
||||
if (!testConfig.useTestContainers) {
|
||||
throw new Error('restartMattermostContainer requires PW_USE_TESTCONTAINERS=true.');
|
||||
}
|
||||
if (!testConfig.mattermostContainerId || !testConfig.testcontainersNetworkName) {
|
||||
throw new Error(
|
||||
'No running Testcontainers stack to restart (missing Mattermost container id or network name).',
|
||||
);
|
||||
}
|
||||
|
||||
extendTimeoutForRestart();
|
||||
|
||||
testConfig.bootEnvOverrides = {...testConfig.bootEnvOverrides, ...env};
|
||||
|
||||
await execFileAsync('docker', ['rm', '-f', testConfig.mattermostContainerId]);
|
||||
|
||||
const mattermost = await startMattermostContainer(
|
||||
testConfig.testcontainersNetworkName,
|
||||
testConfig.bootEnvOverrides,
|
||||
);
|
||||
|
||||
testConfig.baseURL = resolveUrl(mattermost, MATTERMOST_PORT, MATTERMOST_ALIAS);
|
||||
testConfig.mattermostContainerId = mattermost.getId();
|
||||
clearClientCache();
|
||||
|
||||
appendEnvFile(`restart requested by ${describeCurrentTest()} — env ${JSON.stringify(env)}`);
|
||||
|
||||
logTestcontainers(`restarted server with ${JSON.stringify(env)}.`);
|
||||
}
|
||||
|
||||
// Identifies whichever spec/test is currently driving a restart, so .env.testcontainers's history
|
||||
// shows why the server ended up in its current state. restartMattermostContainer() is always
|
||||
// called from inside a running test, so test.info() should resolve; the fallback only guards a
|
||||
// future caller that isn't.
|
||||
function describeCurrentTest(): string {
|
||||
try {
|
||||
const info = test.info();
|
||||
return `${path.relative(process.cwd(), info.file)} > ${info.title}`;
|
||||
} catch {
|
||||
return 'unknown caller (not running inside a test)';
|
||||
}
|
||||
}
|
||||
|
||||
// startMattermostContainer()'s wait strategy blocks on a scheduler log line whose first tick is
|
||||
// deliberately delayed 60s after startup (see that function's comment), so a restart alone can
|
||||
// exceed the suite's default 60s test timeout before the test has done any of its own work.
|
||||
// Playwright's timeout wouldn't cancel the still-in-flight restart, so a timed-out retry can race
|
||||
// it and hit a container mid-swap. Raising (not just extending) the timeout avoids ratcheting it
|
||||
// down if a later restart in the same test calls this again after some budget is already spent.
|
||||
function extendTimeoutForRestart(): void {
|
||||
try {
|
||||
const info = test.info();
|
||||
info.setTimeout(Math.max(info.timeout, duration.four_min));
|
||||
} catch {
|
||||
// Not running inside a test (e.g. called from a script) — nothing to extend.
|
||||
}
|
||||
}
|
||||
|
||||
// containerRunner mode: join the calling `playwright` container to the same network so its own
|
||||
// connections can use aliases too, instead of mapped ports. Relies on Docker setting the
|
||||
// container's hostname to its own container ID by default, and on the `docker` CLI being
|
||||
// present alongside the mounted socket.
|
||||
//
|
||||
// Takes a network name/ID string rather than a StartedNetwork object: reuseExistingStack() only
|
||||
// has testConfig.testcontainersNetworkName (read from .env.testcontainers) to work with, not a
|
||||
// live handle — and the docker CLI resolves either form the same way, so the freshly-created path
|
||||
// below just passes network.getId() instead.
|
||||
async function joinSelfToNetwork(networkId: string): Promise<void> {
|
||||
const selfContainerId = os.hostname();
|
||||
try {
|
||||
await execFileAsync('docker', ['network', 'connect', networkId, selfContainerId]);
|
||||
} catch (error) {
|
||||
// A CI dispatcher running one spec per process reuses the same stack (and this same
|
||||
// runner container) on every invocation, so this join is attempted again every time —
|
||||
// already-connected isn't a failure, it's the expected steady state after the first.
|
||||
if (String(error).includes('already exists in network')) {
|
||||
return;
|
||||
}
|
||||
throw new Error(
|
||||
'containerRunner mode (PW_TESTCONTAINERS_CONTAINER_RUNNER=true) requires the calling container ' +
|
||||
'to join the Testcontainers network, but "docker network connect" failed for container ' +
|
||||
`"${selfContainerId}": ${String(error)}. Ensure /var/run/docker.sock is mounted and the docker ` +
|
||||
'CLI is installed in this image.',
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
function resolveUrl(container: StartedTestContainer, port: number, alias: string): string {
|
||||
if (testConfig.containerRunner) {
|
||||
return `http://${alias}:${port}`;
|
||||
}
|
||||
return `http://${container.getHost()}:${container.getMappedPort(port)}`;
|
||||
}
|
||||
|
||||
function resolveHostAndPort(container: StartedTestContainer, port: number, alias: string): [string, number] {
|
||||
if (testConfig.containerRunner) {
|
||||
return [alias, port];
|
||||
}
|
||||
return [container.getHost(), container.getMappedPort(port)];
|
||||
}
|
||||
|
||||
// Same containerRunner-aware resolution as resolveUrl()/resolveHostAndPort() above: the alias in
|
||||
// containerRunner mode (the test process is on the Testcontainers network), the host-mapped port
|
||||
// otherwise — direct-DB specs are just another client connecting from wherever the test process
|
||||
// runs.
|
||||
function resolvePostgresUrl(postgres: StartedPostgreSqlContainer): string {
|
||||
const [host, port] = resolveHostAndPort(postgres, POSTGRES_PORT, POSTGRES_ALIAS);
|
||||
return `postgres://${POSTGRES_USER}:${POSTGRES_PASSWORD}@${host}:${port}/${POSTGRES_DB}?sslmode=disable&connect_timeout=10&binary_parameters=yes`;
|
||||
}
|
||||
|
||||
type ContainerMetadata = {alias: string; port: number; image: string};
|
||||
|
||||
const ADDITIONAL_CONTAINER_METADATA: Record<TestContainersServiceName, ContainerMetadata> = {
|
||||
openldap: {alias: OPENLDAP_ALIAS, port: OPENLDAP_PORT, image: OPENLDAP_IMAGE},
|
||||
keycloak: {alias: KEYCLOAK_ALIAS, port: KEYCLOAK_PORT, image: KEYCLOAK_IMAGE},
|
||||
elasticsearch: {
|
||||
alias: ELASTICSEARCH_ALIAS,
|
||||
port: ELASTICSEARCH_PORT,
|
||||
image: `built, Elasticsearch ${ELASTICSEARCH_VERSION}`,
|
||||
},
|
||||
opensearch: {alias: OPENSEARCH_ALIAS, port: OPENSEARCH_PORT, image: `built, OpenSearch ${OPENSEARCH_VERSION}`},
|
||||
minio: {alias: MINIO_ALIAS, port: MINIO_PORT, image: MINIO_IMAGE},
|
||||
azurite: {alias: AZURITE_ALIAS, port: AZURITE_BLOB_PORT, image: AZURITE_IMAGE},
|
||||
};
|
||||
|
||||
function containerEntries(stack: StartedStack): Array<[string, StartedTestContainer, ContainerMetadata]> {
|
||||
const base: Array<[string, StartedTestContainer, ContainerMetadata]> = [
|
||||
['server', stack.mattermost, {alias: MATTERMOST_ALIAS, port: MATTERMOST_PORT, image: testConfig.serverImage}],
|
||||
['postgres', stack.postgres, {alias: POSTGRES_ALIAS, port: POSTGRES_PORT, image: POSTGRES_IMAGE}],
|
||||
['inbucket', stack.inbucket, {alias: INBUCKET_ALIAS, port: INBUCKET_WEB_PORT, image: INBUCKET_IMAGE}],
|
||||
];
|
||||
if (stack.webhook) {
|
||||
base.push([
|
||||
'webhook',
|
||||
stack.webhook,
|
||||
{alias: WEBHOOK_ALIAS, port: WEBHOOK_PORT, image: 'built, webhook sidecar'},
|
||||
]);
|
||||
}
|
||||
|
||||
const additional = Object.entries(stack.additional)
|
||||
.filter((entry): entry is [TestContainersServiceName, StartedTestContainer] => entry[1] !== undefined)
|
||||
.map((entry): [string, StartedTestContainer, ContainerMetadata] => [
|
||||
entry[0],
|
||||
entry[1],
|
||||
ADDITIONAL_CONTAINER_METADATA[entry[0]],
|
||||
]);
|
||||
|
||||
return [...base, ...additional];
|
||||
}
|
||||
|
||||
function formatContainerLine(name: string, container: StartedTestContainer, metadata: ContainerMetadata): string {
|
||||
const host = `${container.getHost()}:${container.getMappedPort(metadata.port)}`;
|
||||
return ` - ${name.padEnd(13)} = ${metadata.image} (network: ${metadata.alias}:${metadata.port}, host: ${host})`;
|
||||
}
|
||||
|
||||
function elapsedSeconds(startedAt: number): string {
|
||||
return ((Date.now() - startedAt) / 1000).toFixed(1);
|
||||
}
|
||||
|
||||
// `master`/`release-*` tags get rebuilt continuously, so a cached copy can silently go stale;
|
||||
// pinned version tags (e.g. `:11.10.0`) never change, so they're excluded.
|
||||
const MUTABLE_IMAGE_TAG_PATTERN = /:(master|release-.+)$/;
|
||||
|
||||
async function logServerImageAge(image: string): Promise<void> {
|
||||
let created: Date;
|
||||
try {
|
||||
const {stdout} = await execFileAsync('docker', ['image', 'inspect', image, '--format', '{{.Created}}']);
|
||||
created = new Date(stdout.trim());
|
||||
} catch {
|
||||
// Not cached locally — Testcontainers will pull it fresh as part of starting the
|
||||
// container, so whatever comes up is already the latest build. Nothing to warn about.
|
||||
logTestcontainers(`server image "${image}" isn't cached locally yet — will pull the latest build.`);
|
||||
return;
|
||||
}
|
||||
|
||||
const ageHours = (Date.now() - created.getTime()) / (60 * 60 * 1000);
|
||||
const age = ageHours >= 48 ? `${(ageHours / 24).toFixed()}d` : `${ageHours.toFixed()}h`;
|
||||
const looksStale = MUTABLE_IMAGE_TAG_PATTERN.test(image) && ageHours > 24;
|
||||
|
||||
logTestcontainers(
|
||||
`server image "${image}" (built ${created.toISOString()}, ${age} ago).` +
|
||||
(looksStale
|
||||
? ` This is a moving tag and the cached copy may be outdated — run "docker pull ${image}" for the latest build.`
|
||||
: ''),
|
||||
);
|
||||
}
|
||||
|
||||
function logStackStarted(stack: StartedStack): void {
|
||||
const lines = containerEntries(stack).map(([name, container, metadata]) =>
|
||||
formatContainerLine(name, container, metadata),
|
||||
);
|
||||
// eslint-disable-next-line no-console
|
||||
console.log(`Testcontainers (network ${stack.network.getId()}):\n${lines.join('\n')}`);
|
||||
}
|
||||
|
||||
function logStackStopped(stack: StartedStack): void {
|
||||
const names = containerEntries(stack).map(([name]) => name);
|
||||
logTestcontainers(`stopped ${names.join(', ')}.`);
|
||||
}
|
||||
|
||||
function logStackLeftRunning(stack: StartedStack): void {
|
||||
const serverUrl = resolveUrl(stack.mattermost, MATTERMOST_PORT, MATTERMOST_ALIAS);
|
||||
logTestcontainers(
|
||||
`left running (PW_TESTCONTAINERS_REUSE=true) — server reachable at ${serverUrl}; may tear down with: "npm run testcontainers:down"`,
|
||||
);
|
||||
}
|
||||
|
||||
// Mutates the testConfig singleton in place so the rest of globalSetup (same process) sees the
|
||||
// real resolved values immediately — the generated env file (resetEnvFile) is what hands these
|
||||
// same values to worker processes.
|
||||
function applyResolvedConfig(stack: StartedStack): void {
|
||||
testConfig.baseURL = resolveUrl(stack.mattermost, MATTERMOST_PORT, MATTERMOST_ALIAS);
|
||||
testConfig.smtpURL = resolveUrl(stack.inbucket, INBUCKET_WEB_PORT, INBUCKET_ALIAS);
|
||||
testConfig.postgresUrl = resolvePostgresUrl(stack.postgres);
|
||||
if (stack.webhook) {
|
||||
testConfig.webhookBaseUrl = resolveUrl(stack.webhook, WEBHOOK_PORT, WEBHOOK_ALIAS);
|
||||
}
|
||||
testConfig.testcontainersNetworkName = stack.network.getName();
|
||||
testConfig.mattermostContainerId = stack.mattermost.getId();
|
||||
|
||||
if (stack.additional.openldap) {
|
||||
const [host, port] = resolveHostAndPort(stack.additional.openldap, OPENLDAP_PORT, OPENLDAP_ALIAS);
|
||||
testConfig.ldapHost = host;
|
||||
testConfig.ldapPort = port;
|
||||
}
|
||||
if (stack.additional.keycloak) {
|
||||
testConfig.keycloakUrl = resolveUrl(stack.additional.keycloak, KEYCLOAK_PORT, KEYCLOAK_ALIAS);
|
||||
}
|
||||
if (stack.additional.elasticsearch) {
|
||||
testConfig.elasticsearchUrl = resolveUrl(
|
||||
stack.additional.elasticsearch,
|
||||
ELASTICSEARCH_PORT,
|
||||
ELASTICSEARCH_ALIAS,
|
||||
);
|
||||
}
|
||||
if (stack.additional.opensearch) {
|
||||
testConfig.opensearchUrl = resolveUrl(stack.additional.opensearch, OPENSEARCH_PORT, OPENSEARCH_ALIAS);
|
||||
}
|
||||
if (stack.additional.minio) {
|
||||
testConfig.minioUrl = resolveUrl(stack.additional.minio, MINIO_PORT, MINIO_ALIAS);
|
||||
}
|
||||
if (stack.additional.azurite) {
|
||||
testConfig.azuriteUrl = resolveUrl(stack.additional.azurite, AZURITE_BLOB_PORT, AZURITE_ALIAS);
|
||||
}
|
||||
}
|
||||
|
||||
// One block per write: a human-readable `# [timestamp] label` comment line (dotenv ignores
|
||||
// `#`-led lines) followed by the current resolved KEY=VALUE state, including bootEnvOverrides
|
||||
// JSON-encoded and single-quoted so its embedded double quotes/braces survive dotenv's parser.
|
||||
function envFileLines(label: string): string[] {
|
||||
// Omit unset optional URLs/hosts — otherwise dotenv would load the literal string
|
||||
// "undefined" (truthy) when webhook/sidecars are not started.
|
||||
const entries: Array<[string, unknown]> = [
|
||||
['PW_BASE_URL', testConfig.baseURL],
|
||||
['PW_SMTP_URL', testConfig.smtpURL],
|
||||
['PW_POSTGRES_URL', testConfig.postgresUrl],
|
||||
['PW_WEBHOOK_BASE_URL', testConfig.webhookBaseUrl],
|
||||
['PW_TESTCONTAINERS_NETWORK_GATEWAY_IP', testConfig.testcontainersNetworkGatewayIp],
|
||||
['PW_LDAP_HOST', testConfig.ldapHost],
|
||||
['PW_LDAP_PORT', testConfig.ldapPort],
|
||||
['PW_KEYCLOAK_URL', testConfig.keycloakUrl],
|
||||
['PW_ELASTICSEARCH_URL', testConfig.elasticsearchUrl],
|
||||
['PW_OPENSEARCH_URL', testConfig.opensearchUrl],
|
||||
['PW_MINIO_URL', testConfig.minioUrl],
|
||||
['PW_AZURITE_URL', testConfig.azuriteUrl],
|
||||
['PW_TESTCONTAINERS_NETWORK_NAME', testConfig.testcontainersNetworkName],
|
||||
['PW_TESTCONTAINERS_MATTERMOST_CONTAINER_ID', testConfig.mattermostContainerId],
|
||||
];
|
||||
return [
|
||||
`# [${new Date().toISOString()}] ${label}`,
|
||||
...entries
|
||||
.filter(([, value]) => value !== undefined && value !== null && value !== '')
|
||||
.map(([key, value]) => `${key}=${value}`),
|
||||
`PW_TESTCONTAINERS_BOOT_ENV='${JSON.stringify(testConfig.bootEnvOverrides)}'`,
|
||||
'',
|
||||
];
|
||||
}
|
||||
|
||||
// (Re)creates .env.testcontainers from scratch — only called once, when a brand new stack boots,
|
||||
// so a leftover file from an earlier (now-dead) stack never bleeds into this one.
|
||||
function resetEnvFile(label: string): void {
|
||||
fs.writeFileSync(ENV_FILE_PATH, envFileLines(label).join('\n') + '\n', 'utf-8');
|
||||
}
|
||||
|
||||
/**
|
||||
* Appends a new snapshot block instead of overwriting — dotenv resolves the correct current value
|
||||
* per key when a fresh process parses the file (later occurrences win), while the file as a whole
|
||||
* becomes a chronological log of every restart: which spec/test triggered it, what env diff was
|
||||
* requested, and the full resolved state right after. That's what's needed to investigate
|
||||
* server-state drift after the fact, since in the CI dispatch model no single process ever sees
|
||||
* the whole picture on its own.
|
||||
*/
|
||||
function appendEnvFile(label: string): void {
|
||||
fs.appendFileSync(ENV_FILE_PATH, envFileLines(label).join('\n') + '\n', 'utf-8');
|
||||
}
|
||||
|
||||
// Preserves the full restart history as a debug artifact before it's deleted — logs/ is already
|
||||
// what CI's upload-debug-artifacts step picks up, so this needs no separate wiring.
|
||||
function archiveEnvFile(): void {
|
||||
if (!fs.existsSync(ENV_FILE_PATH)) {
|
||||
return;
|
||||
}
|
||||
fs.mkdirSync(LOG_DIR, {recursive: true});
|
||||
fs.copyFileSync(ENV_FILE_PATH, path.join(LOG_DIR, 'testcontainers_env_history.log'));
|
||||
}
|
||||
|
||||
function removeEnvFile(): void {
|
||||
if (fs.existsSync(ENV_FILE_PATH)) {
|
||||
fs.rmSync(ENV_FILE_PATH);
|
||||
}
|
||||
}
|
||||
|
||||
async function collectLogs(stack: StartedStack): Promise<void> {
|
||||
fs.mkdirSync(LOG_DIR, {recursive: true});
|
||||
|
||||
const targets: Array<[string, StartedTestContainer]> = [
|
||||
['mattermost', stack.mattermost],
|
||||
['postgres', stack.postgres],
|
||||
['inbucket', stack.inbucket],
|
||||
...(stack.webhook ? [['webhook', stack.webhook] as [string, StartedTestContainer]] : []),
|
||||
...Object.entries(stack.additional).filter(
|
||||
(entry): entry is [string, StartedTestContainer] => entry[1] !== undefined,
|
||||
),
|
||||
];
|
||||
|
||||
await Promise.allSettled(
|
||||
targets.map(async ([name, container]) => {
|
||||
const logStream = await container.logs();
|
||||
const outFile = fs.createWriteStream(path.join(LOG_DIR, `${name}.log`));
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
logStream.pipe(outFile);
|
||||
// Don't let a stalled log stream hold up teardown indefinitely.
|
||||
const timer = setTimeout(() => {
|
||||
outFile.end();
|
||||
resolve();
|
||||
}, 10_000);
|
||||
logStream.on('end', () => {
|
||||
clearTimeout(timer);
|
||||
resolve();
|
||||
});
|
||||
logStream.on('error', (error) => {
|
||||
clearTimeout(timer);
|
||||
reject(error);
|
||||
});
|
||||
});
|
||||
}),
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,38 @@
|
||||
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
||||
// See LICENSE.txt for license information.
|
||||
|
||||
import {GenericContainer, Wait} from 'testcontainers';
|
||||
import type {StartedNetwork, StartedTestContainer} from 'testcontainers';
|
||||
|
||||
import {TESTCONTAINERS_LABELS, WEBHOOK_ALIAS, WEBHOOK_PORT} from './constants';
|
||||
import {containerAssetPath} from './paths';
|
||||
import {startWithRetry} from './retry';
|
||||
|
||||
import {testConfig} from '@/test_config';
|
||||
|
||||
// Vendored from e2e-tests/cypress/webhook_serve.js — the interactive-message/dialog callback
|
||||
// sidecar tests point PW_WEBHOOK_BASE_URL at. Always started, unlike the other optional services,
|
||||
// since testConfig.webhookBaseUrl defaults to localhost:3000 outside Testcontainers mode too.
|
||||
export async function startWebhookContainer(network: StartedNetwork): Promise<StartedTestContainer> {
|
||||
return startWithRetry('webhook', async () => {
|
||||
// deleteOnExit: false — otherwise the built image bakes in this session's Ryuk id, so Ryuk
|
||||
// reaps the "reused" container the moment this session ends, defeating withReuse() below.
|
||||
const image = await GenericContainer.fromDockerfile(containerAssetPath('webhook'), 'Dockerfile.webhook').build(
|
||||
undefined,
|
||||
{deleteOnExit: false},
|
||||
);
|
||||
|
||||
let builder = image
|
||||
.withExposedPorts(WEBHOOK_PORT)
|
||||
.withNetwork(network)
|
||||
.withNetworkAliases(WEBHOOK_ALIAS)
|
||||
.withLabels(TESTCONTAINERS_LABELS)
|
||||
.withWaitStrategy(Wait.forHttp('/', WEBHOOK_PORT).forStatusCode(200));
|
||||
|
||||
if (testConfig.testcontainersReuse) {
|
||||
builder = builder.withReuse();
|
||||
}
|
||||
|
||||
return builder.start();
|
||||
});
|
||||
}
|
||||
@@ -2,15 +2,47 @@
|
||||
// See LICENSE.txt for license information.
|
||||
|
||||
export {test, expect, PlaywrightExtended} from './test_fixture';
|
||||
export {testConfig} from './test_config';
|
||||
export type {ExtendedFixtures} from './test_fixture';
|
||||
export {testConfig, resolveAppUrl, TESTCONTAINERS_SERVICE_NAMES} from './test_config';
|
||||
export type {TestContainersServiceName} from './test_config';
|
||||
export {baseGlobalSetup} from './global_setup';
|
||||
export {TestBrowser} from './browser_context';
|
||||
export {bindPageToLiveBaseURL, TestBrowser} from './browser_context';
|
||||
export {getBlobFromAsset, getFileFromAsset} from './file';
|
||||
export {decomposeKorean, koreanTestPhrase, typeHangulCharacterWithIme, typeHangulWithIme} from './ime';
|
||||
export {duration, getRandomId, wait, newTestPassword} from './util';
|
||||
export {LicenseSkus, appsPluginId, callsPluginId, playbooksPluginId} from './constant';
|
||||
|
||||
export {getAdminClient, mergeWithOnPremServerConfig, getOnPremServerConfig} from './server';
|
||||
export {
|
||||
getAdminClient,
|
||||
mergeWithOnPremServerConfig,
|
||||
getOnPremServerConfig,
|
||||
generateLdapUser,
|
||||
createLdapUser,
|
||||
updateLdapUser,
|
||||
deleteLdapUser,
|
||||
ldapServerConfig,
|
||||
ensureOpenldap,
|
||||
createKeycloakUser,
|
||||
deleteKeycloakUser,
|
||||
listMinioObjectKeys,
|
||||
ensureMinio,
|
||||
samlServerConfig,
|
||||
ensureKeycloak,
|
||||
elasticsearchServerConfig,
|
||||
opensearchServerConfig,
|
||||
ensureElasticsearch,
|
||||
ensureOpensearch,
|
||||
ensureAzurite,
|
||||
listAzuriteBlobNames,
|
||||
ensureLocalFile,
|
||||
ensurePostgresSearch,
|
||||
ensureFeatureFlag,
|
||||
runMmctl,
|
||||
ensureMmctl,
|
||||
} from './server';
|
||||
export type {LdapUser, KeycloakUser, MmctlResult} from './server';
|
||||
|
||||
export {startStack, stopStack} from './containers';
|
||||
|
||||
export {
|
||||
ChannelsPage,
|
||||
|
||||
@@ -0,0 +1,94 @@
|
||||
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
||||
// See LICENSE.txt for license information.
|
||||
|
||||
import {BlobServiceClient, StorageSharedKeyCredential} from '@azure/storage-blob';
|
||||
import {test} from '@playwright/test';
|
||||
|
||||
import {
|
||||
AZURITE_ACCOUNT_KEY,
|
||||
AZURITE_ACCOUNT_NAME,
|
||||
AZURITE_ALIAS,
|
||||
AZURITE_BLOB_PORT,
|
||||
AZURITE_CONTAINER,
|
||||
} from '../containers/constants';
|
||||
import {bootEnvMatches, restartMattermostContainer} from '../containers/stack';
|
||||
|
||||
import {uploadProbeImage} from './filestore';
|
||||
import {getAdminClient} from './init';
|
||||
|
||||
import {testConfig} from '@/test_config';
|
||||
|
||||
function getBlobServiceClient(): BlobServiceClient {
|
||||
const credential = new StorageSharedKeyCredential(AZURITE_ACCOUNT_NAME, AZURITE_ACCOUNT_KEY);
|
||||
return new BlobServiceClient(`${testConfig.azuriteUrl}/${AZURITE_ACCOUNT_NAME}`, credential);
|
||||
}
|
||||
|
||||
// Only used internally by ensureAzurite() — not a spec-facing entry point.
|
||||
async function ensureAzuriteContainer(container: string = AZURITE_CONTAINER): Promise<void> {
|
||||
const client = getBlobServiceClient().getContainerClient(container);
|
||||
const exists = await client.exists();
|
||||
if (!exists) {
|
||||
await client.create();
|
||||
}
|
||||
}
|
||||
|
||||
/** Lists every blob name in the container, to confirm the server actually wrote to Azurite. */
|
||||
export async function listAzuriteBlobNames(container: string = AZURITE_CONTAINER): Promise<string[]> {
|
||||
const client = getBlobServiceClient().getContainerClient(container);
|
||||
const names: string[] = [];
|
||||
for await (const blob of client.listBlobsFlat()) {
|
||||
names.push(blob.name);
|
||||
}
|
||||
return names;
|
||||
}
|
||||
|
||||
// FileSettings' backend is chosen once when the Mattermost process boots and never re-read from a
|
||||
// running server's config, so pointing the server at Azurite can only happen via the env vars the
|
||||
// container starts with — always via the network alias, since the server itself always runs
|
||||
// inside the Testcontainers network. AzureEndpoint under the "custom" cloud is the full service
|
||||
// URL (path-style, account name included), unlike the vhost-style URLs real Azure uses.
|
||||
function azuriteServerEnv(): Record<string, string> {
|
||||
return {
|
||||
MM_FILESETTINGS_DRIVERNAME: 'azureblob',
|
||||
MM_FILESETTINGS_AZURESTORAGEACCOUNT: AZURITE_ACCOUNT_NAME,
|
||||
MM_FILESETTINGS_AZUREAUTHMODE: 'shared_key',
|
||||
MM_FILESETTINGS_AZUREACCESSKEY: AZURITE_ACCOUNT_KEY,
|
||||
MM_FILESETTINGS_AZURECONTAINER: AZURITE_CONTAINER,
|
||||
MM_FILESETTINGS_AZURECLOUD: 'custom',
|
||||
MM_FILESETTINGS_AZUREENDPOINT: `http://${AZURITE_ALIAS}:${AZURITE_BLOB_PORT}/${AZURITE_ACCOUNT_NAME}`,
|
||||
MM_FILESETTINGS_AZURESSL: 'false',
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Checks Azurite was started this run, restarts the server onto it if it isn't already the active
|
||||
* file storage backend, and confirms a real upload actually lands in Azurite — skipping the test
|
||||
* otherwise, instead of failing on an unmet precondition.
|
||||
*/
|
||||
export async function ensureAzurite(): Promise<void> {
|
||||
if (!testConfig.testcontainersServices.includes('azurite')) {
|
||||
test.skip(true, 'Skipping test - azurite not started (set PW_TESTCONTAINERS_SERVICES=azurite)');
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
await ensureAzuriteContainer();
|
||||
const env = azuriteServerEnv();
|
||||
if (!bootEnvMatches(env)) {
|
||||
await restartMattermostContainer(env);
|
||||
}
|
||||
|
||||
const {adminClient, adminUser} = await getAdminClient();
|
||||
await uploadProbeImage(adminClient, adminUser);
|
||||
|
||||
const blobNames = await listAzuriteBlobNames();
|
||||
if (blobNames.length === 0) {
|
||||
throw new Error(
|
||||
'Azurite container is still empty after a real upload — the server is not actually using ' +
|
||||
'Azurite as its file backend.',
|
||||
);
|
||||
}
|
||||
} catch (error) {
|
||||
test.skip(true, `Skipping test - Azurite connection test failed: ${String(error)}`);
|
||||
}
|
||||
}
|
||||
@@ -9,6 +9,17 @@ import {testConfig} from '@/test_config';
|
||||
// Variable to hold cache
|
||||
const clients: Record<string, ClientCache> = {};
|
||||
|
||||
/**
|
||||
* Drops every cached client, so the next makeClient()/getAdminClient() call logs in again instead
|
||||
* of reusing a session pointed at a Mattermost container that no longer exists — needed after
|
||||
* restartMattermostContainer() swaps in a fresh container (new base URL, new session).
|
||||
*/
|
||||
export function clearClientCache(): void {
|
||||
for (const key of Object.keys(clients)) {
|
||||
delete clients[key];
|
||||
}
|
||||
}
|
||||
|
||||
export async function makeClient(
|
||||
userRequest?: UserRequest,
|
||||
opts: {useCache?: boolean; skipLog?: boolean} = {useCache: true, skipLog: false},
|
||||
|
||||
@@ -72,7 +72,12 @@ const onPremServerConfig = (): Partial<TestAdminConfig> => {
|
||||
},
|
||||
},
|
||||
ServiceSettings: {
|
||||
SiteURL: testConfig.baseURL,
|
||||
// SiteURL is the server's own view of itself (e.g. for building plugin callback
|
||||
// URLs), so it must use an address the server can reach itself with. In `testcontainers` mode
|
||||
// testConfig.baseURL is a host-mapped port the server's own container can't reach;
|
||||
// internalBaseURL is the Docker network alias there, and the same as baseURL in
|
||||
// `external` mode — correct in both cases.
|
||||
SiteURL: testConfig.internalBaseURL,
|
||||
EnableOnboardingFlow: false,
|
||||
EnableSecurityFixAlert: false,
|
||||
GiphySdkKey: 's0glxvzVg9azvPipKxcPLpXV0q1x1fVP',
|
||||
|
||||
@@ -0,0 +1,52 @@
|
||||
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
||||
// See LICENSE.txt for license information.
|
||||
|
||||
import {test} from '@playwright/test';
|
||||
import type {AdminConfig} from '@mattermost/types/config';
|
||||
|
||||
import {ELASTICSEARCH_ALIAS, ELASTICSEARCH_PORT} from '../containers/constants';
|
||||
import {bootEnvMatches, restartMattermostContainer} from '../containers/stack';
|
||||
|
||||
import {getAdminClient} from './init';
|
||||
|
||||
import {testConfig} from '@/test_config';
|
||||
|
||||
// The Mattermost server always connects to Elasticsearch itself (indexing/search requests), so it
|
||||
// needs the Testcontainers network alias, not a host-mapped address.
|
||||
export function elasticsearchServerConfig(): Partial<AdminConfig['ElasticsearchSettings']> {
|
||||
return {
|
||||
ConnectionURL: `http://${ELASTICSEARCH_ALIAS}:${ELASTICSEARCH_PORT}`,
|
||||
EnableIndexing: true,
|
||||
EnableSearching: true,
|
||||
EnableAutocomplete: true,
|
||||
Sniff: false,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Checks Elasticsearch was started this run, restarts the server onto it if the Elasticsearch Go
|
||||
* client isn't already the registered search engine, and confirms the server can actually reach
|
||||
* it — skipping the test otherwise, instead of failing on an unmet precondition. Backend picks
|
||||
* which of two Go implementations (Elasticsearch vs OpenSearch client) the server registers — a
|
||||
* factory invoked once at startup, never re-invoked by the config-change watcher — so switching it
|
||||
* needs a restart, unlike the rest of ElasticsearchSettings, which the watcher does pick up live.
|
||||
*/
|
||||
export async function ensureElasticsearch(): Promise<void> {
|
||||
if (!testConfig.testcontainersServices.includes('elasticsearch')) {
|
||||
test.skip(true, 'Skipping test - elasticsearch not started (set PW_TESTCONTAINERS_SERVICES=elasticsearch)');
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
const env = {MM_ELASTICSEARCHSETTINGS_BACKEND: 'elasticsearch'};
|
||||
if (!bootEnvMatches(env)) {
|
||||
await restartMattermostContainer(env);
|
||||
}
|
||||
|
||||
const {adminClient} = await getAdminClient();
|
||||
await adminClient.patchConfig({ElasticsearchSettings: elasticsearchServerConfig()});
|
||||
await adminClient.testElasticsearch();
|
||||
} catch (error) {
|
||||
test.skip(true, `Skipping test - Elasticsearch connection test failed: ${String(error)}`);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,47 @@
|
||||
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
||||
// See LICENSE.txt for license information.
|
||||
|
||||
import {test} from '@playwright/test';
|
||||
|
||||
import {bootEnvMatches, restartMattermostContainer} from '../containers/stack';
|
||||
|
||||
import {getAdminClient} from './init';
|
||||
|
||||
import {testConfig} from '@/test_config';
|
||||
|
||||
/**
|
||||
* Restarts the server with the given feature flag set to `value` if it isn't already, and
|
||||
* confirms the running server actually reports that value — skipping the test otherwise, instead
|
||||
* of failing on an unmet precondition.
|
||||
*
|
||||
* FeatureFlags can't be changed via patchConfig on a running server at all: with no Split key
|
||||
* configured (this test setup never sets one), the config store's readOnlyFF handling reverts any
|
||||
* FeatureFlags patch back to its prior value before it's even persisted
|
||||
* (server/config/store.go's Set()/Load()), so only a boot-time MM_FEATUREFLAGS_* env var actually
|
||||
* takes effect.
|
||||
*/
|
||||
export async function ensureFeatureFlag(flagName: string, value: boolean): Promise<void> {
|
||||
if (!testConfig.useTestContainers) {
|
||||
test.skip(true, 'Skipping test - feature flag restart requires PW_USE_TESTCONTAINERS=true');
|
||||
return;
|
||||
}
|
||||
|
||||
const envKey = `MM_FEATUREFLAGS_${flagName.toUpperCase()}`;
|
||||
const envValue = String(value);
|
||||
|
||||
try {
|
||||
const env = {[envKey]: envValue};
|
||||
if (!bootEnvMatches(env)) {
|
||||
await restartMattermostContainer(env);
|
||||
}
|
||||
|
||||
const {adminClient} = await getAdminClient();
|
||||
const config = await adminClient.getConfig();
|
||||
const actual = config.FeatureFlags?.[flagName];
|
||||
if (String(actual) !== envValue) {
|
||||
throw new Error(`Feature flag "${flagName}" is "${String(actual)}" after restart, expected "${envValue}".`);
|
||||
}
|
||||
} catch (error) {
|
||||
test.skip(true, `Skipping test - feature flag "${flagName}" check failed: ${String(error)}`);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,50 @@
|
||||
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
||||
// See LICENSE.txt for license information.
|
||||
|
||||
import {test} from '@playwright/test';
|
||||
import type {Client4} from '@mattermost/client';
|
||||
import type {UserProfile} from '@mattermost/types/users';
|
||||
|
||||
import {bootEnvMatches, restartMattermostContainer} from '../containers/stack';
|
||||
|
||||
import {getAdminClient} from './init';
|
||||
|
||||
import {testConfig} from '@/test_config';
|
||||
|
||||
// A 1x1 transparent PNG, just to exercise a real write through the server's file backend.
|
||||
const PROBE_IMAGE_BASE64_PNG =
|
||||
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII=';
|
||||
|
||||
/** Uploads a throwaway profile image, exercising the server's real, live file backend. */
|
||||
export async function uploadProbeImage(adminClient: Client4, adminUser: UserProfile | null): Promise<void> {
|
||||
if (!adminUser) {
|
||||
throw new Error('No admin user available to probe a real upload with.');
|
||||
}
|
||||
const probeImage = new File([Buffer.from(PROBE_IMAGE_BASE64_PNG, 'base64')], 'probe.png', {type: 'image/png'});
|
||||
await adminClient.uploadProfileImage(adminUser.id, probeImage);
|
||||
}
|
||||
|
||||
/**
|
||||
* Restarts the server onto local disk storage if it isn't already there, and confirms a real
|
||||
* upload actually works — skipping the test otherwise, instead of failing on an unmet
|
||||
* precondition. The counterpart to ensureMinio()/ensureAzurite() for specs that specifically need
|
||||
* local storage active (e.g. after another spec in the same run switched it away).
|
||||
*/
|
||||
export async function ensureLocalFile(): Promise<void> {
|
||||
if (!testConfig.useTestContainers) {
|
||||
test.skip(true, 'Skipping test - local file storage restart requires PW_USE_TESTCONTAINERS=true');
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
const env = {MM_FILESETTINGS_DRIVERNAME: 'local'};
|
||||
if (!bootEnvMatches(env)) {
|
||||
await restartMattermostContainer(env);
|
||||
}
|
||||
|
||||
const {adminClient, adminUser} = await getAdminClient();
|
||||
await uploadProbeImage(adminClient, adminUser);
|
||||
} catch (error) {
|
||||
test.skip(true, `Skipping test - local file storage check failed: ${String(error)}`);
|
||||
}
|
||||
}
|
||||
@@ -34,3 +34,23 @@ export {
|
||||
updateUserAttributes,
|
||||
} from './abac_helpers';
|
||||
export {installAndEnablePlugin, isPluginActive, getPluginStatus} from './plugin';
|
||||
export {
|
||||
generateLdapUser,
|
||||
createLdapUser,
|
||||
updateLdapUser,
|
||||
deleteLdapUser,
|
||||
ldapServerConfig,
|
||||
ensureOpenldap,
|
||||
} from './openldap';
|
||||
export type {LdapUser} from './openldap';
|
||||
export {createKeycloakUser, deleteKeycloakUser, samlServerConfig, ensureKeycloak} from './keycloak';
|
||||
export type {KeycloakUser} from './keycloak';
|
||||
export {listMinioObjectKeys, ensureMinio} from './minio';
|
||||
export {elasticsearchServerConfig, ensureElasticsearch} from './elasticsearch';
|
||||
export {opensearchServerConfig, ensureOpensearch} from './opensearch';
|
||||
export {ensureAzurite, listAzuriteBlobNames} from './azurite';
|
||||
export {ensureLocalFile} from './filestore';
|
||||
export {ensurePostgresSearch} from './postgres_search';
|
||||
export {ensureFeatureFlag} from './feature_flags';
|
||||
export {runMmctl, ensureMmctl} from './mmctl';
|
||||
export type {MmctlResult} from './mmctl';
|
||||
|
||||
@@ -0,0 +1,151 @@
|
||||
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
||||
// See LICENSE.txt for license information.
|
||||
|
||||
import {test} from '@playwright/test';
|
||||
import type {Client4} from '@mattermost/client';
|
||||
import type {AdminConfig} from '@mattermost/types/config';
|
||||
|
||||
import {
|
||||
KEYCLOAK_ADMIN_PASSWORD,
|
||||
KEYCLOAK_ADMIN_USER,
|
||||
KEYCLOAK_ALIAS,
|
||||
KEYCLOAK_PORT,
|
||||
KEYCLOAK_REALM,
|
||||
} from '../containers/constants';
|
||||
|
||||
import {getAdminClient} from './init';
|
||||
|
||||
import {testConfig} from '@/test_config';
|
||||
|
||||
export type KeycloakUser = {
|
||||
username: string;
|
||||
email: string;
|
||||
firstName: string;
|
||||
lastName: string;
|
||||
password: string;
|
||||
};
|
||||
|
||||
async function getAdminToken(): Promise<string> {
|
||||
const response = await fetch(`${testConfig.keycloakUrl}/realms/master/protocol/openid-connect/token`, {
|
||||
method: 'POST',
|
||||
headers: {'Content-Type': 'application/x-www-form-urlencoded'},
|
||||
body: new URLSearchParams({
|
||||
grant_type: 'password',
|
||||
client_id: 'admin-cli',
|
||||
username: KEYCLOAK_ADMIN_USER,
|
||||
password: KEYCLOAK_ADMIN_PASSWORD,
|
||||
}),
|
||||
});
|
||||
if (!response.ok) {
|
||||
throw new Error(`Failed to get Keycloak admin token: ${response.status} ${await response.text()}`);
|
||||
}
|
||||
|
||||
const body = (await response.json()) as {access_token: string};
|
||||
return body.access_token;
|
||||
}
|
||||
|
||||
/** Creates the user in Keycloak and returns its Keycloak user id. */
|
||||
export async function createKeycloakUser(user: KeycloakUser): Promise<string> {
|
||||
const token = await getAdminToken();
|
||||
const response = await fetch(`${testConfig.keycloakUrl}/admin/realms/${KEYCLOAK_REALM}/users`, {
|
||||
method: 'POST',
|
||||
headers: {Authorization: `Bearer ${token}`, 'Content-Type': 'application/json'},
|
||||
body: JSON.stringify({
|
||||
username: user.username,
|
||||
email: user.email,
|
||||
firstName: user.firstName,
|
||||
lastName: user.lastName,
|
||||
enabled: true,
|
||||
credentials: [{type: 'password', value: user.password, temporary: false}],
|
||||
}),
|
||||
});
|
||||
if (!response.ok) {
|
||||
throw new Error(`Failed to create Keycloak user: ${response.status} ${await response.text()}`);
|
||||
}
|
||||
|
||||
const location = response.headers.get('Location');
|
||||
const userId = location?.split('/').pop();
|
||||
if (!userId) {
|
||||
throw new Error('Keycloak user creation response had no Location header to read the new user id from.');
|
||||
}
|
||||
return userId;
|
||||
}
|
||||
|
||||
export async function deleteKeycloakUser(userId: string): Promise<void> {
|
||||
const token = await getAdminToken();
|
||||
const response = await fetch(`${testConfig.keycloakUrl}/admin/realms/${KEYCLOAK_REALM}/users/${userId}`, {
|
||||
method: 'DELETE',
|
||||
headers: {Authorization: `Bearer ${token}`},
|
||||
});
|
||||
if (!response.ok && response.status !== 404) {
|
||||
throw new Error(`Failed to delete Keycloak user: ${response.status} ${await response.text()}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Matches the SAML client's clientId in keycloak-realm-export.json.
|
||||
const SAML_SERVICE_PROVIDER_ID = 'mattermost';
|
||||
|
||||
// The Mattermost server fetches this URL itself (via POST /saml/metadatafromidp), so it must be
|
||||
// reachable from inside the Testcontainers network — unlike the IdpURL/IdpDescriptorURL below,
|
||||
// which the browser follows directly and so must be reachable from the host instead. Only the
|
||||
// certificate from this response is used; its embedded URLs reflect the alias host, not the one
|
||||
// the browser needs.
|
||||
function keycloakSamlDescriptorUrl(): string {
|
||||
return `http://${KEYCLOAK_ALIAS}:${KEYCLOAK_PORT}/realms/${KEYCLOAK_REALM}/protocol/saml/descriptor`;
|
||||
}
|
||||
|
||||
// The metadata response's certificate is the raw base64 DER content straight out of the SAML
|
||||
// metadata XML's <X509Certificate> element — no PEM armor — but IdpCertificateFile parsing
|
||||
// requires a proper PEM block.
|
||||
function toPemCertificate(base64Der: string): string {
|
||||
const lines = base64Der.replace(/\s+/g, '').match(/.{1,64}/g) ?? [];
|
||||
return `-----BEGIN CERTIFICATE-----\n${lines.join('\n')}\n-----END CERTIFICATE-----\n`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Fetches Keycloak's SAML IdP certificate (via the server's own metadata-from-IdP call) and
|
||||
* uploads it, then returns a `SamlSettings` patch pointing the server at Keycloak's SAML IdP and
|
||||
* the users `createKeycloakUser` creates.
|
||||
*/
|
||||
export async function samlServerConfig(adminClient: Client4): Promise<Partial<AdminConfig['SamlSettings']>> {
|
||||
const metadata = await adminClient.getSamlMetadataFromIdp(keycloakSamlDescriptorUrl());
|
||||
const certificate = toPemCertificate(metadata.idp_public_certificate);
|
||||
await adminClient.uploadIdpSamlCertificate(new File([certificate], 'idp-certificate.crt'));
|
||||
|
||||
return {
|
||||
Enable: true,
|
||||
Verify: true,
|
||||
Encrypt: false,
|
||||
SignRequest: false,
|
||||
IdpURL: `${testConfig.keycloakUrl}/realms/${KEYCLOAK_REALM}/protocol/saml`,
|
||||
IdpDescriptorURL: `${testConfig.keycloakUrl}/realms/${KEYCLOAK_REALM}`,
|
||||
ServiceProviderIdentifier: SAML_SERVICE_PROVIDER_ID,
|
||||
AssertionConsumerServiceURL: `${testConfig.baseURL}/login/sso/saml`,
|
||||
IdAttribute: 'id',
|
||||
EmailAttribute: 'email',
|
||||
UsernameAttribute: 'username',
|
||||
FirstNameAttribute: 'givenName',
|
||||
LastNameAttribute: 'surname',
|
||||
LoginButtonText: 'Keycloak SAML',
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Checks Keycloak was started this run, points the server's SAML settings at it (fetching its IdP
|
||||
* metadata/certificate along the way), and skips the test if either step fails, instead of
|
||||
* failing on an unmet precondition.
|
||||
*/
|
||||
export async function ensureKeycloak(): Promise<void> {
|
||||
if (!testConfig.testcontainersServices.includes('keycloak')) {
|
||||
test.skip(true, 'Skipping test - keycloak not started (set PW_TESTCONTAINERS_SERVICES=keycloak)');
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
const {adminClient} = await getAdminClient();
|
||||
const config = await samlServerConfig(adminClient);
|
||||
await adminClient.patchConfig({SamlSettings: config});
|
||||
} catch (error) {
|
||||
test.skip(true, `Skipping test - Keycloak SAML setup failed: ${String(error)}`);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,93 @@
|
||||
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
||||
// See LICENSE.txt for license information.
|
||||
|
||||
import {Client as MinioClient} from 'minio';
|
||||
import {test} from '@playwright/test';
|
||||
|
||||
import {MINIO_ACCESS_KEY, MINIO_ALIAS, MINIO_BUCKET, MINIO_PORT, MINIO_SECRET_KEY} from '../containers/constants';
|
||||
import {bootEnvMatches, restartMattermostContainer} from '../containers/stack';
|
||||
|
||||
import {uploadProbeImage} from './filestore';
|
||||
import {getAdminClient} from './init';
|
||||
|
||||
import {testConfig} from '@/test_config';
|
||||
|
||||
function getMinioClient(): MinioClient {
|
||||
const url = new URL(testConfig.minioUrl);
|
||||
return new MinioClient({
|
||||
endPoint: url.hostname,
|
||||
port: Number(url.port),
|
||||
useSSL: url.protocol === 'https:',
|
||||
accessKey: MINIO_ACCESS_KEY,
|
||||
secretKey: MINIO_SECRET_KEY,
|
||||
});
|
||||
}
|
||||
|
||||
// Only used internally by ensureMinio() — not a spec-facing entry point.
|
||||
async function ensureMinioBucket(bucket: string = MINIO_BUCKET): Promise<void> {
|
||||
const client = getMinioClient();
|
||||
const exists = await client.bucketExists(bucket);
|
||||
if (!exists) {
|
||||
await client.makeBucket(bucket);
|
||||
}
|
||||
}
|
||||
|
||||
/** Lists every object key in the bucket, to confirm the server actually wrote to Minio. */
|
||||
export async function listMinioObjectKeys(bucket: string = MINIO_BUCKET): Promise<string[]> {
|
||||
const client = getMinioClient();
|
||||
const keys: string[] = [];
|
||||
for await (const item of client.listObjectsV2(bucket, '', true)) {
|
||||
if (item.name) {
|
||||
keys.push(item.name);
|
||||
}
|
||||
}
|
||||
return keys;
|
||||
}
|
||||
|
||||
// FileSettings' backend is chosen once when the Mattermost process boots and never re-read from a
|
||||
// running server's config, so pointing the server at Minio can only happen via the env vars the
|
||||
// container starts with — always via the network alias, since the server itself always runs
|
||||
// inside the Testcontainers network.
|
||||
function minioServerEnv(): Record<string, string> {
|
||||
return {
|
||||
MM_FILESETTINGS_DRIVERNAME: 'amazons3',
|
||||
MM_FILESETTINGS_AMAZONS3ENDPOINT: `${MINIO_ALIAS}:${MINIO_PORT}`,
|
||||
MM_FILESETTINGS_AMAZONS3ACCESSKEYID: MINIO_ACCESS_KEY,
|
||||
MM_FILESETTINGS_AMAZONS3SECRETACCESSKEY: MINIO_SECRET_KEY,
|
||||
MM_FILESETTINGS_AMAZONS3BUCKET: MINIO_BUCKET,
|
||||
MM_FILESETTINGS_AMAZONS3SSL: 'false',
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Checks Minio was started this run, restarts the server onto it if it isn't already the active
|
||||
* file storage backend, and confirms a real upload actually lands in Minio — skipping the test
|
||||
* otherwise, instead of failing on an unmet precondition.
|
||||
*/
|
||||
export async function ensureMinio(): Promise<void> {
|
||||
if (!testConfig.testcontainersServices.includes('minio')) {
|
||||
test.skip(true, 'Skipping test - minio not started (set PW_TESTCONTAINERS_SERVICES=minio)');
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
await ensureMinioBucket();
|
||||
const env = minioServerEnv();
|
||||
if (!bootEnvMatches(env)) {
|
||||
await restartMattermostContainer(env);
|
||||
}
|
||||
|
||||
const {adminClient, adminUser} = await getAdminClient();
|
||||
await uploadProbeImage(adminClient, adminUser);
|
||||
|
||||
const objectKeys = await listMinioObjectKeys();
|
||||
if (objectKeys.length === 0) {
|
||||
throw new Error(
|
||||
'Minio bucket is still empty after a real upload — the server is not actually using Minio ' +
|
||||
'as its file backend.',
|
||||
);
|
||||
}
|
||||
} catch (error) {
|
||||
test.skip(true, `Skipping test - Minio connection test failed: ${String(error)}`);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,46 @@
|
||||
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
||||
// See LICENSE.txt for license information.
|
||||
|
||||
import {test} from '@playwright/test';
|
||||
|
||||
import {runMmctl as runMmctlContainer} from '../containers/mmctl_container';
|
||||
import type {MmctlResult} from '../containers/mmctl_container';
|
||||
|
||||
import {getAdminClient} from './init';
|
||||
|
||||
import {testConfig} from '@/test_config';
|
||||
|
||||
export type {MmctlResult};
|
||||
|
||||
/**
|
||||
* Runs an mmctl command as a real remote client: a separate container built from the same server
|
||||
* image, authenticated with the current admin session, reaching the server over the
|
||||
* Testcontainers network rather than the `--local` unix socket the server's own container uses.
|
||||
*/
|
||||
export async function runMmctl(args: string[]): Promise<MmctlResult> {
|
||||
const {adminClient, adminUser} = await getAdminClient();
|
||||
if (!adminUser) {
|
||||
throw new Error('No admin user available to authenticate mmctl with.');
|
||||
}
|
||||
return runMmctlContainer(args, adminUser.username, adminClient.getToken());
|
||||
}
|
||||
|
||||
/**
|
||||
* Checks full (Testcontainers) mode is active and a real remote mmctl invocation actually reaches
|
||||
* the server — skipping the test otherwise, instead of failing on an unmet precondition.
|
||||
*/
|
||||
export async function ensureMmctl(): Promise<void> {
|
||||
if (!testConfig.useTestContainers) {
|
||||
test.skip(true, 'Skipping test - remote mmctl container requires PW_USE_TESTCONTAINERS=true');
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
const result = await runMmctl(['version']);
|
||||
if (result.exitCode !== 0) {
|
||||
throw new Error(`mmctl exited with code ${result.exitCode}: ${result.output}`);
|
||||
}
|
||||
} catch (error) {
|
||||
test.skip(true, `Skipping test - mmctl connectivity check failed: ${String(error)}`);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,162 @@
|
||||
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
||||
// See LICENSE.txt for license information.
|
||||
|
||||
import {AlreadyExistsError, Attribute, Change, Client} from 'ldapts';
|
||||
import {test} from '@playwright/test';
|
||||
import type {AdminConfig} from '@mattermost/types/config';
|
||||
|
||||
import {
|
||||
OPENLDAP_ADMIN_DN,
|
||||
OPENLDAP_ADMIN_PASSWORD,
|
||||
OPENLDAP_ALIAS,
|
||||
OPENLDAP_BASE_DN,
|
||||
OPENLDAP_PORT,
|
||||
} from '../containers/constants';
|
||||
|
||||
import {getAdminClient} from './init';
|
||||
|
||||
import {testConfig} from '@/test_config';
|
||||
import {getRandomId} from '@/util';
|
||||
|
||||
// Typed in-process LDAP client — avoids depending on external ldapadd/ldapmodify binaries.
|
||||
|
||||
export type LdapUser = {
|
||||
username: string;
|
||||
password: string;
|
||||
email: string;
|
||||
firstname: string;
|
||||
lastname: string;
|
||||
};
|
||||
|
||||
const ORG_UNIT = 'e2etest';
|
||||
const ORG_UNIT_DN = `ou=${ORG_UNIT},${OPENLDAP_BASE_DN}`;
|
||||
|
||||
export function generateLdapUser(prefix = 'ldap'): LdapUser {
|
||||
const randomId = getRandomId();
|
||||
const username = `${prefix}user${randomId}`;
|
||||
return {
|
||||
username,
|
||||
password: 'Password1',
|
||||
email: `${username}@mmtest.com`,
|
||||
firstname: `Firstname-${randomId}`,
|
||||
lastname: `Lastname-${randomId}`,
|
||||
};
|
||||
}
|
||||
|
||||
async function withAdminClient<T>(fn: (client: Client) => Promise<T>): Promise<T> {
|
||||
const client = new Client({url: `ldap://${testConfig.ldapHost}:${testConfig.ldapPort}`});
|
||||
try {
|
||||
await client.bind(OPENLDAP_ADMIN_DN, OPENLDAP_ADMIN_PASSWORD);
|
||||
return await fn(client);
|
||||
} finally {
|
||||
await client.unbind();
|
||||
}
|
||||
}
|
||||
|
||||
async function ensureOrgUnit(client: Client): Promise<void> {
|
||||
try {
|
||||
await client.add(ORG_UNIT_DN, {objectClass: 'organizationalUnit', ou: ORG_UNIT});
|
||||
} catch (error) {
|
||||
if (!(error instanceof AlreadyExistsError)) {
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** Creates the user (and the shared org unit, if it doesn't exist yet) and returns it. */
|
||||
export async function createLdapUser(user: LdapUser = generateLdapUser()): Promise<LdapUser> {
|
||||
await withAdminClient(async (client) => {
|
||||
await ensureOrgUnit(client);
|
||||
await client.add(`uid=${user.username},${ORG_UNIT_DN}`, {
|
||||
objectClass: 'inetOrgPerson',
|
||||
cn: user.firstname,
|
||||
sn: user.lastname,
|
||||
uid: user.username,
|
||||
mail: user.email,
|
||||
userPassword: user.password,
|
||||
});
|
||||
});
|
||||
return user;
|
||||
}
|
||||
|
||||
export async function updateLdapUser(username: string, changes: Partial<Omit<LdapUser, 'username'>>): Promise<void> {
|
||||
await withAdminClient(async (client) => {
|
||||
const dn = `uid=${username},${ORG_UNIT_DN}`;
|
||||
const attributeByType: Array<[string, string | undefined]> = [
|
||||
['cn', changes.firstname],
|
||||
['sn', changes.lastname],
|
||||
['mail', changes.email],
|
||||
['userPassword', changes.password],
|
||||
];
|
||||
|
||||
const modifications = attributeByType
|
||||
.filter((entry): entry is [string, string] => entry[1] !== undefined)
|
||||
.map(
|
||||
([type, value]) =>
|
||||
new Change({operation: 'replace', modification: new Attribute({type, values: [value]})}),
|
||||
);
|
||||
|
||||
if (modifications.length > 0) {
|
||||
await client.modify(dn, modifications);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
export async function deleteLdapUser(username: string): Promise<void> {
|
||||
await withAdminClient(async (client) => {
|
||||
await client.del(`uid=${username},${ORG_UNIT_DN}`);
|
||||
});
|
||||
}
|
||||
|
||||
// In `testcontainers` mode the Mattermost server is itself a container on the Testcontainers network, so it
|
||||
// must reach OpenLDAP via its network alias — unlike testConfig.ldapHost/ldapPort, which resolve to
|
||||
// a host-mapped address for this module's own ldapts client (a separate, non-containerized process).
|
||||
function ldapServerAddress(): [string, number] {
|
||||
return testConfig.useTestContainers ? [OPENLDAP_ALIAS, OPENLDAP_PORT] : [testConfig.ldapHost, testConfig.ldapPort];
|
||||
}
|
||||
|
||||
/** `LdapSettings` patch pointing the server at the OpenLDAP container and the users this module creates. */
|
||||
export function ldapServerConfig(): Partial<AdminConfig['LdapSettings']> {
|
||||
const [ldapServer, ldapPort] = ldapServerAddress();
|
||||
return {
|
||||
Enable: true,
|
||||
LdapServer: ldapServer,
|
||||
LdapPort: ldapPort,
|
||||
BaseDN: ORG_UNIT_DN,
|
||||
BindUsername: OPENLDAP_ADMIN_DN,
|
||||
BindPassword: OPENLDAP_ADMIN_PASSWORD,
|
||||
UserFilter: '(objectClass=inetOrgPerson)',
|
||||
IdAttribute: 'uid',
|
||||
LoginIdAttribute: 'uid',
|
||||
UsernameAttribute: 'uid',
|
||||
EmailAttribute: 'mail',
|
||||
FirstNameAttribute: 'cn',
|
||||
LastNameAttribute: 'sn',
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Checks OpenLDAP was started this run, points the server at it, and confirms the server can
|
||||
* actually reach it — skipping the test otherwise, instead of failing on an unmet precondition.
|
||||
*/
|
||||
export async function ensureOpenldap(): Promise<void> {
|
||||
if (!testConfig.testcontainersServices.includes('openldap')) {
|
||||
test.skip(true, 'Skipping test - openldap not started (set PW_TESTCONTAINERS_SERVICES=openldap)');
|
||||
return;
|
||||
}
|
||||
|
||||
const {adminClient} = await getAdminClient();
|
||||
await adminClient.patchConfig({LdapSettings: ldapServerConfig()});
|
||||
|
||||
// testLdap() searches under BaseDN and requires at least one matching user to succeed, so
|
||||
// probe with a throwaway user rather than depending on whatever the spec creates afterward.
|
||||
const probeUser = generateLdapUser('ensureprobe');
|
||||
try {
|
||||
await createLdapUser(probeUser);
|
||||
await adminClient.testLdap();
|
||||
} catch (error) {
|
||||
test.skip(true, `Skipping test - LDAP connection test failed: ${String(error)}`);
|
||||
} finally {
|
||||
await deleteLdapUser(probeUser.username).catch(() => undefined);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,54 @@
|
||||
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
||||
// See LICENSE.txt for license information.
|
||||
|
||||
import {test} from '@playwright/test';
|
||||
import type {AdminConfig} from '@mattermost/types/config';
|
||||
|
||||
import {OPENSEARCH_ADMIN_PASSWORD, OPENSEARCH_ALIAS, OPENSEARCH_PORT} from '../containers/constants';
|
||||
import {bootEnvMatches, restartMattermostContainer} from '../containers/stack';
|
||||
|
||||
import {getAdminClient} from './init';
|
||||
|
||||
import {testConfig} from '@/test_config';
|
||||
|
||||
// The Mattermost server always connects to OpenSearch itself (indexing/search requests), so it
|
||||
// needs the Testcontainers network alias, not a host-mapped address.
|
||||
export function opensearchServerConfig(): Partial<AdminConfig['ElasticsearchSettings']> {
|
||||
return {
|
||||
ConnectionURL: `http://${OPENSEARCH_ALIAS}:${OPENSEARCH_PORT}`,
|
||||
Username: 'admin',
|
||||
Password: OPENSEARCH_ADMIN_PASSWORD,
|
||||
EnableIndexing: true,
|
||||
EnableSearching: true,
|
||||
EnableAutocomplete: true,
|
||||
Sniff: false,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Checks OpenSearch was started this run, restarts the server onto it if the OpenSearch Go client
|
||||
* isn't already the registered search engine, and confirms the server can actually reach it —
|
||||
* skipping the test otherwise, instead of failing on an unmet precondition. Backend picks which of
|
||||
* two Go implementations (Elasticsearch vs OpenSearch client) the server registers — a factory
|
||||
* invoked once at startup, never re-invoked by the config-change watcher — so switching it needs a
|
||||
* restart, unlike the rest of ElasticsearchSettings, which the watcher does pick up live.
|
||||
*/
|
||||
export async function ensureOpensearch(): Promise<void> {
|
||||
if (!testConfig.testcontainersServices.includes('opensearch')) {
|
||||
test.skip(true, 'Skipping test - opensearch not started (set PW_TESTCONTAINERS_SERVICES=opensearch)');
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
const env = {MM_ELASTICSEARCHSETTINGS_BACKEND: 'opensearch'};
|
||||
if (!bootEnvMatches(env)) {
|
||||
await restartMattermostContainer(env);
|
||||
}
|
||||
|
||||
const {adminClient} = await getAdminClient();
|
||||
await adminClient.patchConfig({ElasticsearchSettings: opensearchServerConfig()});
|
||||
await adminClient.testElasticsearch();
|
||||
} catch (error) {
|
||||
test.skip(true, `Skipping test - OpenSearch connection test failed: ${String(error)}`);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
||||
// See LICENSE.txt for license information.
|
||||
|
||||
import {test} from '@playwright/test';
|
||||
|
||||
import {getAdminClient} from './init';
|
||||
|
||||
/**
|
||||
* Disables Elasticsearch/OpenSearch indexing/searching so search falls back to the database —
|
||||
* skipping the test otherwise, instead of failing on an unmet precondition. The counterpart to
|
||||
* ensureElasticsearch()/ensureOpensearch() for specs that specifically need database-backed search
|
||||
* active (e.g. after another spec in the same run switched it away).
|
||||
*
|
||||
* No restart involved, unlike most other ensure*() functions: which search engine (if any) is
|
||||
* active is decided at runtime from these enable flags, not from the boot-time Backend setting, so
|
||||
* disabling them here is enough to make the server fall through to Postgres-backed search.
|
||||
*/
|
||||
export async function ensurePostgresSearch(): Promise<void> {
|
||||
try {
|
||||
const {adminClient} = await getAdminClient();
|
||||
await adminClient.patchConfig({
|
||||
ElasticsearchSettings: {EnableIndexing: false, EnableSearching: false, EnableAutocomplete: false},
|
||||
});
|
||||
} catch (error) {
|
||||
test.skip(true, `Skipping test - database search check failed: ${String(error)}`);
|
||||
}
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user