Wire Microsoft's local Azure Blob Storage emulator into the dev/CI
docker-compose stack so an upcoming Azure FileBackend driver and its
tests have a target to run against, without requiring an Azure
account.
* Define the azurite service in docker-compose.common.yml (used by
both the local-dev and CI compose files).
* Add azurite to the makefile + main docker-compose.yml service maps,
and to docker-compose-generator so callers can include it via
ENABLED_DOCKER_SERVICES.
* Auto-include azurite in `make start-docker` (mirrors how minio is
auto-included today), so existing local workflows keep working
without any per-developer config change.
* Add azurite to the CI start_dependencies wait set so CI brings it up
alongside postgres/minio.
* Set CI_AZURITE_HOST / CI_AZURITE_PORT in dotenv/test.env, mirroring
the CI_MINIO_* pattern.
No production filestore changes — this PR is mergeable in isolation
with no user-visible behavior.
------
AI assisted commit
* Add container name to Docker logs
This will allow for querying Loki by container's name:
{job="docker",container_name="mattermost-postgres"}
* Configue Loki to prepare for OTLP ingestion
- Add a volume to Loki container to get the config
- Configure Loki with the expected labels so that we can query by job,
app, container.name...
* Add OpenTelemetry collector configuration
There are three pipelines:
1. logs/mattermost scrapes the logs from mattermost.log, parsing the
timestamp and severity, and pushes them to Loki.
2. logs/docker scrapes the Docker logs from *-json.log, parsing the
timestamp, the log itself and the container name, and pushes them to Loki.
3. metrics/docker scrapes the Docker socket to retrieve the containers'
uptime values and pushes them to Prometheus.
* Replace Promtail with OpenTelemetry collector
* Update build tooling for OpenTelemetry collector
1. Make sure that the logs directory is created
2. Swap Promtail with OpenTelemetry collector
* Scrape collector to get Docker stats
Prometheus needs to scrape the OpenTelemetry collector in the exposed
port to get the Docker stats, so that we can query the uptime with
metric container_uptime_seconds, which has a container_name label to
filter by container.
* Update Grafana dashboard for Docker health checks
1. Use Prometheus as the datasource in all queries
2. Simplify the mappings to either 0 (offline, red) or 1 (online,
green).
3. Unify all queries on container_uptime_seconds, filtering by
container_name, and making sure that the latest value we got is at most
15 seconds old, so that it does not show stale data.
4. Add Redis health check, that was missing
5. Update the dashboard title to Docker containers
* Tune Loki and OTel collector configs for local dev
- Switch filelog receivers to start_at: beginning so existing logs are
ingested on collector startup, not just new entries.
- Fix Docker log timestamp layout to use 9s (variable-length nanos)
instead of 0s (fixed-width), matching actual Docker JSON log format.
- Add ingester max_chunk_age to keep chunks open longer in the
single-instance dev setup, so that we can ingest older logs (the
window is max_chunk_age/2).
- Relax Loki limits for local development: allow unordered writes,
disable old-sample rejection, and raise ingestion rate/burst to 64 MB
to avoid throttling during bulk ingest.
* Explicitly name Postgres container volume
* Remove unused server/docker-compose.yaml
This file doesn't seem to actually be used. When we run docker compose locally,
it uses docker-compose.makefile.yml merged with the output of
build/docker-compose-generator/main.go.
* Revert "Remove unused server/docker-compose.yaml"
This reverts commit 5a45965217.
* Update volume name
- Update library version.
- Added MaxFlush delay to help reduce CPU usage.
- Fall back to LRU cache for the caches which use SCAN.
- Added mattermost-redis and running for all api layer
tests in Postgres.
https://mattermost.atlassian.net/browse/MM-59934
```release-note
NONE
```
* configure Loki for local development
Extend our docker compose setup to include Loki (along with promtail), scraping `logs/*.log` to ingest development logs as well as all logs from running containers.
While we're in here, teach Prometheus to scrape metrics from Docker containers too.
* tweak promtail labels
* document extra services in config.mk
* update home.json
Alejandro García Montoro