Triages the three divergences backlog#2054 found between the audit and
notify default KVS tables, cross-checked against MinIO upstream
(internal/logger/config.go, internal/config/notify/parse.go):
- webhook: audit's extra batch_size/max_retry/retry_interval/http_timeout
keys match MinIO's DefaultAuditWebhookKVS byte-for-byte, while notify's
table matches MinIO's notify DefaultWebhookKVS (which lacks them).
Intentional, not a copy/paste gap — documented with a doc comment on
each table instead of changed.
- mqtt: audit's stronger QoS/keep-alive/reconnect defaults have no MinIO
precedent (MinIO's audit logging has no MQTT target at all), while
notify's 0/0s/0s defaults match MinIO's DefaultMQTTKVS exactly.
Documented as an intentional RustFS-original choice, not changed.
- auth_token hidden_if_empty: audit had false, notify had true, with no
MinIO precedent either way (this KVS version has no per-key hidden
flag upstream). Fixed audit to true, matching notify and every other
sensitive key in both files (MQTT_PASSWORD, *_TLS_*). Non-empty tokens
were already redacted identically on both sides via ends_with("_token")
pattern matching in config_admin.rs — this only changes how an *unset*
audit webhook auth_token renders in admin config output (omitted
instead of shown as an empty value).
Refs rustfs/backlog#2054
TierAzure.storage_class and .sp_auth round-trip faithfully through the
admin API and on-disk config (ExternalTierAzure encode/decode in
tier.rs), so an operator can configure them, read them back via
ListTier, and never learn they do nothing. They are dropped only at the
WarmBackendAzure construction boundary: the Azure warm backend goes
through the same S3-compatible TransitionClient as every other
provider and has no Azure Blob-native client or Azure AD dependency
(confirmed: no azure_* crate anywhere in the workspace), so neither
field can actually be honored today. MinIO's reference implementation
(cmd/warm-backend-azure.go) treats both as first-class: storage_class
sets the blob access tier on every PUT, and sp_auth is a full
alternative to access/secret-key auth via azidentity, mutually
exclusive with it.
Rather than the larger, riskier options (add a native Azure SDK
dependency and a parallel non-S3 client path, or break the persisted
config format by removing the fields), this closes the silent-failure
gap with the minimal safe fix: TierConfigMgr::add now rejects an Azure
tier config with either field set, before backend construction,
returning ERR_TIER_INVALID_CONFIG with an explicit message instead of
accepting and ignoring. The fields stay in the config type (no format
break); already-persisted tiers with these fields set are grandfathered
in un-rejected (edit does not touch sp_auth or storage_class either).
Full support remains a larger follow-up if ever prioritized.
Also removes TierAzure::is_sp_enabled(), which had zero callers
repo-wide (backlog#2055 flagged this) and would have been misleading
dead weight once this decision was made — reusing it for the new gate
would also have been wrong, since it requires *all three* sp_auth
fields non-empty (&&), while the gate must reject on *any* one being set.
Refs rustfs/backlog#2055
(cherry picked from commit 8d148c4e9b2507a1c5075e3d9513adb8b5851ef5)
Merge newer partial observed usage into the complete authoritative admin baseline instead of replacing the full bucket set.
Keep the merged view partial and non-converged so shared consumers do not treat it as quota-authoritative.
Co-authored-by: heihutu <heihutu@gmail.com>
A PUT with HTTP preconditions took the per-object namespace write lock
before ingesting the request body and held it until commit, so any
concurrent read of the same object queued behind client-paced body
ingestion until the 5s acquire timeout and surfaced as 503. Exposed as
a deterministic S3 Implemented Tests gate failure when #6770 routed
1 MB conditional writes onto the streaming path (rustfs/backlog#2074).
Keep a lock-free advisory precondition check before the body for fast
412/404, and evaluate the authoritative check under the put_object
commit lock, reusing the deferred shape data movement already uses.
Reads during ingestion now return the last committed version, and a
precondition invalidated mid-stream fails closed with 412 at commit.
Validate lifecycle tier references through the tier reference proof path, preserve S3 list CommonPrefix XML compatibility, and make GetObject audit completion use real S3 error status codes.
Co-authored-by: heihutu <heihutu@gmail.com>
refactor(ecstore): huaweicloud/tencent reuse the shared S3 constructor
Migrates the Huaweicloud and Tencent tier warm backends onto the shared
S3-compatible constructor (backlog#2040). Also makes the shared
constructor's outbound-URL validation injectable per provider
(S3CompatibleWarmBackendParams::validate_endpoint) so it can centralize
rustfs/rustfs#6764's SSRF check for the providers that don't need an
exception, while accommodating rustfs/rustfs#6773's RustFS-specific
debug-only loopback opt-in without weakening the other six providers.
Updates scripts/error-other-format-baseline.txt: the one ::other(format!)
call site moves from the two per-provider files into the new shared
call site in warm_backend.rs (net call-site count unchanged).
Refs rustfs/backlog#2042
The heal manager carried its own byte-identical copy of the foreground pressure type and threshold computation that ecstore's data-movement backpressure also carries, so every change to the admission-utilization rules had to be mirrored by hand across two crates. The shared `ForegroundPressure` and `foreground_pressure` added to `rustfs-concurrency` now own that logic, and heal already depends on that crate, so this removes the duplicate without adding a crate edge.
`mainline_throttle_active` keeps the parts that are specific to this call site: the `mainline_throttle_enable` and both-thresholds-zero short circuit that avoids touching the provider at all, the optional-provider unwrap, and the heal-side threshold fields. Everything downstream is untouched — the `reason()` labels `foreground_read_pressure`, `foreground_write_pressure`, and `foreground_pressure` are byte-identical to the removed implementation, so the `rustfs_heal_mainline_throttle_total` reason label and the `heal_mainline_throttle` log fields keep their observability contract.
Refs rustfs/backlog#2049
(cherry picked from commit ec491bcbd8939e5978cd94f9a44cffb70d09fade)
(cherry picked from commit e800f29d6806591689204b3712300800b480beae)
Co-authored-by: houseme <housemecn@gmail.com>
refactor(ecstore): use shared ForegroundPressure for data movement
The data movement backpressure module carried its own byte-identical copy of ForegroundPressure, its reason() label mapping, and the foreground utilization computation. rustfs-concurrency now owns that logic as workload::ForegroundPressure and workload::foreground_pressure, so the local copy was a cross-crate synchronization point that could silently drift from the heal-side and admission-side behavior.
Delete the local type and computation and call the shared function instead. The call site keeps what is specific to data movement: the config.enabled short circuit, the optional provider unwrap, and the read/write threshold percentages read from DataMovementBackpressureConfig. The reason() labels emitted into the rustfs_data_movement_backpressure_total metric and the data_movement_backpressure log event are unchanged, as are the existing tests and their assertions.
Refs rustfs/backlog#2048
(cherry picked from commit 6a26e144e06ced53a8dfd1712ab7aa24589646ff)
(cherry picked from commit ab5ab417e80179265c32b22a5e671eac0b9e43ae)
refactor(ecstore): migrate notify default KVS to shared constructors
The amqp, nats, pulsar, redis, postgres, kafka and mysql default KVS tables in config/notify.rs duplicated the corresponding tables in config/audit.rs literally, leaving seven cross-file sync points that a future default or key-order edit had to keep aligned by hand. Replace those seven table bodies with calls to the shared constructors added in config/target_defaults.rs, passing the notify-side literals where the two subsystems genuinely differ: NOTIFY_REDIS_DEFAULT_CHANNEL for the redis channel and "rustfs_events" for the mysql table.
Key order is part of the admin config contract, so this is a pure restructuring: for all seven tables the ordered key sequence and every key's value and hidden_if_empty flag are unchanged.
DEFAULT_NOTIFY_WEBHOOK_KVS and DEFAULT_NOTIFY_MQTT_KVS are deliberately left untouched. Those two tables really do diverge between audit and notify, so folding them into shared constructors would change runtime behavior; the divergence is tracked separately in rustfs/backlog#2054.
Refs rustfs/backlog#2046
(cherry picked from commit 6df9b53027ef2f0cf9aa7b82ecb2af8c5108f11f)
(cherry picked from commit 14bbf756bea2ee6daacbfdc7d7a452effa649cff)
refactor(ecstore): migrate audit KVS defaults to shared constructors
The amqp, nats, pulsar, redis, postgres, kafka and mysql default KVS tables in config/audit.rs duplicated the corresponding tables in config/notify.rs, leaving seven cross-file sync points where a default could silently drift between the two subsystems. Build them from the shared constructors added in config/target_defaults.rs instead, passing in the two literals that are genuinely audit-specific: the redis pub/sub channel (AUDIT_REDIS_DEFAULT_CHANNEL) and the mysql destination table ("rustfs_audit_logs").
Key order, every default value and every hidden_if_empty flag are preserved exactly, since the key order drives the order admin config output lists keys in. DEFAULT_AUDIT_WEBHOOK_KVS and DEFAULT_AUDIT_MQTT_KVS are left untouched: those two tables really do differ from their notify counterparts, so unifying them would change runtime behavior.
Refs rustfs/backlog#2045
(cherry picked from commit 4de580e6d8901485eef268191924e035322d4d4e)
(cherry picked from commit e5e301fa78e7f27ffa85b05cf5e7962301a5ff00)
- common.rs gains an AdminTransport knob (Signed | Awscurl) with admin_execute_at plus three family wrappers: admin_create_user_via, admin_add_canned_policy_via, admin_attach_user_policy_via; the existing admin_create_user now delegates over the Signed transport.
- Deleted the four signed admin request clones in admin_mfa_test, admin_auth_test, reliant/tiering, and inline_fast_path_cluster_test; each keeps a thin local wrapper over common::admin_request so call sites keep their Option<&str> body shape.
- Deduped the notification_webhook signer onto common::signed_request and the webdav_core signer plus its three admin helpers onto the shared _via helpers.
- Consolidated the S3-client-with-credentials builders: admin_auth s3_client_with, existing_object_tag user_client/sts_session_client, bucket_policy_check create_user_client, and the create_user_s3_client copies in group_delete_test and replication_extension_test now delegate to create_s3_client_with_credentials / build_test_s3_config; replication_extension admin_add_canned_policy and admin_attach_policy_to_user route through the _via helpers on the Signed transport.
- The awscurl-gated suites (existing_object_tag_policy, bucket_policy_check, policy/policy_variables) keep going through the external awscurl binary via AdminTransport::Awscurl, preserving their wire behavior.
Part of rustfs/backlog#1846 (cluster 2).
* fix(s3): round-trip null-version delete-marker identity through listing and delete responses
On a versioning-suspended bucket, a null delete marker's identity was lost on the way back to the client at three points (issue #6745): ListObjectVersions advertised the marker's VersionId as the literal nil UUID instead of null; deleting by that id succeeded but the DeleteObjects/DeleteObject response reported the identity as null with no way to correlate it to the request; and the response lacked DeleteMarker/DeleteMarkerVersionId because the marker-ness comparison mixed the client-facing identity (Some(nil)) with the storage identity (None), so the removal also mis-recorded accounting and fired DeleteMarkerCreated semantics on later paths.
- Listing (bucket_usecase, s3_api/bucket, build_list_versions_next_marker) now maps the synthesized nil UUID to the literal null everywhere it reaches the wire, and VersionMarker::parse folds a nil-UUID marker from older listings into VersionMarker::Null so pagination resumes correctly.
- delete_objects normalizes both sides of the marker-ness comparison via delete_file_info_version_id (matching the adjacent explicit_delete_marker admission check) and reports DeleteMarkerVersionId as null for an explicit null-marker removal.
- resolve_delete_version_state reports delete_marker for an explicit-version delete whose target is a delete marker even when the bucket is versioning-suspended, fixing x-amz-delete-marker on the single-object path.
- The DeleteObjects response entry echoes the version identity the request addressed for marker removals, marker-removal accounting no longer records a marker creation, and notification events fire DeleteMarkerCreated only for actual marker creation.
Fixes#6745
* fix(s3): keep null-marker removal write shape undeleted and report marker semantics response-side
The first cut marked the storage delete request deleted for a null-marker removal, which FileMeta::delete_version interprets as the suspended-bucket delete-mints-a-marker write and re-creates the marker just removed. Carry marker-ness to responses via explicit_delete_removed_marker (single path) and a response-only branch flag (batch path) instead, keeping every storage write shape byte-identical to the pre-fix behavior. Adds an embedded end-to-end regression test covering the full issue #6745 round trip.
WarmBackendS3::new already rejects loopback, private, link-local, and
cloud metadata-service endpoints via validate_outbound_url, but the
Aliyun, Azure, Huaweicloud, Tencent, MinIO, R2, RustFS, and GCS warm
backend constructors built their transition clients directly from
conf.endpoint without the same check.
The endpoint comes from the AddTier admin API, gated only by
SetTierAction, which can be a narrower IAM grant than root. Any
principal holding it could point one of these eight tier types at an
internal address (loopback, RFC1918, link-local, or a cloud metadata
IP) and have the server issue authenticated outbound requests to it, a
server-side SSRF vector that the S3 and Wasabi tier types were already
closed against.
Apply the same validate_outbound_url check at construction time for
all eight providers, before any credentials or network client are
built, mirroring the existing WarmBackendS3 pattern. GCS keeps its
default-endpoint behavior when conf.endpoint is empty and only
validates an explicitly configured endpoint.
Add a regression test per provider asserting that a loopback endpoint
is rejected before any backend/network setup, matching the existing
WarmBackendS3 coverage.
Update the error(format!) ratchet baseline: these are one-shot admin
tier-configuration validation errors returned once per AddTier call,
not per-disk I/O errors that flow through reduce_errs quorum
aggregation (backlog#1845), so the new ::other(format!) call sites do
not introduce a quorum-bucketing hazard. They mirror the pre-existing,
already-baselined warm_backend_s3.rs call site.
TransitionClient::new() in crates/s3-client/src/transition_api.rs computes
trailing_header_support = opts.trailing_headers && override_signer_type == SignatureV4,
but override_signer_type is hardcoded to SignatureDefault at construction
and never mutated afterwards, so the expression is always false regardless
of opts.trailing_headers. The resulting field also has no live reader: its
only reference is inside PutObjectOptions::validate() in
crates/s3-client/src/api_put_object.rs, which is itself
#[allow(dead_code, reason = "MinIO-parity ... no caller in this port")],
and even there the reference to trailing_header_support is commented out.
So trailing_headers: true in the seven warm_backend_*.rs constructors has
never had any effect on request signing or chunked/trailing-header
behavior (stream_sha256 signing is gated separately by
metadata.stream_sha256 && !self.secure). Remove the misleading dead
configuration from the seven provider constructors so it doesn't look
like intentional, load-bearing behavior to future readers.
Found during adversarial self-check while implementing rustfs/backlog#2040 (out of that issue's scope).
* refactor(ecstore,rustfs): reuse canonical starts_with_ignore_ascii_case
`crates/utils/src/http/metadata_compat.rs` owns the internal metadata key helpers, including `starts_with_ignore_ascii_case`. Two files carried their own byte-identical copies of that predicate: `SetDisks::starts_with_ignore_ascii_case` in ecstore and a free function in the S3 options layer. Both drive internal metadata key classification (`internal_metadata_suffix` and quorum hashing on one side, `should_skip_object_metadata_key` and `is_reserved_user_metadata_key` on the other), so keeping three implementations of one predicate is an avoidable drift risk on a path that decides whether an internal key is treated as user metadata.
Delete both local copies and call the canonical implementation. Every prefix used at these call sites is an ASCII constant or literal, where the canonical byte-slice comparison and the removed `str::get(..n)` form are equivalent; that equivalence was checked differentially over 4.6M (key, prefix) pairs, including keys with multi-byte characters straddling the prefix boundary. No other logic in `internal_metadata_suffix` or `should_skip_object_metadata_key` changed.
Add regression tests on both sides pinning the two properties the switch depends on: internal prefixes match case-insensitively (a mixed-case `X-RustFS-Internal-*` key stays internal), and keys shorter than a prefix never match (they stay ordinary user metadata).
Refs rustfs/backlog#2051
* fix(rustfs): avoid typos-checker false positive in prefix-length test
The test literal "x-rustfs-encryptio" (a deliberate truncation of the
x-rustfs-encryption- prefix, used to assert that a key shorter than every
internal prefix falls through to user metadata) reads as a likely typo of
"encryption" to the repo's typos CI check. Derive it from
RUSTFS_ENCRYPTION_PREFIX via slicing instead of a hand-typed literal, which
both satisfies the linter and ties the truncation to the real constant
instead of a copy-typed guess.
Refs rustfs/backlog#2051
`crates/lifecycle/src/tagging.rs` carried a byte-identical copy of the `form_urlencoded` tag decoder already owned by `rustfs-replication`, plus a duplicate of its test. Since `crates/lifecycle` already depends on `rustfs-replication`, replace the copy with a `pub(crate) use` re-export: no new crate edge, one parser, and no second implementation to drift from the replication contract. The `rule.rs` call site is unchanged.
Also drop `crates/ecstore/src/bucket/lifecycle/tagging_boundary.rs`, a migration-era boundary shim with zero call sites in the tree.
crates/s3-client/src/utils.rs carried a verbatim copy of the header
classification tables and predicates owned by
crates/utils/src/http/headers.rs: SUPPORTED_HEADERS (same 11 keys),
SUPPORTED_QUERY_VALUES (same 9 keys), and is_standard_header /
is_storageclass_header / is_amz_header / is_rustfs_header /
is_minio_header with byte-identical bodies. The duplication was already
half-resolved and inconsistent — the local is_amz_header called
rustfs_utils::http::is_sse_header while consulting its own tables — and
s3-client already depends on rustfs-utils with the "full" feature, so
reusing the canonical owner adds no crate edge.
The sole caller, PutObjectOptions::header(), now imports the five
predicates from rustfs_utils::http. Semantics are unchanged: both sides
normalize with to_lowercase(), return false for unknown keys, and the
storage-class constants are the same string ("x-amz-storage-class" from
s3s::header::X_AMZ_STORAGE_CLASS vs rustfs_utils AMZ_STORAGE_CLASS), so
the set of user-metadata headers passed through verbatim rather than
prefixed with x-amz-meta- is identical.
SUPPORTED_QUERY_VALUES is deleted outright: s3-client had no reader for
it (utils consumes its own copy via is_standard_query_value). The
base64_encode/base64_decode helpers and their rustfs/rustfs#4811
regression test stay untouched, and lazy_static remains a dependency
because crates/s3-client/src/constants.rs still uses it.
Refs rustfs/backlog#2050
ForegroundPressure had two definitions with byte-identical pressure computation: one in ecstore data-movement backpressure and one in the heal manager queue. That duplication is a violation of the ARCHITECTURE.md invariant that each type has exactly one definition, and it means any future change to the utilization math has to land twice.
Add the canonical `ForegroundPressure` and a `foreground_pressure(snapshot, read_threshold_pct, write_threshold_pct)` function to `crates/concurrency/src/workload.rs`, which already owns `WorkloadClass`, `AdmissionState`, and the admission snapshot contract. Both existing consumers already depend on `rustfs-concurrency`, so no crate edge is added.
The filter_map pipeline is transferred verbatim, preserving all five boundary behaviors (zero threshold, zero limit, missing entry, missing active count, and the `Saturated` full-utilization special case), the mul-before-div percentage normalization, the `>=` threshold comparison, and the read-then-write ordering that makes `max_by_key` break utilization ties toward the write class. The enable switch is deliberately left out: ecstore gates on `config.enabled` while heal gates on `mainline_throttle_enable` plus a both-thresholds-zero check, so each call site keeps its own condition.
This is the expand step only. The ecstore and heal copies are untouched and are removed by the follow-up migrate task.
Refs rustfs/backlog#2047
The seven S3-compatible warm backend providers (Aliyun, Azure, Huaweicloud, Tencent, MinIO, R2, RustFS) each carry a byte-identical copy of the same statically-credentialed TransitionClient construction and of the same optimal_part_size helper. Add both to the module that already owns the WarmBackend trait and WarmBackendS3, so the per-provider migrate step can drop its duplicate without redesigning anything.
bucket_lookup is a parameter rather than a constant because the providers split into two families: Aliyun, Azure, Huaweicloud, and Tencent pin BucketLookupDNS, while MinIO, R2, and RustFS leave it at the BucketLookupAuto default. Hardcoding either value would silently change bucket addressing for the other family during the migrate step.
Error texts, validation order, prefix and host/port normalization are reproduced exactly from the Aliyun/MinIO family. No provider file is touched and no production caller exists yet, so the new unit tests are the first callers.
Refs rustfs/backlog#2040
The audit and notify subsystems each declare their own default KVS table for the same nine delivery targets. For amqp, nats, pulsar, postgres and kafka the two declarations are byte-identical; for redis and mysql they differ only in a single default literal (the pub/sub channel and the destination table). Keeping two copies means every default or key-order change has to be made twice, and a missed edit silently changes what admin config reports for one subsystem only.
Add `config::target_defaults` with one constructor per shared table, taking the diverging literal as a parameter for redis and mysql, plus a small `kv` helper that replaces the repeated `KV { .. }` literals. Key order is reproduced exactly because it drives the order the admin API lists keys in. Unit tests pin the full ordered key/value/hidden_if_empty triple of every table against hard-coded literals, and cover both the audit and the notify literal for the two parameterized tables.
Webhook and mqtt are deliberately left out: audit's webhook table carries extra batching and retry keys, both webhook tables disagree on key order and on the auth-token hidden_if_empty flag, and mqtt disagrees on qos, keep-alive interval and reconnect interval. Those are real behavioral forks, not duplication, so they stay declared in place.
This is the expand step only. Nothing calls the new module yet, so audit.rs and notify.rs are untouched and no default changes; the constructors carry an item-level allow(dead_code) until the migrate step points both files at them.
Refs rustfs/backlog#2044
Add a default-on, size-aware foreground PUT admission policy so large or
unknown-size PutObject requests are backpressured before body ingest and
erasure/RPC fan-out. Preserve the explicit strict gate semantics, including
limit=0 as an opt-out, and keep small PUTs on the legacy fast path.
Closesrustfs/backlog#2038
Co-authored-by: heihutu <heihutu@gmail.com>
* Revert "perf(ecstore): use AHashMap for FileInfo metadata fields (#6738)"
This reverts commit 13a2ae212e.
* fix(filemeta): restore standard HashMap metadata (#6742)
Remove the direct ahash dependency added for FileInfo metadata and revert the affected filemeta/ecstore call sites back to std::collections::HashMap.
Co-authored-by: heihutu <heihutu@gmail.com>
---------
Co-authored-by: heihutu <heihutu@gmail.com>
Keep /health/ready from riding the generic internode lock RPC and channel keepalive budgets when a peer host is unreachable. Add a health-specific lock online timeout, route ping failures through the existing remote lock RPC eviction path, cache the static ping payload for readiness fan-out, and cover hanging cached channels with focused tests.
Refs rustfs/backlog#2033
Refs rustfs/rustfs#6286
Co-authored-by: heihutu <heihutu@gmail.com>
Config::new() and the external decode path read the process-global
DEFAULT_KVS OnceLock at call time, and config::tests in the same test
binary register it via crate::config::init() mid-run. Several com.rs
tests asserted on unregistered state (heal section absence, equality
with a later Config::new()), so they could flip depending on thread
scheduling under cargo test -p rustfs-ecstore --lib config::.
Assert on the semantic heal diff instead of section presence, normalize
compared configs with a single DEFAULT_KVS snapshot taken after both
sides exist, and compare the snapshot transaction test against the
persisted baseline bytes.
* test(pool): fix warp log path and retry rebalance start
- warp writes now use a unique mktemp log file instead of a fixed
/tmp/rustfs-warp.log: the runner user could not write the stale
root-owned file, which made the background warp process die instantly
(warp never ran). The workflow uploads /tmp/rustfs-warp.*.log.
- rebalance start is retried (6x, 20s apart): nightly builds gate
rebalance activation on a live cross-pool fence fleet capability proof
that takes ~10-20s to re-establish after a pool joins. Verified live:
attempt 1 fails with 500 'pool activation requires a live fleet
capability proof', attempt 2 succeeds.
* test(pool): annotate known server-side issues in failure output
When a node fails to start, grab the rustfs journal tail and match known
server-side error signatures (e.g. the fleet capability proof cold-start
regression, rustfs/backlog#2031), printing a hint with the tracking issue.
Also annotate the rebalance-start retry exhaustion and the rc.3 decommission
metacache-listing failure with actionable guidance.
* fix(ecstore): defer rebalance activation without fleet proof
---------
Co-authored-by: 马登山 <cxymds@qq.com>
Co-authored-by: cxymds <cxymds@gmail.com>
* test(ecstore): decouple server config snapshot test from global defaults
The final assertion of server_config_snapshot_serializes_read_modify_write_transactions
compared the second snapshot against a fresh Config::new(). Config::new()
reads the process-global DEFAULT_KVS OnceLock, which a sibling test in the
same process can register mid-run (crate::config::init()), so the in-process
run 'cargo test -p rustfs-ecstore --lib config::' failed while nextest's
process-per-test isolation hid the coupling. Assert on the snapshot's raw
bytes against the baseline blob instead, which is deterministic and matches
the invariant under test: the second transaction observes the store unchanged
by the first.
* test: deflake presigned tamper helper and relocated-pool resume staging
tamper_signature only remapped '0' and 'a', so a signature containing
neither (about 1 in 5000) left the URI unchanged and tripped the helper's
own guard assert in CI. Complement every hex digit (15 - v) instead: the
map has no fixed point, so the tamper always changes the value while
keeping length and hex shape.
execute_get_object_resumes_from_relocated_pool_without_splicing_body
staged the relocation by reading xl.meta from every source-pool disk, but
a write-quorum commit legitimately leaves a lagging minority disk without
the object directory (#6701) — the test already tolerates that gap when
normalizing the upload pool, and CI suite IO load hit the same gap in the
staging loop. Skip sourceless disks, carry the staged metadata path
explicitly, and assert a write-quorum majority was staged.
* test(targets): ship a builder MockTarget testkit and retire the in-crate Target mocks
Adds crates/targets/src/testkit.rs with a builder-style MockTarget implementing Target<E> for every E: PluginEvent, with orthogonal off-by-default knobs: disabled/active override, health delay plus health-started signal plus a drop-guard counter proving a cancelled probe future was dropped, an init failure budget (usize::MAX = always fail) plus blocking init plus an init counter, a close counter/signal/semaphore gate with a runtime block toggle, a save counter plus save failure budget, caller-supplied store and failed-store handles, and a shared final-failure counter. Clones and clone_dyn share all counters, so an observer clone keeps watching a target after it is boxed into a runtime.
The module is gated as #[cfg(any(test, feature = "test-support"))]: in-crate unit tests get it via cfg(test), and downstream test suites opt in through the new off-by-default test-support cargo feature (test-support = [], activating no dependencies).
Migrates the five in-crate duplicate mocks onto it: the plugin.rs registry-factory TestTarget, the runtime/adapter.rs lifecycle TestTarget (init/close/store knobs), the runtime/mod.rs TestTarget plus HealthDropGuard (close gating and health-probe tests), the target/mod.rs MoveTestTarget (folded into the test_support helper constructors used by the NATS JetStream failed-store tests), and the target/mod.rs StoreBackedTarget (the default send_from_store purge test; the mock deliberately does not override send_from_store or handle_terminal_failure). The forced init failure now uses TargetError::Initialization instead of the old adapter mock's Configuration; the adapter derives its redacted failure summary from the target id alone, so the migrated assertions are unchanged.
Leak guard: testkit unit tests assert the crate manifest still declares default = [] and that test-support = [] stays a pure cfg gate, complementing the compile-level cfg gate that keeps the mock out of production builds.
Part of rustfs/backlog#1846 (cluster 3, step 1).
* test(notify,audit): migrate the Target mocks onto the shared testkit
Retires the hand-written Target mocks in crates/notify and crates/audit in favor of rustfs_targets::testkit::MockTarget: notify's notifier.rs TestTarget/DeferredTestTarget/ClosableTestTarget, lifecycle.rs BlockingInitTarget/RetryInitTarget (rebuilt as observed MockTarget templates cloned by their plugin-descriptor factories, sharing the init signal, close counter, and single-failure init budget across generations), runtime_view.rs TestTarget, runtime_facade.rs TestTarget, and audit's pipeline.rs MockTarget, system.rs TestTarget, registry.rs CloseTestTarget, plus TestTarget and FailingTarget in audit/tests/pipeline_layer_test.rs. Removing the two integration-test mocks also removes their respelled PluginEvent bound (E: Send + Sync + 'static + Clone + Serialize + DeserializeOwned), which the plugin-contract rules require to be spelled only via PluginEvent.
lifecycle.rs ReplayTarget stays bespoke on purpose: its generation tags, mpsc observation channels, gated send_raw delivery, and ObservedQueueStore model the replay pipeline itself and would contort a general-purpose mock. The other bespoke mocks named out of scope in PR-3a (ProgrammedTarget, ClassifyingTarget, the ReloadableTargetTls fakes) are likewise untouched.
New testkit knobs, each defaulted off and unit-tested: with_id (rename a clone while keeping the shared counters, for factory templates), with_first_save_gate (the first save notifies entered and waits on release; several mocks may share one pair), with_health_gate (is_active waits on a release handle after notifying health_started), with_delivery_snapshot (fixed snapshot overriding the store-derived default), with_close_failures (close-failure budget, default TargetError::Storage) with with_close_failure_error to shape the variant (audit's registry test pins TargetError::Unknown), and an always-on is_enabled call counter exposed as enabled_call_count (notifier's generation tests count dispatcher selections through it).
Both crates enable the testkit through a dev-dependency on rustfs-targets with the test-support feature; the feature stays out of default and activates no dependencies, so production builds are unchanged.
Part of rustfs/backlog#1846 (cluster 3, step 2).
refactor(s3-client): remove the superseded per-algorithm checksum plumbing
Deletes the write-only RequestMetadata.add_crc pipeline (assigned but never read since the port), the dead MinIO-parity Checksum constructors and CompletePart accessor, and key_capitalized (identical to key). The five hand-rolled x-amz-checksum-* response-header if-lets in the streaming and multipart paths collapse into one checksum_header_value helper, ChecksumMode's inherent to_string becomes a Display impl, and checksum.rs drops its file-wide allow blanket now that the file is lint-clean.
Refs rustfs/backlog#1844 (PR2 of 3).
refactor(ecstore): retire the set_disk lint blankets by making the prelude explicit
backlog#1823 step 1 / backlog#2029 road 2. Removes the last two module-level lint blankets in ecstore: set_disk/mod.rs #![allow(unused_imports)] and #![allow(unused_variables)], restoring both lints for the whole 40K-line subtree, and deletes the register line for the unused_variables blanket in the same diff (the guard from #6155 is a bidirectional exact match).
The unused_imports blanket existed because 14 submodules consumed mod.rs as a glob prelude (use super::* / use super::super::*), and rustc does not track consumption through glob re-exports. Each glob is now an explicit use super::{...} list, keeping mod.rs as the single import hub while making every import lint-checkable. Names consumed only by test or test-util units carry #[cfg(test)] / #[cfg(all(test, feature = "test-util"))] / #[cfg(any(test, feature = "test-util"))] gates matching their consumers; storage-api traits are routed through the storage_api_contracts facade per the architecture guard.
The sweep then deleted the genuinely dead imports the blanket was hiding (chrono::Utc, glob::Pattern, futures::task::AtomicWaker, rustfs_lock LocalLock, AsyncBatchProcessor, rand::Rng, std::future::Future among others in mod.rs, plus stale scoped imports and one empty test module shell across the subtree). One unused_variables finding surfaced: flush_read_version_coalescer_pending's lane_key is read only by the #[cfg(test)] counter block, handled with the cfg(not(test)) let _ pattern established in #6158.
Verification: cargo check zero warnings versus the 9cf276ed2 baseline on five lanes (default lib / --tests / rio-v2 --tests / test-util --tests / test-util,rio-v2 --tests; the --tests lane keeps the same three pre-existing core/pools.rs and store/object.rs dead-code warnings main already has); clippy --lib --tests -D warnings clean with test-util,rio-v2; cargo nextest run 4567 passed; make pre-commit exit 0.
Zhengchao An