mirror of
https://github.com/nocobase/nocobase.git
synced 2026-09-01 14:57:36 +08:00
docs: clarify file storage access options (#9969)
This commit is contained in:
Junyi
@@ -41,6 +41,11 @@
|
||||
"type": "custom-link",
|
||||
"label": "S3 Pro",
|
||||
"link": "/file-manager/storage/s3-pro"
|
||||
},
|
||||
{
|
||||
"type": "custom-link",
|
||||
"label": "迁移到 S3 Pro",
|
||||
"link": "/file-manager/storage/migrate-to-s3-pro"
|
||||
}
|
||||
]
|
||||
},
|
||||
|
||||
@@ -7,6 +7,12 @@ keywords: "附件字段,field attachment,文件上传,MIME 类型,存储引擎,a
|
||||
|
||||
# 附件字段
|
||||
|
||||
:::warning 注意
|
||||
|
||||
附件字段将在未来版本中废弃。新建应用或调整现有业务时，建议尽早迁移到自定义文件表，并通过关系字段管理相关文件。
|
||||
|
||||
:::
|
||||
|
||||
## 介绍
|
||||
|
||||
系统内置“附件”类型的字段，用于在自定义数据表中支持用户上传文件。
|
||||
|
||||
@@ -9,6 +9,14 @@ keywords: "阿里云 OSS,阿里云存储,AccessKey,存储桶,对象存储,OSS
|
||||
|
||||
基于阿里云 OSS 的存储引擎，使用前需要准备相关账号和权限。
|
||||
|
||||
:::warning 注意
|
||||
|
||||
该引擎不支持私有访问。文件上传后，NocoBase 会生成可直接访问的 URL，任何获得该 URL 的用户都可以访问文件。
|
||||
|
||||
即使 OSS bucket 本身配置为私有，NocoBase 内置的阿里云 OSS 引擎也不会为文件访问生成临时签名 URL。如果需要私有访问，请使用支持签名 URL 的 [S3 Pro](./s3-pro) 存储引擎。已有历史文件时，可参考[迁移到 S3 Pro](./migrate-to-s3-pro.md)。
|
||||
|
||||
:::
|
||||
|
||||
## 配置参数
|
||||
|
||||
|
||||
|
||||
@@ -9,6 +9,14 @@ keywords: "Amazon S3,AWS,存储桶,AccessKey,云存储,S3 配置,NocoBase"
|
||||
|
||||
基于 Amazon S3 的存储引擎，使用前需要准备相关账号和权限。
|
||||
|
||||
:::warning 注意
|
||||
|
||||
该引擎不支持私有访问。文件上传后，NocoBase 会生成可直接访问的 URL，任何获得该 URL 的用户都可以访问文件。
|
||||
|
||||
即使 S3 bucket 本身配置为私有，NocoBase 内置的 Amazon S3 引擎也不会为文件访问生成临时签名 URL。如果需要私有访问，请使用支持签名 URL 的 [S3 Pro](./s3-pro) 存储引擎。已有历史文件时，可参考[迁移到 S3 Pro](./migrate-to-s3-pro.md)。
|
||||
|
||||
:::
|
||||
|
||||
## 配置参数
|
||||
|
||||
|
||||
|
||||
@@ -25,6 +25,28 @@ keywords: "存储引擎,Storage,本地存储,S3,OSS,COS,文件大小限制,MIME
|
||||
|
||||
系统安装时会自动添加一个本地存储引擎，可直接使用。也可以添加新的或编辑已有的引擎参数。
|
||||
|
||||
## 文件可访问性
|
||||
|
||||
不同存储引擎对文件访问控制的支持不同。配置前先确认文件是否需要私有访问：
|
||||
|
||||
| 存储引擎 | 文件可访问性 |
|
||||
| --- | --- |
|
||||
| [本地存储](./local) | 仅支持公开访问，不支持私有访问 |
|
||||
| [Amazon S3](./amazon-s3) | 仅支持公开访问，不支持私有访问 |
|
||||
| [阿里云 OSS](./aliyun-oss) | 仅支持公开访问，不支持私有访问 |
|
||||
| [腾讯云 COS](./tencent-cos) | 仅支持公开访问，不支持私有访问 |
|
||||
| [S3 Pro](./s3-pro) | 支持私有访问，可通过临时签名 URL 访问文件 |
|
||||
|
||||
:::warning 注意
|
||||
|
||||
本地存储、Amazon S3、阿里云 OSS、腾讯云 COS 这几个内置引擎不会为文件访问做登录鉴权，也不会生成临时签名 URL。文件上传后，任何获得文件访问 URL 的用户都可以直接访问该文件。
|
||||
|
||||
如果需要保存合同、证件、内部资料等不应公开的文件，请使用 [S3 Pro](./s3-pro) 并开启私有访问。
|
||||
|
||||
:::
|
||||
|
||||
如果你已经在使用仅支持公开访问的存储引擎，并希望把历史文件迁移到 S3 Pro，可参考[迁移到 S3 Pro](./migrate-to-s3-pro.md)。
|
||||
|
||||
## 通用参数
|
||||
|
||||
除了不同引擎类别的特有参数外，以下部分为通用参数（以本地存储为例）：
|
||||
|
||||
@@ -9,6 +9,14 @@ keywords: "本地存储,Local Storage,服务器硬盘,存储路径,文件存储,
|
||||
|
||||
上传文件将保存在服务器本地硬盘目录中，适用于系统管理的上传文件总量较少或试验性的场景。
|
||||
|
||||
:::warning 注意
|
||||
|
||||
本地存储不支持私有访问。文件上传后，NocoBase 会生成可直接访问的 URL，任何获得该 URL 的用户都可以访问文件。
|
||||
|
||||
如果需要保存合同、证件、内部资料等不应公开的文件，请使用支持私有访问的 [S3 Pro](./s3-pro) 存储引擎。已有历史文件时，可参考[迁移到 S3 Pro](./migrate-to-s3-pro.md)。
|
||||
|
||||
:::
|
||||
|
||||
## 配置参数
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,271 @@
|
||||
---
|
||||
title: "迁移到 S3 Pro"
|
||||
description: "将本地存储、Amazon S3、阿里云 OSS、腾讯云 COS 等仅支持公开访问的存储引擎迁移到 S3 Pro，覆盖维护窗口、备份、对象 key 映射、文件迁移、记录更新和风险检查。"
|
||||
keywords: "S3 Pro,存储迁移,私有访问,文件迁移,本地存储,S3,OSS,COS,NocoBase"
|
||||
---
|
||||
|
||||
# 迁移到 S3 Pro
|
||||
|
||||
如果你已经使用了[本地存储](./local.md)、[Amazon S3](./amazon-s3.md)、[阿里云 OSS](./aliyun-oss.md) 或 [腾讯云 COS](./tencent-cos.md)，需要把文件改为私有访问，可以把历史文件迁移到 [S3 Pro](./s3-pro.md)。
|
||||
|
||||
迁移需要同时处理三件事：把物理文件或对象复制到新的存储桶、更新文件表里的存储记录、切换以后上传使用的新存储。
|
||||
|
||||
:::warning 注意
|
||||
|
||||
迁移前务必在测试环境演练一遍，并准备可回滚的数据库备份和文件备份。迁移期间如果继续上传或删除附件，容易出现漏迁、覆盖或记录不一致。
|
||||
|
||||
:::
|
||||
|
||||
## 适用范围
|
||||
|
||||
这篇文档适用于从以下存储引擎迁移到 S3 Pro：
|
||||
|
||||
| 原存储引擎 | 历史文件位置 |
|
||||
| --- | --- |
|
||||
| 本地存储 | `<documentRoot>/<record.path>/<record.filename>` |
|
||||
| Amazon S3 | 原 bucket 中的 `join(record.path, record.filename)` |
|
||||
| 阿里云 OSS | 原 bucket 中的 `join(record.path, record.filename)` |
|
||||
| 腾讯云 COS | 原 bucket 中的 `join(record.path, record.filename)` |
|
||||
|
||||
其中，`record.path` 和 `record.filename` 来自文件记录本身，包括内置的 `attachments` 表和其他文件表。
|
||||
|
||||
:::tip 提示
|
||||
|
||||
本地存储的 `documentRoot` 以 `storages` 表中该存储的 `options.documentRoot` 为准。默认配置值是 `storage/uploads`，实际绝对路径取决于 NocoBase 运行时的 storage 目录。
|
||||
|
||||
:::
|
||||
|
||||
## 操作步骤
|
||||
|
||||
### 第一步：停写或进入维护窗口
|
||||
|
||||
迁移期间先暂停用户上传、更新和删除附件。可以通过维护窗口、临时下线入口、冻结相关业务流程等方式实现。
|
||||
|
||||
这一步的目标是让文件记录和物理文件保持静止。如果迁移过程中仍有用户上传或删除文件，那么迁移脚本统计到的记录可能已经不是最新状态。
|
||||
|
||||
### 第二步：备份数据库和文件
|
||||
|
||||
至少准备两类备份：
|
||||
|
||||
1. 数据库备份
|
||||
2. 原存储中的文件备份或对象快照
|
||||
|
||||
对于本地存储，需要备份 `documentRoot` 对应的目录。历史文件的物理路径通常是：
|
||||
|
||||
```text
|
||||
<documentRoot>/<record.path>/<record.filename>
|
||||
```
|
||||
|
||||
对于 Amazon S3、阿里云 OSS、腾讯云 COS，需要确认原 bucket 的对象仍可读取，并记录原存储引擎的 `id`、`name`、`type`、`path`、`baseUrl` 和 `options`。
|
||||
|
||||
另外建议导出一份迁移清单，用于回滚和人工核对：
|
||||
|
||||
```text
|
||||
collection
|
||||
id
|
||||
oldStorageId
|
||||
oldPath
|
||||
oldFilename
|
||||
oldUrl
|
||||
newKey
|
||||
size
|
||||
mimetype
|
||||
```
|
||||
|
||||
### 第三步：创建并验证 S3 Pro 存储
|
||||
|
||||
按 [S3 Pro](./s3-pro.md) 文档创建新的 `s3-pro` 存储。至少确认这些配置可用：
|
||||
|
||||
- bucket
|
||||
- endpoint
|
||||
- region
|
||||
- accessKey
|
||||
- secret
|
||||
- public / private
|
||||
- access endpoint
|
||||
- forcePathStyle
|
||||
- 文件大小和 MIME 类型规则
|
||||
|
||||
创建后先用新存储上传一个测试文件，确认上传、预览、下载、删除都正常。如果目标是私有访问，还要确认访问 URL 是临时签名 URL，并且过期时间符合预期。
|
||||
|
||||
:::warning 注意
|
||||
|
||||
S3 Pro 使用客户端直传。目标 bucket 需要配置允许 NocoBase 站点上传的 CORS 规则，否则新文件上传会失败。
|
||||
|
||||
:::
|
||||
|
||||
### 第四步：确定对象 key 映射
|
||||
|
||||
历史文件迁移到 S3 Pro 后，建议继续使用原来的相对路径作为 S3 object key：
|
||||
|
||||
```text
|
||||
oldKey = join(record.path, record.filename)
|
||||
```
|
||||
|
||||
比如：
|
||||
|
||||
```text
|
||||
record.path = "avatars"
|
||||
record.filename = "a-123.png"
|
||||
oldKey = "avatars/a-123.png"
|
||||
```
|
||||
|
||||
S3 Pro 针对已经落库的文件记录，访问文件时会直接把 `file.filename` 当作完整 object key，不会再拼接 `file.path`。所以迁移后文件记录应更新为：
|
||||
|
||||
```text
|
||||
storageId = <s3-pro-storage-id>
|
||||
filename = <oldKey>
|
||||
path = ""
|
||||
url = ""
|
||||
```
|
||||
|
||||
不要迁移成下面这种形式：
|
||||
|
||||
```text
|
||||
filename = "a-123.png"
|
||||
path = "avatars"
|
||||
```
|
||||
|
||||
否则 S3 Pro 可能只把 `a-123.png` 当作 object key，导致历史文件访问失败。
|
||||
|
||||
生成 object key 时要使用 `/` 作为分隔符，不要使用操作系统的路径分隔符。对于以 `/` 开头的旧路径，也要去掉开头的 `/`。
|
||||
|
||||
### 第五步：迁移物理文件或确认对象位置
|
||||
|
||||
遍历所有文件表记录，包括内置的 `attachments` 和其他文件表。只处理 `storageId = <old-storage-id>` 的记录。
|
||||
|
||||
如果原存储是本地存储，需要把本地文件上传到 S3 Pro 使用的 bucket。如果原存储本来就是 Amazon S3、阿里云 OSS 或腾讯云 COS，并且新的 S3 Pro 配置仍指向同一个 bucket、同一个 endpoint，且访问凭证有权限读取同一批对象，通常不需要复制对象。此时只要确认第四步生成的 `oldKey` 能被 S3 Pro 正确访问即可。
|
||||
|
||||
不同原存储的处理方式如下：
|
||||
|
||||
| 原存储引擎 | 是否需要复制文件 | 目标 object key |
|
||||
| --- | --- | --- |
|
||||
| 本地存储 | 需要上传到 S3 Pro bucket | `join(record.path, record.filename)` |
|
||||
| Amazon S3 | 复用同一 bucket 时通常不需要；换 bucket、换账号或换区域时需要复制 | `join(record.path, record.filename)` |
|
||||
| 阿里云 OSS | 复用同一 bucket 时通常不需要；换 bucket、换账号或换区域时需要复制 | `join(record.path, record.filename)` |
|
||||
| 腾讯云 COS | 复用同一 bucket 时通常不需要；换 bucket、换账号或换区域时需要复制 | `join(record.path, record.filename)` |
|
||||
|
||||
以下情况仍然需要复制文件或对象：
|
||||
|
||||
- 从本地存储迁移到 S3 Pro
|
||||
- 换 bucket、换账号、换 region 或换云厂商
|
||||
- 需要把历史对象从公开 bucket 搬到新的私有 bucket
|
||||
- 原对象 key 需要重命名或重新组织目录
|
||||
- 原 bucket 权限策略不适合 S3 Pro 的签名访问或客户端直传
|
||||
- S3 Pro 无法通过当前 endpoint 和凭证直接访问原对象
|
||||
|
||||
正式迁移前先做 dry-run。至少输出：
|
||||
|
||||
- 待迁移记录数
|
||||
- 待迁移总大小
|
||||
- 本地缺失文件数或源对象缺失数
|
||||
- 重复 object key 数
|
||||
- 不能识别 `storageId` 的记录数
|
||||
- 待人工处理列表
|
||||
|
||||
如果出现重复 object key，不要直接覆盖。先比较文件大小、ETag 或 hash，确认它们是否指向同一个文件。不是同一个文件时，需要为其中一条记录生成新的 key，并在后续记录更新时使用这个新 key。
|
||||
|
||||
### 第六步：校验目标对象可访问
|
||||
|
||||
文件复制完成后，或确认可以复用原云存储 bucket 后，对每条迁移记录执行 S3 HEAD Object 或等价检查，确认 S3 Pro 能通过目标 object key 访问对象。
|
||||
|
||||
建议输出这些结果：
|
||||
|
||||
- 成功数
|
||||
- 源文件缺失数
|
||||
- 上传或复制失败数
|
||||
- 目标对象缺失数
|
||||
- 重复 key 数
|
||||
- 待人工处理列表
|
||||
|
||||
不要只看脚本退出码。对象存储可能出现部分失败、重试后成功、同名覆盖、权限可写但不可读等情况，迁移清单和 HEAD Object 校验更可靠。
|
||||
|
||||
### 第七步：更新文件记录
|
||||
|
||||
确认目标对象全部存在后，再更新文件表记录。只更新文件集合里的文件记录，保留原记录 `id` 不变，不需要更新附件字段产生的多对多关系表。
|
||||
|
||||
核心字段如下：
|
||||
|
||||
| 字段 | 迁移后取值 |
|
||||
| --- | --- |
|
||||
| `storageId` | 新的 S3 Pro 存储 ID |
|
||||
| `filename` | `join(oldPath, oldFilename)` |
|
||||
| `path` | 空字符串 |
|
||||
| `url` | 私有存储填空字符串；公开存储可填公开 URL 或空字符串 |
|
||||
|
||||
如果 S3 Pro 是私有存储，`url` 必须为空。S3 Pro 对非公开存储保存记录时也会清空 `url`，访问文件时会动态生成临时签名 URL。
|
||||
|
||||
:::warning 注意
|
||||
|
||||
建议在事务中批量更新记录，并把第二步导出的迁移清单保存到迁移完成之后。回滚时需要用这份清单把 `storageId`、`path`、`filename` 和 `url` 改回旧值。
|
||||
|
||||
:::
|
||||
|
||||
### 第八步：切换以后上传使用的存储
|
||||
|
||||
历史记录更新完成后，还要切换新文件的上传目标。
|
||||
|
||||
需要检查三类配置：
|
||||
|
||||
1. 默认存储引擎：如果附件字段或文件表没有指定存储，则会使用默认存储。需要把新的 S3 Pro 存储设为默认。
|
||||
2. 附件字段：如果某些附件字段配置了 `options.storage = <old-storage-name>`，需要改成新的 S3 Pro 存储 `name`。
|
||||
3. 文件表：如果某些文件表配置了旧存储引擎，也需要改成新的 S3 Pro 存储 `name`。
|
||||
|
||||
这里使用的是存储引擎的 `name`，不是 `id`。历史文件记录里的 `storageId` 才使用存储引擎 ID。
|
||||
|
||||
### 第九步：重启或刷新缓存
|
||||
|
||||
文件管理器会缓存 `storages` 配置。通过 NocoBase 界面或资源 API 更新存储配置时，通常会触发 `reloadStorages`。如果你是直接改数据库，迁移完成后需要重启应用，或确保已经触发存储缓存刷新。
|
||||
|
||||
如果你直接修改了附件字段或文件表的元数据配置，也建议重启应用，确保集合配置、字段配置和前端缓存都使用新存储。
|
||||
|
||||
### 第十步：抽样验证并保留旧文件
|
||||
|
||||
迁移完成后至少验证这些场景：
|
||||
|
||||
- 附件字段中查看、预览、下载历史文件
|
||||
- 文件表中查看、预览、下载历史文件
|
||||
- 图片缩略图或预处理参数
|
||||
- Office 文件预览等依赖外部服务读取 URL 的能力
|
||||
- 新上传文件的保存、预览、下载和删除
|
||||
- 删除文件记录后的对象删除行为
|
||||
|
||||
旧存储中的文件不要马上删除。建议保留到业务方确认、备份周期覆盖、访问日志没有异常之后，再分批清理旧文件。
|
||||
|
||||
## 风险点
|
||||
|
||||
### 写入期间的数据不一致
|
||||
|
||||
迁移期间如果仍有用户上传或删除文件，可能出现对象已复制但记录被删除、记录已更新但对象还在旧存储、或新上传文件没有进入迁移清单等问题。默认做法是进入维护窗口。
|
||||
|
||||
### S3 Pro 的 object key 规则不同
|
||||
|
||||
迁移后 `filename` 应保存完整 object key，`path` 应为空。这个规则跟本地存储和内置 S3 / OSS / COS 的历史记录形态不同，是迁移里最容易出错的地方。
|
||||
|
||||
### URL 可访问性变化
|
||||
|
||||
从公开访问存储迁移到私有 S3 Pro 后，历史公开 URL 可能不再长期有效。NocoBase 内部读取文件会动态生成临时签名 URL，不过外部系统里已经保存的旧 URL 不会自动改写。
|
||||
|
||||
如果有第三方系统、邮件模板、导出文件或富文本内容直接保存了旧 URL，需要单独评估替换策略。
|
||||
|
||||
:::warning 注意
|
||||
|
||||
如果 Markdown（Vditor）字段内容中已经保存了文件 URL，需要单独处理这些内容。S3 Pro 私有访问生成的是临时签名 URL，会过期，目前不支持把这类私有链接长期存储在 Markdown（Vditor）字段中使用。
|
||||
|
||||
如果这些字段必须继续引用文件，建议先保留公开访问的文件地址。
|
||||
|
||||
:::
|
||||
|
||||
### 预览服务和私有文件
|
||||
|
||||
某些预览能力依赖外部服务访问文件 URL。私有 S3 Pro 会生成临时签名 URL，通常可以访问，但会受到签名过期时间、网络可达性、bucket 权限和服务端缓存影响。
|
||||
|
||||
如果文件非常敏感，需要重新评估是否允许外部预览服务读取这些文件。
|
||||
|
||||
### 直接改数据库不会触发所有钩子
|
||||
|
||||
如果用 SQL 或自写脚本直接更新 `storages`、集合配置、字段配置和文件记录，NocoBase 的部分缓存刷新和保存钩子不会自动执行。更新完成后要重启应用，并重新验证存储配置。
|
||||
|
||||
### 删除行为会指向新存储
|
||||
|
||||
文件记录的 `storageId` 更新为 S3 Pro 后，后续删除文件记录时，NocoBase 会尝试删除新 bucket 中的对象。旧存储中的对象不会随之删除，需要单独清理。
|
||||
@@ -9,6 +9,14 @@ keywords: "腾讯云 COS,腾讯云存储,SecretId,SecretKey,存储桶,对象存
|
||||
|
||||
基于腾讯云 COS 的存储引擎，使用前需要准备相关账号和权限。
|
||||
|
||||
:::warning 注意
|
||||
|
||||
该引擎不支持私有访问。文件上传后，NocoBase 会生成可直接访问的 URL，任何获得该 URL 的用户都可以访问文件。
|
||||
|
||||
即使 COS bucket 本身配置为私有，NocoBase 内置的腾讯云 COS 引擎也不会为文件访问生成临时签名 URL。如果需要私有访问，请使用支持签名 URL 的 [S3 Pro](./s3-pro) 存储引擎。已有历史文件时，可参考[迁移到 S3 Pro](./migrate-to-s3-pro.md)。
|
||||
|
||||
:::
|
||||
|
||||
## 配置参数
|
||||
|
||||
|
||||
|
||||
@@ -44,6 +44,11 @@
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "custom-link",
|
||||
"label": "Migration zu S3 Pro",
|
||||
"link": "/file-manager/storage/migrate-to-s3-pro"
|
||||
},
|
||||
{
|
||||
"type": "custom-link",
|
||||
"label": "Dateitabelle",
|
||||
|
||||
@@ -1,5 +1,11 @@
|
||||
# Anlagenfeld
|
||||
|
||||
:::warning Hinweis
|
||||
|
||||
Das Anhangsfeld wird in einer zukünftigen Version eingestellt. Verwenden Sie für neue Anwendungen oder Anpassungen bestehender Geschäftsprozesse möglichst früh eine benutzerdefinierte Dateitabelle und verwalten Sie zugehörige Dateien über Beziehungsfelder.
|
||||
|
||||
:::
|
||||
|
||||
## Einführung
|
||||
|
||||
Das System verfügt über einen integrierten Feldtyp „Anhang“, der das Hochladen von Dateien in benutzerdefinierten Sammlungen ermöglicht.
|
||||
|
||||
@@ -2,6 +2,15 @@
|
||||
|
||||
Die Speicher-Engine basiert auf Aliyun OSS. Bevor Sie sie verwenden, müssen Sie die entsprechenden Konten und Berechtigungen vorbereiten.
|
||||
|
||||
|
||||
:::warning Hinweis
|
||||
|
||||
Diese Engine unterstützt keinen privaten Zugriff. Nach dem Hochladen erzeugt NocoBase eine direkt zugängliche URL, und jeder, der diese URL besitzt, kann auf die Datei zugreifen.
|
||||
|
||||
Auch wenn der OSS-Bucket selbst privat ist, erzeugt die integrierte Aliyun OSS-Engine keine temporären signierten URLs für den Dateizugriff. Wenn Sie privaten Zugriff benötigen, verwenden Sie [S3 Pro](./s3-pro). Wenn bereits historische Dateien vorhanden sind, lesen Sie [Migration zu S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
:::
|
||||
|
||||
## Konfigurationsparameter
|
||||
|
||||
|
||||
|
||||
@@ -2,6 +2,15 @@
|
||||
|
||||
Die Amazon S3 Speicher-Engine erfordert vor der Verwendung die Einrichtung der entsprechenden Konten und Berechtigungen.
|
||||
|
||||
|
||||
:::warning Hinweis
|
||||
|
||||
Diese Engine unterstützt keinen privaten Zugriff. Nach dem Hochladen erzeugt NocoBase eine direkt zugängliche URL, und jeder, der diese URL besitzt, kann auf die Datei zugreifen.
|
||||
|
||||
Auch wenn der S3-Bucket selbst privat ist, erzeugt die integrierte Amazon S3-Engine keine temporären signierten URLs für den Dateizugriff. Wenn Sie privaten Zugriff benötigen, verwenden Sie [S3 Pro](./s3-pro). Wenn bereits historische Dateien vorhanden sind, lesen Sie [Migration zu S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
:::
|
||||
|
||||
## Konfigurationsparameter
|
||||
|
||||
|
||||
|
||||
@@ -18,6 +18,29 @@ NocoBase unterstützt derzeit die folgenden integrierten Engine-Typen:
|
||||
|
||||
Bei der Systeminstallation wird automatisch eine lokale Speicher-Engine hinzugefügt, die Sie direkt verwenden können. Sie haben auch die Möglichkeit, neue Engines hinzuzufügen oder die Parameter bestehender Engines zu bearbeiten.
|
||||
|
||||
|
||||
Wenn Sie bereits eine Speicher-Engine verwenden, die nur öffentlichen Zugriff unterstützt, und historische Dateien zu S3 Pro migrieren möchten, lesen Sie [Migration zu S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
## Dateizugriff
|
||||
|
||||
Verschiedene Speicher-Engines unterstützen unterschiedliche Zugriffskontrollen für Dateien. Prüfen Sie vor der Konfiguration, ob Ihre Dateien privaten Zugriff benötigen:
|
||||
|
||||
| Speicher-Engine | Dateizugriff |
|
||||
| --- | --- |
|
||||
| [Local Storage](./local) | Nur öffentlicher Zugriff wird unterstützt; privater Zugriff wird nicht unterstützt |
|
||||
| [Amazon S3](./amazon-s3) | Nur öffentlicher Zugriff wird unterstützt; privater Zugriff wird nicht unterstützt |
|
||||
| [Aliyun OSS](./aliyun-oss) | Nur öffentlicher Zugriff wird unterstützt; privater Zugriff wird nicht unterstützt |
|
||||
| [Tencent COS](./tencent-cos) | Nur öffentlicher Zugriff wird unterstützt; privater Zugriff wird nicht unterstützt |
|
||||
| [S3 Pro](./s3-pro) | Privater Zugriff wird über temporäre signierte URLs unterstützt |
|
||||
|
||||
:::warning Hinweis
|
||||
|
||||
Lokaler Speicher, Amazon S3, Aliyun OSS und Tencent COS führen keine Login-Prüfung für den Dateizugriff durch und erzeugen keine temporären signierten URLs. Nach dem Hochladen kann jeder, der die Datei-URL besitzt, direkt auf die Datei zugreifen.
|
||||
|
||||
Wenn Sie Verträge, Ausweisdokumente, interne Unterlagen oder andere nicht öffentliche Dateien speichern müssen, verwenden Sie [S3 Pro](./s3-pro) und aktivieren Sie privaten Zugriff.
|
||||
|
||||
:::
|
||||
|
||||
## Allgemeine Parameter
|
||||
|
||||
Neben den spezifischen Parametern für die verschiedenen Engine-Typen sind die folgenden Abschnitte allgemeine Parameter (am Beispiel des lokalen Speichers):
|
||||
|
||||
@@ -2,6 +2,15 @@
|
||||
|
||||
Hochgeladene Dateien werden direkt auf der lokalen Festplatte des Servers gespeichert. Dies ist ideal für Szenarien, in denen das System eine geringe Gesamtmenge an hochgeladenen Dateien verwaltet oder für experimentelle Zwecke.
|
||||
|
||||
|
||||
:::warning Hinweis
|
||||
|
||||
Lokaler Speicher unterstützt keinen privaten Zugriff. Nach dem Hochladen erzeugt NocoBase eine direkt zugängliche URL, und jeder, der diese URL besitzt, kann auf die Datei zugreifen.
|
||||
|
||||
Wenn Sie Verträge, Ausweisdokumente, interne Unterlagen oder andere nicht öffentliche Dateien speichern müssen, verwenden Sie [S3 Pro](./s3-pro). Wenn bereits historische Dateien vorhanden sind, lesen Sie [Migration zu S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
:::
|
||||
|
||||
## Konfigurationsparameter
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,270 @@
|
||||
---
|
||||
pkg: '@nocobase/plugin-file-storage-s3-pro'
|
||||
title: "Migration zu S3 Pro"
|
||||
description: "Migrate files from public-only storage engines to S3 Pro, including maintenance windows, backups, object key mapping, record updates, and risk checks."
|
||||
keywords: "S3 Pro,storage migration,private access,file migration,Local Storage,S3,OSS,COS,NocoBase"
|
||||
---
|
||||
|
||||
# Migration zu S3 Pro
|
||||
|
||||
If you already use [Local Storage](./local.md), [Amazon S3](./amazon-s3.md), [Aliyun OSS](./aliyun-oss.md), or [Tencent COS](./tencent-cos.md), and later need private file access, you can migrate historical files to [S3 Pro](./s3-pro.md).
|
||||
|
||||
This is not only a storage configuration change. A migration must handle three things: make the physical files or objects available from the new storage, update file records, and switch future uploads to the new storage.
|
||||
|
||||
:::warning Hinweis
|
||||
|
||||
Run the migration once in a test environment first, and prepare restorable database and file backups. If users continue to upload or delete attachments during the migration, records can be missed, overwritten, or become inconsistent.
|
||||
|
||||
:::
|
||||
|
||||
## Scope
|
||||
|
||||
This guide applies when migrating from these storage engines to S3 Pro:
|
||||
|
||||
| Source storage engine | Historical file location |
|
||||
| --- | --- |
|
||||
| Local Storage | `<documentRoot>/<record.path>/<record.filename>` |
|
||||
| Amazon S3 | `join(record.path, record.filename)` in the source bucket |
|
||||
| Aliyun OSS | `join(record.path, record.filename)` in the source bucket |
|
||||
| Tencent COS | `join(record.path, record.filename)` in the source bucket |
|
||||
|
||||
Here, `record.path` and `record.filename` come from the file record itself, including the built-in `attachments` table and other file collections.
|
||||
|
||||
:::tip Hinweis
|
||||
|
||||
For Local Storage, `documentRoot` must come from `options.documentRoot` in the corresponding `storages` record. The default value is `storage/uploads`, but the final absolute path depends on the runtime storage directory of NocoBase.
|
||||
|
||||
:::
|
||||
|
||||
## Operation Steps
|
||||
|
||||
### Step 1: Stop writes or enter a maintenance window
|
||||
|
||||
Pause user uploads, updates, and deletions of attachments during the migration. You can use a maintenance window, temporarily disable entry points, or freeze related business flows.
|
||||
|
||||
The goal is to keep file records and physical files stable. If users upload or delete files during migration, the migration script may no longer be working against the latest records.
|
||||
|
||||
### Step 2: Back up the database and files
|
||||
|
||||
Prepare at least two backups:
|
||||
|
||||
1. Database backup
|
||||
2. File backup or object snapshot from the source storage
|
||||
|
||||
For Local Storage, back up the directory pointed to by `documentRoot`. Historical files are usually stored at:
|
||||
|
||||
```text
|
||||
<documentRoot>/<record.path>/<record.filename>
|
||||
```
|
||||
|
||||
For Amazon S3, Aliyun OSS, and Tencent COS, confirm that objects in the source bucket are still readable, and record the source storage engine's `id`, `name`, `type`, `path`, `baseUrl`, and `options`.
|
||||
|
||||
It is also recommended to export a migration manifest for rollback and manual review:
|
||||
|
||||
```text
|
||||
collection
|
||||
id
|
||||
oldStorageId
|
||||
oldPath
|
||||
oldFilename
|
||||
oldUrl
|
||||
newKey
|
||||
size
|
||||
mimetype
|
||||
```
|
||||
|
||||
### Step 3: Create and verify the S3 Pro storage
|
||||
|
||||
Create a new `s3-compatible` storage according to [S3 Pro](./s3-pro.md). At least verify these settings:
|
||||
|
||||
- bucket
|
||||
- endpoint
|
||||
- region
|
||||
- accessKey
|
||||
- secret
|
||||
- public / private
|
||||
- access endpoint
|
||||
- forcePathStyle
|
||||
- file size and MIME type rules
|
||||
|
||||
Upload a test file with the new storage and confirm upload, preview, download, and deletion all work. If the target is private access, also confirm that the access URL is a temporary signed URL and that its expiration is expected.
|
||||
|
||||
:::warning Hinweis
|
||||
|
||||
S3 Pro uses direct client upload. The target bucket must have CORS rules that allow uploads from the NocoBase site, otherwise new uploads will fail.
|
||||
|
||||
:::
|
||||
|
||||
### Step 4: Decide the object key mapping
|
||||
|
||||
For historical files migrated to S3 Pro, it is recommended to continue using the old relative path as the S3 object key:
|
||||
|
||||
```text
|
||||
oldKey = join(record.path, record.filename)
|
||||
```
|
||||
|
||||
For example:
|
||||
|
||||
```text
|
||||
record.path = "avatars"
|
||||
record.filename = "a-123.png"
|
||||
oldKey = "avatars/a-123.png"
|
||||
```
|
||||
|
||||
For persisted file records, S3 Pro uses `file.filename` directly as the full object key when accessing the file. It does not append `file.path`. Therefore, migrated records should be updated to:
|
||||
|
||||
```text
|
||||
storageId = <s3-pro-storage-id>
|
||||
filename = <oldKey>
|
||||
path = ""
|
||||
url = ""
|
||||
```
|
||||
|
||||
Do not migrate to this shape:
|
||||
|
||||
```text
|
||||
filename = "a-123.png"
|
||||
path = "avatars"
|
||||
```
|
||||
|
||||
Otherwise S3 Pro may treat only `a-123.png` as the object key, causing historical files to fail to load.
|
||||
|
||||
Use `/` as the separator when generating object keys. Do not use operating-system-specific path separators. If an old path starts with `/`, remove the leading slash.
|
||||
|
||||
### Step 5: Move physical files or confirm object location
|
||||
|
||||
Traverse all file records, including the built-in `attachments` table and other file collections. Only process records where `storageId = <old-storage-id>`.
|
||||
|
||||
If the source is Local Storage, upload the local files to the bucket used by S3 Pro. If the source is already Amazon S3, Aliyun OSS, or Tencent COS, and the new S3 Pro storage points to the same bucket and endpoint, with credentials that can read the same objects, you usually do not need to copy objects. In that case, confirm that the `oldKey` generated in Step 4 is accessible through S3 Pro.
|
||||
|
||||
| Source storage engine | Whether files need to be copied | Target object key |
|
||||
| --- | --- | --- |
|
||||
| Local Storage | Must be uploaded to the S3 Pro bucket | `join(record.path, record.filename)` |
|
||||
| Amazon S3 | Usually no copy if reusing the same bucket; copy is needed when changing bucket, account, or region | `join(record.path, record.filename)` |
|
||||
| Aliyun OSS | Usually no copy if reusing the same bucket; copy is needed when changing bucket, account, or region | `join(record.path, record.filename)` |
|
||||
| Tencent COS | Usually no copy if reusing the same bucket; copy is needed when changing bucket, account, or region | `join(record.path, record.filename)` |
|
||||
|
||||
Files or objects still need to be copied in these cases:
|
||||
|
||||
- Migrating from Local Storage to S3 Pro
|
||||
- Changing bucket, account, region, or cloud provider
|
||||
- Moving historical objects from a public bucket to a new private bucket
|
||||
- Renaming object keys or reorganizing directories
|
||||
- The source bucket policy is not suitable for S3 Pro signed access or direct client upload
|
||||
- S3 Pro cannot directly access the source objects with the current endpoint and credentials
|
||||
|
||||
Before the real migration, run a dry-run. At least output:
|
||||
|
||||
- Number of records to migrate
|
||||
- Total size to migrate
|
||||
- Number of missing local files or missing source objects
|
||||
- Number of duplicate object keys
|
||||
- Number of records with unrecognized `storageId`
|
||||
- List requiring manual handling
|
||||
|
||||
If duplicate object keys appear, do not overwrite directly. Compare file size, ETag, or hash to confirm whether they point to the same file. If they are different files, generate a new key for one record and use that new key when updating the record.
|
||||
|
||||
### Step 6: Verify target object accessibility
|
||||
|
||||
After files are copied, or after confirming that the source cloud bucket can be reused, run S3 HEAD Object or an equivalent check for every migrated record. Confirm that S3 Pro can access the object through the target object key.
|
||||
|
||||
Recommended output:
|
||||
|
||||
- Success count
|
||||
- Missing source file count
|
||||
- Upload or copy failure count
|
||||
- Missing target object count
|
||||
- Duplicate key count
|
||||
- List requiring manual handling
|
||||
|
||||
Do not rely only on the script exit code. Object storage can have partial failures, successful retries, same-key overwrites, or write-only-but-not-readable permission problems. The migration manifest and HEAD Object checks are more reliable.
|
||||
|
||||
### Step 7: Update file records
|
||||
|
||||
After confirming that all target objects exist, update the file records. Only update records in file collections, keep the original `id`, and do not update many-to-many junction tables generated by attachment fields.
|
||||
|
||||
Core fields:
|
||||
|
||||
| Field | Value after migration |
|
||||
| --- | --- |
|
||||
| `storageId` | New S3 Pro storage ID |
|
||||
| `filename` | `join(oldPath, oldFilename)` |
|
||||
| `path` | Empty string |
|
||||
| `url` | Empty string for private storage; public URL or empty string for public storage |
|
||||
|
||||
If S3 Pro is private, `url` must be empty. S3 Pro also clears `url` before saving non-public storage records. File access will dynamically generate a temporary signed URL.
|
||||
|
||||
:::warning Hinweis
|
||||
|
||||
Batch update records in a transaction if possible, and keep the migration manifest from Step 2 until after the migration is fully verified. Rollback needs that manifest to restore `storageId`, `path`, `filename`, and `url` to their old values.
|
||||
|
||||
:::
|
||||
|
||||
### Step 8: Switch future uploads to the new storage
|
||||
|
||||
After historical records are updated, switch the target storage for new uploads.
|
||||
|
||||
Check three types of configuration:
|
||||
|
||||
1. Default storage engine: if attachment fields or file collections do not specify storage, uploaded files use the default storage. Set the new S3 Pro storage as the default.
|
||||
2. Attachment fields: if an attachment field is configured with `options.storage = <old-storage-name>`, change it to the new S3 Pro storage `name`.
|
||||
3. File collections: if a file collection is configured with the old storage engine, change it to the new S3 Pro storage `name`.
|
||||
|
||||
Here, `name` is the storage engine name, not the storage ID. Historical file records use `storageId`.
|
||||
|
||||
### Step 9: Restart or refresh storage cache
|
||||
|
||||
The file manager caches `storages` configuration. If you update storage configuration through the NocoBase UI or resource API, `reloadStorages` is usually triggered. If you update the database directly, restart the application after migration or make sure the storage cache has been refreshed.
|
||||
|
||||
If you directly changed attachment field metadata or file collection metadata, it is also recommended to restart the application so collection configuration, field configuration, and frontend cache all use the new storage.
|
||||
|
||||
### Step 10: Sample-check and keep old files
|
||||
|
||||
After migration, verify at least these scenarios:
|
||||
|
||||
- View, preview, and download historical files in attachment fields
|
||||
- View, preview, and download historical files in file collections
|
||||
- Image thumbnails or preprocessing parameters
|
||||
- Office preview and other features that depend on external services reading file URLs
|
||||
- Save, preview, download, and delete newly uploaded files
|
||||
- Object deletion after deleting file records
|
||||
|
||||
Do not delete files from the old storage immediately. Keep them until the business side confirms the migration, backup cycles have covered the new state, and access logs show no abnormal behavior.
|
||||
|
||||
## Risks
|
||||
|
||||
### Data inconsistency during writes
|
||||
|
||||
If users can still upload or delete files during migration, objects may be copied after records are deleted, records may be updated while objects remain only in the old storage, or new uploads may not be included in the migration manifest. The default approach is to enter a maintenance window.
|
||||
|
||||
### Different object key rules in S3 Pro
|
||||
|
||||
After migration, `filename` should store the full object key and `path` should be empty. This differs from the historical record shape of Local Storage and built-in S3 / OSS / COS, and is the easiest place to make mistakes.
|
||||
|
||||
### URL accessibility changes
|
||||
|
||||
After migrating from public storage to private S3 Pro, historical public URLs may no longer remain valid. NocoBase dynamically generates temporary signed URLs for internal file access, but old URLs already saved in external systems are not automatically rewritten.
|
||||
|
||||
If third-party systems, email templates, exported files, or rich text content directly stores old URLs, evaluate a separate replacement strategy.
|
||||
|
||||
:::warning Hinweis
|
||||
|
||||
If Markdown (Vditor) field content already stores file URLs, handle those contents separately. Private S3 Pro access generates temporary signed URLs that expire, and currently these private links are not supported for long-term storage and use inside Markdown (Vditor) fields.
|
||||
|
||||
If these fields must continue referencing files, keep publicly accessible file URLs for now.
|
||||
|
||||
:::
|
||||
|
||||
### Preview services and private files
|
||||
|
||||
Some preview features depend on external services accessing file URLs. Private S3 Pro generates temporary signed URLs, which can usually be accessed, but they are affected by signature expiration, network reachability, bucket permissions, and server-side caching.
|
||||
|
||||
If files are very sensitive, reassess whether external preview services should be allowed to read them.
|
||||
|
||||
### Direct database changes do not trigger all hooks
|
||||
|
||||
If you update `storages`, collection configuration, field configuration, and file records directly through SQL or a custom script, some NocoBase cache refreshes and save hooks are not triggered automatically. Restart the application after the update and verify storage configuration again.
|
||||
|
||||
### Delete behavior points to the new storage
|
||||
|
||||
After a file record's `storageId` is updated to S3 Pro, deleting that file record later will make NocoBase try to delete the object from the new bucket. Objects in the old storage will not be deleted automatically and need to be cleaned up separately.
|
||||
@@ -2,6 +2,15 @@
|
||||
|
||||
Dies ist ein Speicher-Engine, das auf Tencent Cloud COS basiert. Bevor Sie es nutzen können, müssen Sie die entsprechenden Konten und Berechtigungen einrichten.
|
||||
|
||||
|
||||
:::warning Hinweis
|
||||
|
||||
Diese Engine unterstützt keinen privaten Zugriff. Nach dem Hochladen erzeugt NocoBase eine direkt zugängliche URL, und jeder, der diese URL besitzt, kann auf die Datei zugreifen.
|
||||
|
||||
Auch wenn der COS-Bucket selbst privat ist, erzeugt die integrierte Tencent COS-Engine keine temporären signierten URLs für den Dateizugriff. Wenn Sie privaten Zugriff benötigen, verwenden Sie [S3 Pro](./s3-pro). Wenn bereits historische Dateien vorhanden sind, lesen Sie [Migration zu S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
:::
|
||||
|
||||
## Konfigurationsparameter
|
||||
|
||||
|
||||
|
||||
@@ -44,6 +44,11 @@
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "custom-link",
|
||||
"label": "Migrate to S3 Pro",
|
||||
"link": "/file-manager/storage/migrate-to-s3-pro"
|
||||
},
|
||||
{
|
||||
"type": "custom-link",
|
||||
"label": "File Collection",
|
||||
|
||||
@@ -1,5 +1,11 @@
|
||||
# Attachment
|
||||
|
||||
:::warning Note
|
||||
|
||||
The Attachment field will be deprecated in a future version. For new apps or when adjusting existing business flows, migrate to a custom file collection as early as possible and manage related files with association fields.
|
||||
|
||||
:::
|
||||
|
||||
## Introduction
|
||||
|
||||
The system has a built-in "Attachment" field type to support file uploads in custom collections.
|
||||
|
||||
@@ -2,6 +2,15 @@
|
||||
|
||||
A storage engine based on Aliyun OSS. Before use, you need to prepare the relevant account and permissions.
|
||||
|
||||
|
||||
:::warning Note
|
||||
|
||||
This engine does not support private access. After a file is uploaded, NocoBase generates a directly accessible URL, and anyone who has that URL can access the file.
|
||||
|
||||
Even if the OSS bucket itself is private, the built-in Aliyun OSS engine does not generate temporary signed URLs for file access. If you need private access, use [S3 Pro](./s3-pro). If historical files already exist, see [Migrate to S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
:::
|
||||
|
||||
## Configuration Parameters
|
||||
|
||||
|
||||
|
||||
@@ -2,6 +2,15 @@
|
||||
|
||||
A storage engine based on Amazon S3. You need to prepare the relevant account and permissions before use.
|
||||
|
||||
|
||||
:::warning Note
|
||||
|
||||
This engine does not support private access. After a file is uploaded, NocoBase generates a directly accessible URL, and anyone who has that URL can access the file.
|
||||
|
||||
Even if the S3 bucket itself is private, the built-in Amazon S3 engine does not generate temporary signed URLs for file access. If you need private access, use [S3 Pro](./s3-pro). If historical files already exist, see [Migrate to S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
:::
|
||||
|
||||
## Configuration Parameters
|
||||
|
||||
|
||||
|
||||
@@ -18,6 +18,29 @@ Currently, NocoBase has built-in support for the following engine types:
|
||||
|
||||
The system automatically adds a local storage engine during installation, which can be used directly. You can also add new engines or edit the parameters of existing ones.
|
||||
|
||||
|
||||
If you are already using a storage engine that only supports public access and want to migrate historical files to S3 Pro, see [Migrate to S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
## File Accessibility
|
||||
|
||||
Different storage engines support different file access controls. Before configuring a storage engine, confirm whether your files need private access:
|
||||
|
||||
| Storage engine | File accessibility |
|
||||
| --- | --- |
|
||||
| [Local Storage](./local) | Only public access is supported; private access is not supported |
|
||||
| [Amazon S3](./amazon-s3) | Only public access is supported; private access is not supported |
|
||||
| [Aliyun OSS](./aliyun-oss) | Only public access is supported; private access is not supported |
|
||||
| [Tencent COS](./tencent-cos) | Only public access is supported; private access is not supported |
|
||||
| [S3 Pro](./s3-pro) | Private access is supported through temporary signed URLs |
|
||||
|
||||
:::warning Note
|
||||
|
||||
Local Storage, Amazon S3, Aliyun OSS, and Tencent COS do not perform login authentication for file access and do not generate temporary signed URLs. After a file is uploaded, anyone who has the file access URL can access it directly.
|
||||
|
||||
If you need to store contracts, identity documents, internal materials, or other files that should not be public, use [S3 Pro](./s3-pro) and enable private access.
|
||||
|
||||
:::
|
||||
|
||||
## Common Parameters
|
||||
|
||||
In addition to the specific parameters for different engine types, the following are common parameters (using local storage as an example):
|
||||
|
||||
@@ -2,6 +2,15 @@
|
||||
|
||||
Uploaded files will be saved in a local directory on the server's hard drive. This is suitable for scenarios with a small total volume of uploaded files managed by the system or for experimental purposes.
|
||||
|
||||
|
||||
:::warning Note
|
||||
|
||||
Local Storage does not support private access. After a file is uploaded, NocoBase generates a directly accessible URL, and anyone who has that URL can access the file.
|
||||
|
||||
If you need to store contracts, identity documents, internal materials, or other files that should not be public, use [S3 Pro](./s3-pro). If historical files already exist, see [Migrate to S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
:::
|
||||
|
||||
## Configuration Parameters
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,270 @@
|
||||
---
|
||||
pkg: '@nocobase/plugin-file-storage-s3-pro'
|
||||
title: "Migrate to S3 Pro"
|
||||
description: "Migrate files from public-only storage engines to S3 Pro, including maintenance windows, backups, object key mapping, record updates, and risk checks."
|
||||
keywords: "S3 Pro,storage migration,private access,file migration,Local Storage,S3,OSS,COS,NocoBase"
|
||||
---
|
||||
|
||||
# Migrate to S3 Pro
|
||||
|
||||
If you already use [Local Storage](./local.md), [Amazon S3](./amazon-s3.md), [Aliyun OSS](./aliyun-oss.md), or [Tencent COS](./tencent-cos.md), and later need private file access, you can migrate historical files to [S3 Pro](./s3-pro.md).
|
||||
|
||||
This is not only a storage configuration change. A migration must handle three things: make the physical files or objects available from the new storage, update file records, and switch future uploads to the new storage.
|
||||
|
||||
:::warning Note
|
||||
|
||||
Run the migration once in a test environment first, and prepare restorable database and file backups. If users continue to upload or delete attachments during the migration, records can be missed, overwritten, or become inconsistent.
|
||||
|
||||
:::
|
||||
|
||||
## Scope
|
||||
|
||||
This guide applies when migrating from these storage engines to S3 Pro:
|
||||
|
||||
| Source storage engine | Historical file location |
|
||||
| --- | --- |
|
||||
| Local Storage | `<documentRoot>/<record.path>/<record.filename>` |
|
||||
| Amazon S3 | `join(record.path, record.filename)` in the source bucket |
|
||||
| Aliyun OSS | `join(record.path, record.filename)` in the source bucket |
|
||||
| Tencent COS | `join(record.path, record.filename)` in the source bucket |
|
||||
|
||||
Here, `record.path` and `record.filename` come from the file record itself, including the built-in `attachments` table and other file collections.
|
||||
|
||||
:::tip Note
|
||||
|
||||
For Local Storage, `documentRoot` must come from `options.documentRoot` in the corresponding `storages` record. The default value is `storage/uploads`, but the final absolute path depends on the runtime storage directory of NocoBase.
|
||||
|
||||
:::
|
||||
|
||||
## Operation Steps
|
||||
|
||||
### Step 1: Stop writes or enter a maintenance window
|
||||
|
||||
Pause user uploads, updates, and deletions of attachments during the migration. You can use a maintenance window, temporarily disable entry points, or freeze related business flows.
|
||||
|
||||
The goal is to keep file records and physical files stable. If users upload or delete files during migration, the migration script may no longer be working against the latest records.
|
||||
|
||||
### Step 2: Back up the database and files
|
||||
|
||||
Prepare at least two backups:
|
||||
|
||||
1. Database backup
|
||||
2. File backup or object snapshot from the source storage
|
||||
|
||||
For Local Storage, back up the directory pointed to by `documentRoot`. Historical files are usually stored at:
|
||||
|
||||
```text
|
||||
<documentRoot>/<record.path>/<record.filename>
|
||||
```
|
||||
|
||||
For Amazon S3, Aliyun OSS, and Tencent COS, confirm that objects in the source bucket are still readable, and record the source storage engine's `id`, `name`, `type`, `path`, `baseUrl`, and `options`.
|
||||
|
||||
It is also recommended to export a migration manifest for rollback and manual review:
|
||||
|
||||
```text
|
||||
collection
|
||||
id
|
||||
oldStorageId
|
||||
oldPath
|
||||
oldFilename
|
||||
oldUrl
|
||||
newKey
|
||||
size
|
||||
mimetype
|
||||
```
|
||||
|
||||
### Step 3: Create and verify the S3 Pro storage
|
||||
|
||||
Create a new `s3-compatible` storage according to [S3 Pro](./s3-pro.md). At least verify these settings:
|
||||
|
||||
- bucket
|
||||
- endpoint
|
||||
- region
|
||||
- accessKey
|
||||
- secret
|
||||
- public / private
|
||||
- access endpoint
|
||||
- forcePathStyle
|
||||
- file size and MIME type rules
|
||||
|
||||
Upload a test file with the new storage and confirm upload, preview, download, and deletion all work. If the target is private access, also confirm that the access URL is a temporary signed URL and that its expiration is expected.
|
||||
|
||||
:::warning Note
|
||||
|
||||
S3 Pro uses direct client upload. The target bucket must have CORS rules that allow uploads from the NocoBase site, otherwise new uploads will fail.
|
||||
|
||||
:::
|
||||
|
||||
### Step 4: Decide the object key mapping
|
||||
|
||||
For historical files migrated to S3 Pro, it is recommended to continue using the old relative path as the S3 object key:
|
||||
|
||||
```text
|
||||
oldKey = join(record.path, record.filename)
|
||||
```
|
||||
|
||||
For example:
|
||||
|
||||
```text
|
||||
record.path = "avatars"
|
||||
record.filename = "a-123.png"
|
||||
oldKey = "avatars/a-123.png"
|
||||
```
|
||||
|
||||
For persisted file records, S3 Pro uses `file.filename` directly as the full object key when accessing the file. It does not append `file.path`. Therefore, migrated records should be updated to:
|
||||
|
||||
```text
|
||||
storageId = <s3-pro-storage-id>
|
||||
filename = <oldKey>
|
||||
path = ""
|
||||
url = ""
|
||||
```
|
||||
|
||||
Do not migrate to this shape:
|
||||
|
||||
```text
|
||||
filename = "a-123.png"
|
||||
path = "avatars"
|
||||
```
|
||||
|
||||
Otherwise S3 Pro may treat only `a-123.png` as the object key, causing historical files to fail to load.
|
||||
|
||||
Use `/` as the separator when generating object keys. Do not use operating-system-specific path separators. If an old path starts with `/`, remove the leading slash.
|
||||
|
||||
### Step 5: Move physical files or confirm object location
|
||||
|
||||
Traverse all file records, including the built-in `attachments` table and other file collections. Only process records where `storageId = <old-storage-id>`.
|
||||
|
||||
If the source is Local Storage, upload the local files to the bucket used by S3 Pro. If the source is already Amazon S3, Aliyun OSS, or Tencent COS, and the new S3 Pro storage points to the same bucket and endpoint, with credentials that can read the same objects, you usually do not need to copy objects. In that case, confirm that the `oldKey` generated in Step 4 is accessible through S3 Pro.
|
||||
|
||||
| Source storage engine | Whether files need to be copied | Target object key |
|
||||
| --- | --- | --- |
|
||||
| Local Storage | Must be uploaded to the S3 Pro bucket | `join(record.path, record.filename)` |
|
||||
| Amazon S3 | Usually no copy if reusing the same bucket; copy is needed when changing bucket, account, or region | `join(record.path, record.filename)` |
|
||||
| Aliyun OSS | Usually no copy if reusing the same bucket; copy is needed when changing bucket, account, or region | `join(record.path, record.filename)` |
|
||||
| Tencent COS | Usually no copy if reusing the same bucket; copy is needed when changing bucket, account, or region | `join(record.path, record.filename)` |
|
||||
|
||||
Files or objects still need to be copied in these cases:
|
||||
|
||||
- Migrating from Local Storage to S3 Pro
|
||||
- Changing bucket, account, region, or cloud provider
|
||||
- Moving historical objects from a public bucket to a new private bucket
|
||||
- Renaming object keys or reorganizing directories
|
||||
- The source bucket policy is not suitable for S3 Pro signed access or direct client upload
|
||||
- S3 Pro cannot directly access the source objects with the current endpoint and credentials
|
||||
|
||||
Before the real migration, run a dry-run. At least output:
|
||||
|
||||
- Number of records to migrate
|
||||
- Total size to migrate
|
||||
- Number of missing local files or missing source objects
|
||||
- Number of duplicate object keys
|
||||
- Number of records with unrecognized `storageId`
|
||||
- List requiring manual handling
|
||||
|
||||
If duplicate object keys appear, do not overwrite directly. Compare file size, ETag, or hash to confirm whether they point to the same file. If they are different files, generate a new key for one record and use that new key when updating the record.
|
||||
|
||||
### Step 6: Verify target object accessibility
|
||||
|
||||
After files are copied, or after confirming that the source cloud bucket can be reused, run S3 HEAD Object or an equivalent check for every migrated record. Confirm that S3 Pro can access the object through the target object key.
|
||||
|
||||
Recommended output:
|
||||
|
||||
- Success count
|
||||
- Missing source file count
|
||||
- Upload or copy failure count
|
||||
- Missing target object count
|
||||
- Duplicate key count
|
||||
- List requiring manual handling
|
||||
|
||||
Do not rely only on the script exit code. Object storage can have partial failures, successful retries, same-key overwrites, or write-only-but-not-readable permission problems. The migration manifest and HEAD Object checks are more reliable.
|
||||
|
||||
### Step 7: Update file records
|
||||
|
||||
After confirming that all target objects exist, update the file records. Only update records in file collections, keep the original `id`, and do not update many-to-many junction tables generated by attachment fields.
|
||||
|
||||
Core fields:
|
||||
|
||||
| Field | Value after migration |
|
||||
| --- | --- |
|
||||
| `storageId` | New S3 Pro storage ID |
|
||||
| `filename` | `join(oldPath, oldFilename)` |
|
||||
| `path` | Empty string |
|
||||
| `url` | Empty string for private storage; public URL or empty string for public storage |
|
||||
|
||||
If S3 Pro is private, `url` must be empty. S3 Pro also clears `url` before saving non-public storage records. File access will dynamically generate a temporary signed URL.
|
||||
|
||||
:::warning Note
|
||||
|
||||
Batch update records in a transaction if possible, and keep the migration manifest from Step 2 until after the migration is fully verified. Rollback needs that manifest to restore `storageId`, `path`, `filename`, and `url` to their old values.
|
||||
|
||||
:::
|
||||
|
||||
### Step 8: Switch future uploads to the new storage
|
||||
|
||||
After historical records are updated, switch the target storage for new uploads.
|
||||
|
||||
Check three types of configuration:
|
||||
|
||||
1. Default storage engine: if attachment fields or file collections do not specify storage, uploaded files use the default storage. Set the new S3 Pro storage as the default.
|
||||
2. Attachment fields: if an attachment field is configured with `options.storage = <old-storage-name>`, change it to the new S3 Pro storage `name`.
|
||||
3. File collections: if a file collection is configured with the old storage engine, change it to the new S3 Pro storage `name`.
|
||||
|
||||
Here, `name` is the storage engine name, not the storage ID. Historical file records use `storageId`.
|
||||
|
||||
### Step 9: Restart or refresh storage cache
|
||||
|
||||
The file manager caches `storages` configuration. If you update storage configuration through the NocoBase UI or resource API, `reloadStorages` is usually triggered. If you update the database directly, restart the application after migration or make sure the storage cache has been refreshed.
|
||||
|
||||
If you directly changed attachment field metadata or file collection metadata, it is also recommended to restart the application so collection configuration, field configuration, and frontend cache all use the new storage.
|
||||
|
||||
### Step 10: Sample-check and keep old files
|
||||
|
||||
After migration, verify at least these scenarios:
|
||||
|
||||
- View, preview, and download historical files in attachment fields
|
||||
- View, preview, and download historical files in file collections
|
||||
- Image thumbnails or preprocessing parameters
|
||||
- Office preview and other features that depend on external services reading file URLs
|
||||
- Save, preview, download, and delete newly uploaded files
|
||||
- Object deletion after deleting file records
|
||||
|
||||
Do not delete files from the old storage immediately. Keep them until the business side confirms the migration, backup cycles have covered the new state, and access logs show no abnormal behavior.
|
||||
|
||||
## Risks
|
||||
|
||||
### Data inconsistency during writes
|
||||
|
||||
If users can still upload or delete files during migration, objects may be copied after records are deleted, records may be updated while objects remain only in the old storage, or new uploads may not be included in the migration manifest. The default approach is to enter a maintenance window.
|
||||
|
||||
### Different object key rules in S3 Pro
|
||||
|
||||
After migration, `filename` should store the full object key and `path` should be empty. This differs from the historical record shape of Local Storage and built-in S3 / OSS / COS, and is the easiest place to make mistakes.
|
||||
|
||||
### URL accessibility changes
|
||||
|
||||
After migrating from public storage to private S3 Pro, historical public URLs may no longer remain valid. NocoBase dynamically generates temporary signed URLs for internal file access, but old URLs already saved in external systems are not automatically rewritten.
|
||||
|
||||
If third-party systems, email templates, exported files, or rich text content directly stores old URLs, evaluate a separate replacement strategy.
|
||||
|
||||
:::warning Note
|
||||
|
||||
If Markdown (Vditor) field content already stores file URLs, handle those contents separately. Private S3 Pro access generates temporary signed URLs that expire, and currently these private links are not supported for long-term storage and use inside Markdown (Vditor) fields.
|
||||
|
||||
If these fields must continue referencing files, keep publicly accessible file URLs for now.
|
||||
|
||||
:::
|
||||
|
||||
### Preview services and private files
|
||||
|
||||
Some preview features depend on external services accessing file URLs. Private S3 Pro generates temporary signed URLs, which can usually be accessed, but they are affected by signature expiration, network reachability, bucket permissions, and server-side caching.
|
||||
|
||||
If files are very sensitive, reassess whether external preview services should be allowed to read them.
|
||||
|
||||
### Direct database changes do not trigger all hooks
|
||||
|
||||
If you update `storages`, collection configuration, field configuration, and file records directly through SQL or a custom script, some NocoBase cache refreshes and save hooks are not triggered automatically. Restart the application after the update and verify storage configuration again.
|
||||
|
||||
### Delete behavior points to the new storage
|
||||
|
||||
After a file record's `storageId` is updated to S3 Pro, deleting that file record later will make NocoBase try to delete the object from the new bucket. Objects in the old storage will not be deleted automatically and need to be cleaned up separately.
|
||||
@@ -2,6 +2,15 @@
|
||||
|
||||
A storage engine based on Tencent Cloud COS. You need to prepare the relevant account and permissions before use.
|
||||
|
||||
|
||||
:::warning Note
|
||||
|
||||
This engine does not support private access. After a file is uploaded, NocoBase generates a directly accessible URL, and anyone who has that URL can access the file.
|
||||
|
||||
Even if the COS bucket itself is private, the built-in Tencent COS engine does not generate temporary signed URLs for file access. If you need private access, use [S3 Pro](./s3-pro). If historical files already exist, see [Migrate to S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
:::
|
||||
|
||||
## Configuration Parameters
|
||||
|
||||
|
||||
|
||||
@@ -44,6 +44,11 @@
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "custom-link",
|
||||
"label": "Migrar a S3 Pro",
|
||||
"link": "/file-manager/storage/migrate-to-s3-pro"
|
||||
},
|
||||
{
|
||||
"type": "custom-link",
|
||||
"label": "Tabla de archivos",
|
||||
|
||||
@@ -1,5 +1,11 @@
|
||||
# Campo de Adjunto
|
||||
|
||||
:::warning Nota
|
||||
|
||||
El campo de adjunto quedará obsoleto en una versión futura. Para nuevas aplicaciones o al ajustar procesos existentes, migre cuanto antes a una tabla de archivos personalizada y gestione los archivos relacionados mediante campos de relación.
|
||||
|
||||
:::
|
||||
|
||||
## Introducción
|
||||
|
||||
El sistema incluye un tipo de campo "Adjunto" que permite a los usuarios subir archivos en sus colecciones personalizadas.
|
||||
|
||||
@@ -2,6 +2,15 @@
|
||||
|
||||
Este motor de almacenamiento se basa en Aliyun OSS. Antes de usarlo, usted necesitará preparar la cuenta y los permisos correspondientes.
|
||||
|
||||
|
||||
:::warning Nota
|
||||
|
||||
Este motor no admite acceso privado. Después de subir un archivo, NocoBase genera una URL de acceso directo, y cualquier persona que tenga esa URL puede acceder al archivo.
|
||||
|
||||
Aunque el bucket OSS sea privado, el motor integrado Aliyun OSS no genera URL firmadas temporales para el acceso a archivos. Si necesita acceso privado, utilice [S3 Pro](./s3-pro). Si ya existen archivos históricos, consulte [Migrar a S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
:::
|
||||
|
||||
## Parámetros de Configuración
|
||||
|
||||
|
||||
|
||||
@@ -2,6 +2,15 @@
|
||||
|
||||
Este es un motor de almacenamiento basado en Amazon S3. Antes de usarlo, usted necesita preparar la cuenta y los permisos correspondientes.
|
||||
|
||||
|
||||
:::warning Nota
|
||||
|
||||
Este motor no admite acceso privado. Después de subir un archivo, NocoBase genera una URL de acceso directo, y cualquier persona que tenga esa URL puede acceder al archivo.
|
||||
|
||||
Aunque el bucket S3 sea privado, el motor integrado Amazon S3 no genera URL firmadas temporales para el acceso a archivos. Si necesita acceso privado, utilice [S3 Pro](./s3-pro). Si ya existen archivos históricos, consulte [Migrar a S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
:::
|
||||
|
||||
## Parámetros de Configuración
|
||||
|
||||
|
||||
|
||||
@@ -18,6 +18,29 @@ Actualmente, NocoBase ofrece soporte integrado para los siguientes tipos de moto
|
||||
|
||||
El sistema añade automáticamente un motor de almacenamiento local durante la instalación, el cual puede utilizar directamente. También tiene la opción de añadir nuevos motores o editar los parámetros de los ya existentes.
|
||||
|
||||
|
||||
Si ya utiliza un motor de almacenamiento que solo admite acceso público y quiere migrar archivos históricos a S3 Pro, consulte [Migrar a S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
## Accesibilidad de archivos
|
||||
|
||||
Cada motor de almacenamiento admite controles de acceso diferentes. Antes de configurarlo, confirme si los archivos necesitan acceso privado:
|
||||
|
||||
| Motor de almacenamiento | Accesibilidad de archivos |
|
||||
| --- | --- |
|
||||
| [Local Storage](./local) | Solo se admite acceso público; no se admite acceso privado |
|
||||
| [Amazon S3](./amazon-s3) | Solo se admite acceso público; no se admite acceso privado |
|
||||
| [Aliyun OSS](./aliyun-oss) | Solo se admite acceso público; no se admite acceso privado |
|
||||
| [Tencent COS](./tencent-cos) | Solo se admite acceso público; no se admite acceso privado |
|
||||
| [S3 Pro](./s3-pro) | Se admite acceso privado mediante URL firmadas temporales |
|
||||
|
||||
:::warning Nota
|
||||
|
||||
Almacenamiento local, Amazon S3, Aliyun OSS y Tencent COS no realizan autenticación de inicio de sesión para acceder a los archivos y no generan URL firmadas temporales. Después de subir un archivo, cualquier persona que tenga la URL puede acceder directamente a él.
|
||||
|
||||
Si necesita guardar contratos, documentos de identidad, materiales internos u otros archivos que no deben ser públicos, utilice [S3 Pro](./s3-pro) y active el acceso privado.
|
||||
|
||||
:::
|
||||
|
||||
## Parámetros comunes
|
||||
|
||||
Además de los parámetros específicos de cada tipo de motor, las siguientes secciones describen los parámetros comunes (tomando como ejemplo el almacenamiento local):
|
||||
|
||||
@@ -2,6 +2,15 @@
|
||||
|
||||
Los archivos que suba se guardarán en un directorio local del disco duro del servidor. Esta opción es ideal para escenarios donde el volumen total de archivos subidos gestionados por el sistema es pequeño o para fines experimentales.
|
||||
|
||||
|
||||
:::warning Nota
|
||||
|
||||
El almacenamiento local no admite acceso privado. Después de subir un archivo, NocoBase genera una URL de acceso directo, y cualquier persona que tenga esa URL puede acceder al archivo.
|
||||
|
||||
Si necesita guardar contratos, documentos de identidad, materiales internos u otros archivos que no deben ser públicos, utilice [S3 Pro](./s3-pro). Si ya existen archivos históricos, consulte [Migrar a S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
:::
|
||||
|
||||
## Parámetros de Configuración
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,270 @@
|
||||
---
|
||||
pkg: '@nocobase/plugin-file-storage-s3-pro'
|
||||
title: "Migrar a S3 Pro"
|
||||
description: "Migrate files from public-only storage engines to S3 Pro, including maintenance windows, backups, object key mapping, record updates, and risk checks."
|
||||
keywords: "S3 Pro,storage migration,private access,file migration,Local Storage,S3,OSS,COS,NocoBase"
|
||||
---
|
||||
|
||||
# Migrar a S3 Pro
|
||||
|
||||
If you already use [Local Storage](./local.md), [Amazon S3](./amazon-s3.md), [Aliyun OSS](./aliyun-oss.md), or [Tencent COS](./tencent-cos.md), and later need private file access, you can migrate historical files to [S3 Pro](./s3-pro.md).
|
||||
|
||||
This is not only a storage configuration change. A migration must handle three things: make the physical files or objects available from the new storage, update file records, and switch future uploads to the new storage.
|
||||
|
||||
:::warning Nota
|
||||
|
||||
Run the migration once in a test environment first, and prepare restorable database and file backups. If users continue to upload or delete attachments during the migration, records can be missed, overwritten, or become inconsistent.
|
||||
|
||||
:::
|
||||
|
||||
## Scope
|
||||
|
||||
This guide applies when migrating from these storage engines to S3 Pro:
|
||||
|
||||
| Source storage engine | Historical file location |
|
||||
| --- | --- |
|
||||
| Local Storage | `<documentRoot>/<record.path>/<record.filename>` |
|
||||
| Amazon S3 | `join(record.path, record.filename)` in the source bucket |
|
||||
| Aliyun OSS | `join(record.path, record.filename)` in the source bucket |
|
||||
| Tencent COS | `join(record.path, record.filename)` in the source bucket |
|
||||
|
||||
Here, `record.path` and `record.filename` come from the file record itself, including the built-in `attachments` table and other file collections.
|
||||
|
||||
:::tip Nota
|
||||
|
||||
For Local Storage, `documentRoot` must come from `options.documentRoot` in the corresponding `storages` record. The default value is `storage/uploads`, but the final absolute path depends on the runtime storage directory of NocoBase.
|
||||
|
||||
:::
|
||||
|
||||
## Operation Steps
|
||||
|
||||
### Step 1: Stop writes or enter a maintenance window
|
||||
|
||||
Pause user uploads, updates, and deletions of attachments during the migration. You can use a maintenance window, temporarily disable entry points, or freeze related business flows.
|
||||
|
||||
The goal is to keep file records and physical files stable. If users upload or delete files during migration, the migration script may no longer be working against the latest records.
|
||||
|
||||
### Step 2: Back up the database and files
|
||||
|
||||
Prepare at least two backups:
|
||||
|
||||
1. Database backup
|
||||
2. File backup or object snapshot from the source storage
|
||||
|
||||
For Local Storage, back up the directory pointed to by `documentRoot`. Historical files are usually stored at:
|
||||
|
||||
```text
|
||||
<documentRoot>/<record.path>/<record.filename>
|
||||
```
|
||||
|
||||
For Amazon S3, Aliyun OSS, and Tencent COS, confirm that objects in the source bucket are still readable, and record the source storage engine's `id`, `name`, `type`, `path`, `baseUrl`, and `options`.
|
||||
|
||||
It is also recommended to export a migration manifest for rollback and manual review:
|
||||
|
||||
```text
|
||||
collection
|
||||
id
|
||||
oldStorageId
|
||||
oldPath
|
||||
oldFilename
|
||||
oldUrl
|
||||
newKey
|
||||
size
|
||||
mimetype
|
||||
```
|
||||
|
||||
### Step 3: Create and verify the S3 Pro storage
|
||||
|
||||
Create a new `s3-compatible` storage according to [S3 Pro](./s3-pro.md). At least verify these settings:
|
||||
|
||||
- bucket
|
||||
- endpoint
|
||||
- region
|
||||
- accessKey
|
||||
- secret
|
||||
- public / private
|
||||
- access endpoint
|
||||
- forcePathStyle
|
||||
- file size and MIME type rules
|
||||
|
||||
Upload a test file with the new storage and confirm upload, preview, download, and deletion all work. If the target is private access, also confirm that the access URL is a temporary signed URL and that its expiration is expected.
|
||||
|
||||
:::warning Nota
|
||||
|
||||
S3 Pro uses direct client upload. The target bucket must have CORS rules that allow uploads from the NocoBase site, otherwise new uploads will fail.
|
||||
|
||||
:::
|
||||
|
||||
### Step 4: Decide the object key mapping
|
||||
|
||||
For historical files migrated to S3 Pro, it is recommended to continue using the old relative path as the S3 object key:
|
||||
|
||||
```text
|
||||
oldKey = join(record.path, record.filename)
|
||||
```
|
||||
|
||||
For example:
|
||||
|
||||
```text
|
||||
record.path = "avatars"
|
||||
record.filename = "a-123.png"
|
||||
oldKey = "avatars/a-123.png"
|
||||
```
|
||||
|
||||
For persisted file records, S3 Pro uses `file.filename` directly as the full object key when accessing the file. It does not append `file.path`. Therefore, migrated records should be updated to:
|
||||
|
||||
```text
|
||||
storageId = <s3-pro-storage-id>
|
||||
filename = <oldKey>
|
||||
path = ""
|
||||
url = ""
|
||||
```
|
||||
|
||||
Do not migrate to this shape:
|
||||
|
||||
```text
|
||||
filename = "a-123.png"
|
||||
path = "avatars"
|
||||
```
|
||||
|
||||
Otherwise S3 Pro may treat only `a-123.png` as the object key, causing historical files to fail to load.
|
||||
|
||||
Use `/` as the separator when generating object keys. Do not use operating-system-specific path separators. If an old path starts with `/`, remove the leading slash.
|
||||
|
||||
### Step 5: Move physical files or confirm object location
|
||||
|
||||
Traverse all file records, including the built-in `attachments` table and other file collections. Only process records where `storageId = <old-storage-id>`.
|
||||
|
||||
If the source is Local Storage, upload the local files to the bucket used by S3 Pro. If the source is already Amazon S3, Aliyun OSS, or Tencent COS, and the new S3 Pro storage points to the same bucket and endpoint, with credentials that can read the same objects, you usually do not need to copy objects. In that case, confirm that the `oldKey` generated in Step 4 is accessible through S3 Pro.
|
||||
|
||||
| Source storage engine | Whether files need to be copied | Target object key |
|
||||
| --- | --- | --- |
|
||||
| Local Storage | Must be uploaded to the S3 Pro bucket | `join(record.path, record.filename)` |
|
||||
| Amazon S3 | Usually no copy if reusing the same bucket; copy is needed when changing bucket, account, or region | `join(record.path, record.filename)` |
|
||||
| Aliyun OSS | Usually no copy if reusing the same bucket; copy is needed when changing bucket, account, or region | `join(record.path, record.filename)` |
|
||||
| Tencent COS | Usually no copy if reusing the same bucket; copy is needed when changing bucket, account, or region | `join(record.path, record.filename)` |
|
||||
|
||||
Files or objects still need to be copied in these cases:
|
||||
|
||||
- Migrating from Local Storage to S3 Pro
|
||||
- Changing bucket, account, region, or cloud provider
|
||||
- Moving historical objects from a public bucket to a new private bucket
|
||||
- Renaming object keys or reorganizing directories
|
||||
- The source bucket policy is not suitable for S3 Pro signed access or direct client upload
|
||||
- S3 Pro cannot directly access the source objects with the current endpoint and credentials
|
||||
|
||||
Before the real migration, run a dry-run. At least output:
|
||||
|
||||
- Number of records to migrate
|
||||
- Total size to migrate
|
||||
- Number of missing local files or missing source objects
|
||||
- Number of duplicate object keys
|
||||
- Number of records with unrecognized `storageId`
|
||||
- List requiring manual handling
|
||||
|
||||
If duplicate object keys appear, do not overwrite directly. Compare file size, ETag, or hash to confirm whether they point to the same file. If they are different files, generate a new key for one record and use that new key when updating the record.
|
||||
|
||||
### Step 6: Verify target object accessibility
|
||||
|
||||
After files are copied, or after confirming that the source cloud bucket can be reused, run S3 HEAD Object or an equivalent check for every migrated record. Confirm that S3 Pro can access the object through the target object key.
|
||||
|
||||
Recommended output:
|
||||
|
||||
- Success count
|
||||
- Missing source file count
|
||||
- Upload or copy failure count
|
||||
- Missing target object count
|
||||
- Duplicate key count
|
||||
- List requiring manual handling
|
||||
|
||||
Do not rely only on the script exit code. Object storage can have partial failures, successful retries, same-key overwrites, or write-only-but-not-readable permission problems. The migration manifest and HEAD Object checks are more reliable.
|
||||
|
||||
### Step 7: Update file records
|
||||
|
||||
After confirming that all target objects exist, update the file records. Only update records in file collections, keep the original `id`, and do not update many-to-many junction tables generated by attachment fields.
|
||||
|
||||
Core fields:
|
||||
|
||||
| Field | Value after migration |
|
||||
| --- | --- |
|
||||
| `storageId` | New S3 Pro storage ID |
|
||||
| `filename` | `join(oldPath, oldFilename)` |
|
||||
| `path` | Empty string |
|
||||
| `url` | Empty string for private storage; public URL or empty string for public storage |
|
||||
|
||||
If S3 Pro is private, `url` must be empty. S3 Pro also clears `url` before saving non-public storage records. File access will dynamically generate a temporary signed URL.
|
||||
|
||||
:::warning Nota
|
||||
|
||||
Batch update records in a transaction if possible, and keep the migration manifest from Step 2 until after the migration is fully verified. Rollback needs that manifest to restore `storageId`, `path`, `filename`, and `url` to their old values.
|
||||
|
||||
:::
|
||||
|
||||
### Step 8: Switch future uploads to the new storage
|
||||
|
||||
After historical records are updated, switch the target storage for new uploads.
|
||||
|
||||
Check three types of configuration:
|
||||
|
||||
1. Default storage engine: if attachment fields or file collections do not specify storage, uploaded files use the default storage. Set the new S3 Pro storage as the default.
|
||||
2. Attachment fields: if an attachment field is configured with `options.storage = <old-storage-name>`, change it to the new S3 Pro storage `name`.
|
||||
3. File collections: if a file collection is configured with the old storage engine, change it to the new S3 Pro storage `name`.
|
||||
|
||||
Here, `name` is the storage engine name, not the storage ID. Historical file records use `storageId`.
|
||||
|
||||
### Step 9: Restart or refresh storage cache
|
||||
|
||||
The file manager caches `storages` configuration. If you update storage configuration through the NocoBase UI or resource API, `reloadStorages` is usually triggered. If you update the database directly, restart the application after migration or make sure the storage cache has been refreshed.
|
||||
|
||||
If you directly changed attachment field metadata or file collection metadata, it is also recommended to restart the application so collection configuration, field configuration, and frontend cache all use the new storage.
|
||||
|
||||
### Step 10: Sample-check and keep old files
|
||||
|
||||
After migration, verify at least these scenarios:
|
||||
|
||||
- View, preview, and download historical files in attachment fields
|
||||
- View, preview, and download historical files in file collections
|
||||
- Image thumbnails or preprocessing parameters
|
||||
- Office preview and other features that depend on external services reading file URLs
|
||||
- Save, preview, download, and delete newly uploaded files
|
||||
- Object deletion after deleting file records
|
||||
|
||||
Do not delete files from the old storage immediately. Keep them until the business side confirms the migration, backup cycles have covered the new state, and access logs show no abnormal behavior.
|
||||
|
||||
## Risks
|
||||
|
||||
### Data inconsistency during writes
|
||||
|
||||
If users can still upload or delete files during migration, objects may be copied after records are deleted, records may be updated while objects remain only in the old storage, or new uploads may not be included in the migration manifest. The default approach is to enter a maintenance window.
|
||||
|
||||
### Different object key rules in S3 Pro
|
||||
|
||||
After migration, `filename` should store the full object key and `path` should be empty. This differs from the historical record shape of Local Storage and built-in S3 / OSS / COS, and is the easiest place to make mistakes.
|
||||
|
||||
### URL accessibility changes
|
||||
|
||||
After migrating from public storage to private S3 Pro, historical public URLs may no longer remain valid. NocoBase dynamically generates temporary signed URLs for internal file access, but old URLs already saved in external systems are not automatically rewritten.
|
||||
|
||||
If third-party systems, email templates, exported files, or rich text content directly stores old URLs, evaluate a separate replacement strategy.
|
||||
|
||||
:::warning Nota
|
||||
|
||||
If Markdown (Vditor) field content already stores file URLs, handle those contents separately. Private S3 Pro access generates temporary signed URLs that expire, and currently these private links are not supported for long-term storage and use inside Markdown (Vditor) fields.
|
||||
|
||||
If these fields must continue referencing files, keep publicly accessible file URLs for now.
|
||||
|
||||
:::
|
||||
|
||||
### Preview services and private files
|
||||
|
||||
Some preview features depend on external services accessing file URLs. Private S3 Pro generates temporary signed URLs, which can usually be accessed, but they are affected by signature expiration, network reachability, bucket permissions, and server-side caching.
|
||||
|
||||
If files are very sensitive, reassess whether external preview services should be allowed to read them.
|
||||
|
||||
### Direct database changes do not trigger all hooks
|
||||
|
||||
If you update `storages`, collection configuration, field configuration, and file records directly through SQL or a custom script, some NocoBase cache refreshes and save hooks are not triggered automatically. Restart the application after the update and verify storage configuration again.
|
||||
|
||||
### Delete behavior points to the new storage
|
||||
|
||||
After a file record's `storageId` is updated to S3 Pro, deleting that file record later will make NocoBase try to delete the object from the new bucket. Objects in the old storage will not be deleted automatically and need to be cleaned up separately.
|
||||
@@ -2,6 +2,15 @@
|
||||
|
||||
Es un motor de almacenamiento basado en Tencent Cloud COS. Antes de usarlo, necesitará preparar la cuenta y los permisos correspondientes.
|
||||
|
||||
|
||||
:::warning Nota
|
||||
|
||||
Este motor no admite acceso privado. Después de subir un archivo, NocoBase genera una URL de acceso directo, y cualquier persona que tenga esa URL puede acceder al archivo.
|
||||
|
||||
Aunque el bucket COS sea privado, el motor integrado Tencent COS no genera URL firmadas temporales para el acceso a archivos. Si necesita acceso privado, utilice [S3 Pro](./s3-pro). Si ya existen archivos históricos, consulte [Migrar a S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
:::
|
||||
|
||||
## Parámetros de configuración
|
||||
|
||||
|
||||
|
||||
@@ -44,6 +44,11 @@
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "custom-link",
|
||||
"label": "Migrer vers S3 Pro",
|
||||
"link": "/file-manager/storage/migrate-to-s3-pro"
|
||||
},
|
||||
{
|
||||
"type": "custom-link",
|
||||
"label": "Table de fichiers",
|
||||
|
||||
@@ -1,5 +1,11 @@
|
||||
# Champ Pièce jointe
|
||||
|
||||
:::warning Remarque
|
||||
|
||||
Le champ pièce jointe sera déprécié dans une future version. Pour les nouvelles applications ou lors de l’ajustement de processus existants, migrez dès que possible vers une table de fichiers personnalisée et gérez les fichiers associés avec des champs de relation.
|
||||
|
||||
:::
|
||||
|
||||
## Introduction
|
||||
|
||||
Le système intègre un type de champ "Pièce jointe" pour permettre aux utilisateurs de téléverser des fichiers dans les collections personnalisées.
|
||||
|
||||
@@ -2,6 +2,15 @@
|
||||
|
||||
Un moteur de stockage basé sur Aliyun OSS. Avant de l'utiliser, vous devez préparer les comptes et les autorisations nécessaires.
|
||||
|
||||
|
||||
:::warning Remarque
|
||||
|
||||
Ce moteur ne prend pas en charge l’accès privé. Après l’envoi d’un fichier, NocoBase génère une URL directement accessible, et toute personne disposant de cette URL peut accéder au fichier.
|
||||
|
||||
Même si le bucket OSS est privé, le moteur intégré Aliyun OSS ne génère pas d’URL signées temporaires pour l’accès aux fichiers. Si vous avez besoin d’un accès privé, utilisez [S3 Pro](./s3-pro). Si des fichiers historiques existent déjà, consultez [Migrer vers S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
:::
|
||||
|
||||
## Paramètres de configuration
|
||||
|
||||
|
||||
|
||||
@@ -2,6 +2,15 @@
|
||||
|
||||
Le moteur de stockage basé sur Amazon S3 nécessite la préparation des comptes et des autorisations nécessaires avant utilisation.
|
||||
|
||||
|
||||
:::warning Remarque
|
||||
|
||||
Ce moteur ne prend pas en charge l’accès privé. Après l’envoi d’un fichier, NocoBase génère une URL directement accessible, et toute personne disposant de cette URL peut accéder au fichier.
|
||||
|
||||
Même si le bucket S3 est privé, le moteur intégré Amazon S3 ne génère pas d’URL signées temporaires pour l’accès aux fichiers. Si vous avez besoin d’un accès privé, utilisez [S3 Pro](./s3-pro). Si des fichiers historiques existent déjà, consultez [Migrer vers S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
:::
|
||||
|
||||
## Paramètres de configuration
|
||||
|
||||
|
||||
|
||||
@@ -18,6 +18,29 @@ Actuellement, NocoBase prend en charge les types de moteurs de stockage intégr
|
||||
|
||||
Le système ajoute automatiquement un moteur de stockage local lors de l'installation, que vous pouvez utiliser directement. Vous pouvez également ajouter de nouveaux moteurs ou modifier les paramètres de ceux qui existent déjà.
|
||||
|
||||
|
||||
Si vous utilisez déjà un moteur de stockage qui prend uniquement en charge l’accès public et souhaitez migrer les fichiers historiques vers S3 Pro, consultez [Migrer vers S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
## Accessibilité des fichiers
|
||||
|
||||
Les moteurs de stockage ne prennent pas tous en charge les mêmes contrôles d’accès. Avant la configuration, vérifiez si vos fichiers nécessitent un accès privé :
|
||||
|
||||
| Moteur de stockage | Accessibilité des fichiers |
|
||||
| --- | --- |
|
||||
| [Local Storage](./local) | Seul l’accès public est pris en charge ; l’accès privé ne l’est pas |
|
||||
| [Amazon S3](./amazon-s3) | Seul l’accès public est pris en charge ; l’accès privé ne l’est pas |
|
||||
| [Aliyun OSS](./aliyun-oss) | Seul l’accès public est pris en charge ; l’accès privé ne l’est pas |
|
||||
| [Tencent COS](./tencent-cos) | Seul l’accès public est pris en charge ; l’accès privé ne l’est pas |
|
||||
| [S3 Pro](./s3-pro) | L’accès privé est pris en charge via des URL signées temporaires |
|
||||
|
||||
:::warning Remarque
|
||||
|
||||
Le stockage local, Amazon S3, Aliyun OSS et Tencent COS ne vérifient pas la connexion pour l’accès aux fichiers et ne génèrent pas d’URL signées temporaires. Après l’envoi d’un fichier, toute personne disposant de son URL peut y accéder directement.
|
||||
|
||||
Si vous devez stocker des contrats, pièces d’identité, documents internes ou autres fichiers non publics, utilisez [S3 Pro](./s3-pro) et activez l’accès privé.
|
||||
|
||||
:::
|
||||
|
||||
## Paramètres communs
|
||||
|
||||
En plus des paramètres spécifiques à chaque type de moteur, les sections suivantes décrivent les paramètres communs (en prenant le stockage local comme exemple) :
|
||||
|
||||
@@ -2,6 +2,15 @@
|
||||
|
||||
Les fichiers téléchargés seront enregistrés dans un répertoire local sur le disque dur du serveur. Cette option convient aux scénarios où le volume total de fichiers gérés par le système est faible, ou à des fins expérimentales.
|
||||
|
||||
|
||||
:::warning Remarque
|
||||
|
||||
Le stockage local ne prend pas en charge l’accès privé. Après l’envoi d’un fichier, NocoBase génère une URL directement accessible, et toute personne disposant de cette URL peut accéder au fichier.
|
||||
|
||||
Si vous devez stocker des contrats, pièces d’identité, documents internes ou autres fichiers non publics, utilisez [S3 Pro](./s3-pro). Si des fichiers historiques existent déjà, consultez [Migrer vers S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
:::
|
||||
|
||||
## Paramètres de configuration
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,270 @@
|
||||
---
|
||||
pkg: '@nocobase/plugin-file-storage-s3-pro'
|
||||
title: "Migrer vers S3 Pro"
|
||||
description: "Migrate files from public-only storage engines to S3 Pro, including maintenance windows, backups, object key mapping, record updates, and risk checks."
|
||||
keywords: "S3 Pro,storage migration,private access,file migration,Local Storage,S3,OSS,COS,NocoBase"
|
||||
---
|
||||
|
||||
# Migrer vers S3 Pro
|
||||
|
||||
If you already use [Local Storage](./local.md), [Amazon S3](./amazon-s3.md), [Aliyun OSS](./aliyun-oss.md), or [Tencent COS](./tencent-cos.md), and later need private file access, you can migrate historical files to [S3 Pro](./s3-pro.md).
|
||||
|
||||
This is not only a storage configuration change. A migration must handle three things: make the physical files or objects available from the new storage, update file records, and switch future uploads to the new storage.
|
||||
|
||||
:::warning Remarque
|
||||
|
||||
Run the migration once in a test environment first, and prepare restorable database and file backups. If users continue to upload or delete attachments during the migration, records can be missed, overwritten, or become inconsistent.
|
||||
|
||||
:::
|
||||
|
||||
## Scope
|
||||
|
||||
This guide applies when migrating from these storage engines to S3 Pro:
|
||||
|
||||
| Source storage engine | Historical file location |
|
||||
| --- | --- |
|
||||
| Local Storage | `<documentRoot>/<record.path>/<record.filename>` |
|
||||
| Amazon S3 | `join(record.path, record.filename)` in the source bucket |
|
||||
| Aliyun OSS | `join(record.path, record.filename)` in the source bucket |
|
||||
| Tencent COS | `join(record.path, record.filename)` in the source bucket |
|
||||
|
||||
Here, `record.path` and `record.filename` come from the file record itself, including the built-in `attachments` table and other file collections.
|
||||
|
||||
:::tip Remarque
|
||||
|
||||
For Local Storage, `documentRoot` must come from `options.documentRoot` in the corresponding `storages` record. The default value is `storage/uploads`, but the final absolute path depends on the runtime storage directory of NocoBase.
|
||||
|
||||
:::
|
||||
|
||||
## Operation Steps
|
||||
|
||||
### Step 1: Stop writes or enter a maintenance window
|
||||
|
||||
Pause user uploads, updates, and deletions of attachments during the migration. You can use a maintenance window, temporarily disable entry points, or freeze related business flows.
|
||||
|
||||
The goal is to keep file records and physical files stable. If users upload or delete files during migration, the migration script may no longer be working against the latest records.
|
||||
|
||||
### Step 2: Back up the database and files
|
||||
|
||||
Prepare at least two backups:
|
||||
|
||||
1. Database backup
|
||||
2. File backup or object snapshot from the source storage
|
||||
|
||||
For Local Storage, back up the directory pointed to by `documentRoot`. Historical files are usually stored at:
|
||||
|
||||
```text
|
||||
<documentRoot>/<record.path>/<record.filename>
|
||||
```
|
||||
|
||||
For Amazon S3, Aliyun OSS, and Tencent COS, confirm that objects in the source bucket are still readable, and record the source storage engine's `id`, `name`, `type`, `path`, `baseUrl`, and `options`.
|
||||
|
||||
It is also recommended to export a migration manifest for rollback and manual review:
|
||||
|
||||
```text
|
||||
collection
|
||||
id
|
||||
oldStorageId
|
||||
oldPath
|
||||
oldFilename
|
||||
oldUrl
|
||||
newKey
|
||||
size
|
||||
mimetype
|
||||
```
|
||||
|
||||
### Step 3: Create and verify the S3 Pro storage
|
||||
|
||||
Create a new `s3-compatible` storage according to [S3 Pro](./s3-pro.md). At least verify these settings:
|
||||
|
||||
- bucket
|
||||
- endpoint
|
||||
- region
|
||||
- accessKey
|
||||
- secret
|
||||
- public / private
|
||||
- access endpoint
|
||||
- forcePathStyle
|
||||
- file size and MIME type rules
|
||||
|
||||
Upload a test file with the new storage and confirm upload, preview, download, and deletion all work. If the target is private access, also confirm that the access URL is a temporary signed URL and that its expiration is expected.
|
||||
|
||||
:::warning Remarque
|
||||
|
||||
S3 Pro uses direct client upload. The target bucket must have CORS rules that allow uploads from the NocoBase site, otherwise new uploads will fail.
|
||||
|
||||
:::
|
||||
|
||||
### Step 4: Decide the object key mapping
|
||||
|
||||
For historical files migrated to S3 Pro, it is recommended to continue using the old relative path as the S3 object key:
|
||||
|
||||
```text
|
||||
oldKey = join(record.path, record.filename)
|
||||
```
|
||||
|
||||
For example:
|
||||
|
||||
```text
|
||||
record.path = "avatars"
|
||||
record.filename = "a-123.png"
|
||||
oldKey = "avatars/a-123.png"
|
||||
```
|
||||
|
||||
For persisted file records, S3 Pro uses `file.filename` directly as the full object key when accessing the file. It does not append `file.path`. Therefore, migrated records should be updated to:
|
||||
|
||||
```text
|
||||
storageId = <s3-pro-storage-id>
|
||||
filename = <oldKey>
|
||||
path = ""
|
||||
url = ""
|
||||
```
|
||||
|
||||
Do not migrate to this shape:
|
||||
|
||||
```text
|
||||
filename = "a-123.png"
|
||||
path = "avatars"
|
||||
```
|
||||
|
||||
Otherwise S3 Pro may treat only `a-123.png` as the object key, causing historical files to fail to load.
|
||||
|
||||
Use `/` as the separator when generating object keys. Do not use operating-system-specific path separators. If an old path starts with `/`, remove the leading slash.
|
||||
|
||||
### Step 5: Move physical files or confirm object location
|
||||
|
||||
Traverse all file records, including the built-in `attachments` table and other file collections. Only process records where `storageId = <old-storage-id>`.
|
||||
|
||||
If the source is Local Storage, upload the local files to the bucket used by S3 Pro. If the source is already Amazon S3, Aliyun OSS, or Tencent COS, and the new S3 Pro storage points to the same bucket and endpoint, with credentials that can read the same objects, you usually do not need to copy objects. In that case, confirm that the `oldKey` generated in Step 4 is accessible through S3 Pro.
|
||||
|
||||
| Source storage engine | Whether files need to be copied | Target object key |
|
||||
| --- | --- | --- |
|
||||
| Local Storage | Must be uploaded to the S3 Pro bucket | `join(record.path, record.filename)` |
|
||||
| Amazon S3 | Usually no copy if reusing the same bucket; copy is needed when changing bucket, account, or region | `join(record.path, record.filename)` |
|
||||
| Aliyun OSS | Usually no copy if reusing the same bucket; copy is needed when changing bucket, account, or region | `join(record.path, record.filename)` |
|
||||
| Tencent COS | Usually no copy if reusing the same bucket; copy is needed when changing bucket, account, or region | `join(record.path, record.filename)` |
|
||||
|
||||
Files or objects still need to be copied in these cases:
|
||||
|
||||
- Migrating from Local Storage to S3 Pro
|
||||
- Changing bucket, account, region, or cloud provider
|
||||
- Moving historical objects from a public bucket to a new private bucket
|
||||
- Renaming object keys or reorganizing directories
|
||||
- The source bucket policy is not suitable for S3 Pro signed access or direct client upload
|
||||
- S3 Pro cannot directly access the source objects with the current endpoint and credentials
|
||||
|
||||
Before the real migration, run a dry-run. At least output:
|
||||
|
||||
- Number of records to migrate
|
||||
- Total size to migrate
|
||||
- Number of missing local files or missing source objects
|
||||
- Number of duplicate object keys
|
||||
- Number of records with unrecognized `storageId`
|
||||
- List requiring manual handling
|
||||
|
||||
If duplicate object keys appear, do not overwrite directly. Compare file size, ETag, or hash to confirm whether they point to the same file. If they are different files, generate a new key for one record and use that new key when updating the record.
|
||||
|
||||
### Step 6: Verify target object accessibility
|
||||
|
||||
After files are copied, or after confirming that the source cloud bucket can be reused, run S3 HEAD Object or an equivalent check for every migrated record. Confirm that S3 Pro can access the object through the target object key.
|
||||
|
||||
Recommended output:
|
||||
|
||||
- Success count
|
||||
- Missing source file count
|
||||
- Upload or copy failure count
|
||||
- Missing target object count
|
||||
- Duplicate key count
|
||||
- List requiring manual handling
|
||||
|
||||
Do not rely only on the script exit code. Object storage can have partial failures, successful retries, same-key overwrites, or write-only-but-not-readable permission problems. The migration manifest and HEAD Object checks are more reliable.
|
||||
|
||||
### Step 7: Update file records
|
||||
|
||||
After confirming that all target objects exist, update the file records. Only update records in file collections, keep the original `id`, and do not update many-to-many junction tables generated by attachment fields.
|
||||
|
||||
Core fields:
|
||||
|
||||
| Field | Value after migration |
|
||||
| --- | --- |
|
||||
| `storageId` | New S3 Pro storage ID |
|
||||
| `filename` | `join(oldPath, oldFilename)` |
|
||||
| `path` | Empty string |
|
||||
| `url` | Empty string for private storage; public URL or empty string for public storage |
|
||||
|
||||
If S3 Pro is private, `url` must be empty. S3 Pro also clears `url` before saving non-public storage records. File access will dynamically generate a temporary signed URL.
|
||||
|
||||
:::warning Remarque
|
||||
|
||||
Batch update records in a transaction if possible, and keep the migration manifest from Step 2 until after the migration is fully verified. Rollback needs that manifest to restore `storageId`, `path`, `filename`, and `url` to their old values.
|
||||
|
||||
:::
|
||||
|
||||
### Step 8: Switch future uploads to the new storage
|
||||
|
||||
After historical records are updated, switch the target storage for new uploads.
|
||||
|
||||
Check three types of configuration:
|
||||
|
||||
1. Default storage engine: if attachment fields or file collections do not specify storage, uploaded files use the default storage. Set the new S3 Pro storage as the default.
|
||||
2. Attachment fields: if an attachment field is configured with `options.storage = <old-storage-name>`, change it to the new S3 Pro storage `name`.
|
||||
3. File collections: if a file collection is configured with the old storage engine, change it to the new S3 Pro storage `name`.
|
||||
|
||||
Here, `name` is the storage engine name, not the storage ID. Historical file records use `storageId`.
|
||||
|
||||
### Step 9: Restart or refresh storage cache
|
||||
|
||||
The file manager caches `storages` configuration. If you update storage configuration through the NocoBase UI or resource API, `reloadStorages` is usually triggered. If you update the database directly, restart the application after migration or make sure the storage cache has been refreshed.
|
||||
|
||||
If you directly changed attachment field metadata or file collection metadata, it is also recommended to restart the application so collection configuration, field configuration, and frontend cache all use the new storage.
|
||||
|
||||
### Step 10: Sample-check and keep old files
|
||||
|
||||
After migration, verify at least these scenarios:
|
||||
|
||||
- View, preview, and download historical files in attachment fields
|
||||
- View, preview, and download historical files in file collections
|
||||
- Image thumbnails or preprocessing parameters
|
||||
- Office preview and other features that depend on external services reading file URLs
|
||||
- Save, preview, download, and delete newly uploaded files
|
||||
- Object deletion after deleting file records
|
||||
|
||||
Do not delete files from the old storage immediately. Keep them until the business side confirms the migration, backup cycles have covered the new state, and access logs show no abnormal behavior.
|
||||
|
||||
## Risks
|
||||
|
||||
### Data inconsistency during writes
|
||||
|
||||
If users can still upload or delete files during migration, objects may be copied after records are deleted, records may be updated while objects remain only in the old storage, or new uploads may not be included in the migration manifest. The default approach is to enter a maintenance window.
|
||||
|
||||
### Different object key rules in S3 Pro
|
||||
|
||||
After migration, `filename` should store the full object key and `path` should be empty. This differs from the historical record shape of Local Storage and built-in S3 / OSS / COS, and is the easiest place to make mistakes.
|
||||
|
||||
### URL accessibility changes
|
||||
|
||||
After migrating from public storage to private S3 Pro, historical public URLs may no longer remain valid. NocoBase dynamically generates temporary signed URLs for internal file access, but old URLs already saved in external systems are not automatically rewritten.
|
||||
|
||||
If third-party systems, email templates, exported files, or rich text content directly stores old URLs, evaluate a separate replacement strategy.
|
||||
|
||||
:::warning Remarque
|
||||
|
||||
If Markdown (Vditor) field content already stores file URLs, handle those contents separately. Private S3 Pro access generates temporary signed URLs that expire, and currently these private links are not supported for long-term storage and use inside Markdown (Vditor) fields.
|
||||
|
||||
If these fields must continue referencing files, keep publicly accessible file URLs for now.
|
||||
|
||||
:::
|
||||
|
||||
### Preview services and private files
|
||||
|
||||
Some preview features depend on external services accessing file URLs. Private S3 Pro generates temporary signed URLs, which can usually be accessed, but they are affected by signature expiration, network reachability, bucket permissions, and server-side caching.
|
||||
|
||||
If files are very sensitive, reassess whether external preview services should be allowed to read them.
|
||||
|
||||
### Direct database changes do not trigger all hooks
|
||||
|
||||
If you update `storages`, collection configuration, field configuration, and file records directly through SQL or a custom script, some NocoBase cache refreshes and save hooks are not triggered automatically. Restart the application after the update and verify storage configuration again.
|
||||
|
||||
### Delete behavior points to the new storage
|
||||
|
||||
After a file record's `storageId` is updated to S3 Pro, deleting that file record later will make NocoBase try to delete the object from the new bucket. Objects in the old storage will not be deleted automatically and need to be cleaned up separately.
|
||||
@@ -2,6 +2,15 @@
|
||||
|
||||
Un moteur de stockage basé sur Tencent Cloud COS. Avant de l'utiliser, vous devrez préparer les comptes et les autorisations nécessaires.
|
||||
|
||||
|
||||
:::warning Remarque
|
||||
|
||||
Ce moteur ne prend pas en charge l’accès privé. Après l’envoi d’un fichier, NocoBase génère une URL directement accessible, et toute personne disposant de cette URL peut accéder au fichier.
|
||||
|
||||
Même si le bucket COS est privé, le moteur intégré Tencent COS ne génère pas d’URL signées temporaires pour l’accès aux fichiers. Si vous avez besoin d’un accès privé, utilisez [S3 Pro](./s3-pro). Si des fichiers historiques existent déjà, consultez [Migrer vers S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
:::
|
||||
|
||||
## Paramètres de configuration
|
||||
|
||||
|
||||
|
||||
@@ -44,6 +44,11 @@
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "custom-link",
|
||||
"label": "Migrasi ke S3 Pro",
|
||||
"link": "/file-manager/storage/migrate-to-s3-pro"
|
||||
},
|
||||
{
|
||||
"type": "custom-link",
|
||||
"label": "File Collection",
|
||||
|
||||
@@ -7,6 +7,12 @@ keywords: "field attachment,field attachment,upload file,tipe MIME,storage engin
|
||||
|
||||
# Field Attachment
|
||||
|
||||
:::warning Perhatian
|
||||
|
||||
Field Attachment akan dihentikan pada versi mendatang. Untuk aplikasi baru atau saat menyesuaikan proses yang sudah ada, segera migrasikan ke file collection kustom dan kelola file terkait dengan field relasi.
|
||||
|
||||
:::
|
||||
|
||||
## Pengantar
|
||||
|
||||
Field tipe "Attachment" built-in sistem, digunakan untuk mendukung user upload file di tabel data kustom.
|
||||
|
||||
@@ -9,6 +9,15 @@ keywords: "Aliyun OSS,Aliyun Storage,AccessKey,bucket,object storage,konfigurasi
|
||||
|
||||
Storage engine berdasarkan Aliyun OSS, perlu menyiapkan akun dan permission terkait sebelum digunakan.
|
||||
|
||||
|
||||
:::warning Perhatian
|
||||
|
||||
Engine ini tidak mendukung akses privat. Setelah file di-upload, NocoBase membuat URL yang dapat diakses langsung, dan siapa pun yang memiliki URL tersebut dapat mengakses file.
|
||||
|
||||
Meskipun bucket OSS disetel privat, engine bawaan Aliyun OSS tidak membuat URL bertanda tangan sementara untuk akses file. Jika memerlukan akses privat, gunakan [S3 Pro](./s3-pro). Jika file historis sudah ada, lihat [Migrasi ke S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
:::
|
||||
|
||||
## Parameter Konfigurasi
|
||||
|
||||
|
||||
|
||||
@@ -9,6 +9,15 @@ keywords: "Amazon S3,AWS,bucket,AccessKey,cloud storage,konfigurasi S3,NocoBase"
|
||||
|
||||
Storage engine berdasarkan Amazon S3, perlu menyiapkan akun dan permission terkait sebelum digunakan.
|
||||
|
||||
|
||||
:::warning Perhatian
|
||||
|
||||
Engine ini tidak mendukung akses privat. Setelah file di-upload, NocoBase membuat URL yang dapat diakses langsung, dan siapa pun yang memiliki URL tersebut dapat mengakses file.
|
||||
|
||||
Meskipun bucket S3 disetel privat, engine bawaan Amazon S3 tidak membuat URL bertanda tangan sementara untuk akses file. Jika memerlukan akses privat, gunakan [S3 Pro](./s3-pro). Jika file historis sudah ada, lihat [Migrasi ke S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
:::
|
||||
|
||||
## Parameter Konfigurasi
|
||||
|
||||
|
||||
|
||||
@@ -25,6 +25,29 @@ Saat ini tipe engine yang didukung built-in oleh NocoBase adalah sebagai berikut
|
||||
|
||||
Sistem akan otomatis menambahkan satu local storage engine saat instalasi, yang dapat langsung digunakan. Anda juga dapat menambahkan engine baru atau mengedit parameter engine yang sudah ada.
|
||||
|
||||
|
||||
Jika Anda sudah menggunakan storage engine yang hanya mendukung akses publik dan ingin memigrasikan file historis ke S3 Pro, lihat [Migrasi ke S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
## Aksesibilitas file
|
||||
|
||||
Setiap storage engine mendukung kontrol akses file yang berbeda. Sebelum konfigurasi, pastikan apakah file memerlukan akses privat:
|
||||
|
||||
| Storage engine | Aksesibilitas file |
|
||||
| --- | --- |
|
||||
| [Local Storage](./local) | Hanya mendukung akses publik; akses privat tidak didukung |
|
||||
| [Amazon S3](./amazon-s3) | Hanya mendukung akses publik; akses privat tidak didukung |
|
||||
| [Aliyun OSS](./aliyun-oss) | Hanya mendukung akses publik; akses privat tidak didukung |
|
||||
| [Tencent COS](./tencent-cos) | Hanya mendukung akses publik; akses privat tidak didukung |
|
||||
| [S3 Pro](./s3-pro) | Mendukung akses privat melalui URL bertanda tangan sementara |
|
||||
|
||||
:::warning Perhatian
|
||||
|
||||
Local Storage, Amazon S3, Aliyun OSS, dan Tencent COS tidak melakukan autentikasi login untuk akses file dan tidak membuat URL bertanda tangan sementara. Setelah file di-upload, siapa pun yang memiliki URL akses file dapat mengakses file tersebut secara langsung.
|
||||
|
||||
Jika perlu menyimpan kontrak, dokumen identitas, materi internal, atau file lain yang tidak boleh publik, gunakan [S3 Pro](./s3-pro) dan aktifkan akses privat.
|
||||
|
||||
:::
|
||||
|
||||
## Parameter Umum Engine
|
||||
|
||||
Selain parameter spesifik untuk setiap kategori engine, bagian berikut adalah parameter umum (contoh dengan local storage):
|
||||
|
||||
@@ -9,6 +9,15 @@ keywords: "local storage,Local Storage,hard disk server,storage path,file storag
|
||||
|
||||
File yang di-upload akan disimpan di direktori hard disk lokal server, cocok untuk skenario di mana total file yang dikelola sistem sedikit atau eksperimen.
|
||||
|
||||
|
||||
:::warning Perhatian
|
||||
|
||||
Local Storage tidak mendukung akses privat. Setelah file di-upload, NocoBase membuat URL yang dapat diakses langsung, dan siapa pun yang memiliki URL tersebut dapat mengakses file.
|
||||
|
||||
Jika perlu menyimpan kontrak, dokumen identitas, materi internal, atau file lain yang tidak boleh publik, gunakan [S3 Pro](./s3-pro). Jika file historis sudah ada, lihat [Migrasi ke S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
:::
|
||||
|
||||
## Parameter Konfigurasi
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,270 @@
|
||||
---
|
||||
pkg: '@nocobase/plugin-file-storage-s3-pro'
|
||||
title: "Migrasi ke S3 Pro"
|
||||
description: "Migrate files from public-only storage engines to S3 Pro, including maintenance windows, backups, object key mapping, record updates, and risk checks."
|
||||
keywords: "S3 Pro,storage migration,private access,file migration,Local Storage,S3,OSS,COS,NocoBase"
|
||||
---
|
||||
|
||||
# Migrasi ke S3 Pro
|
||||
|
||||
If you already use [Local Storage](./local.md), [Amazon S3](./amazon-s3.md), [Aliyun OSS](./aliyun-oss.md), or [Tencent COS](./tencent-cos.md), and later need private file access, you can migrate historical files to [S3 Pro](./s3-pro.md).
|
||||
|
||||
This is not only a storage configuration change. A migration must handle three things: make the physical files or objects available from the new storage, update file records, and switch future uploads to the new storage.
|
||||
|
||||
:::warning Perhatian
|
||||
|
||||
Run the migration once in a test environment first, and prepare restorable database and file backups. If users continue to upload or delete attachments during the migration, records can be missed, overwritten, or become inconsistent.
|
||||
|
||||
:::
|
||||
|
||||
## Scope
|
||||
|
||||
This guide applies when migrating from these storage engines to S3 Pro:
|
||||
|
||||
| Source storage engine | Historical file location |
|
||||
| --- | --- |
|
||||
| Local Storage | `<documentRoot>/<record.path>/<record.filename>` |
|
||||
| Amazon S3 | `join(record.path, record.filename)` in the source bucket |
|
||||
| Aliyun OSS | `join(record.path, record.filename)` in the source bucket |
|
||||
| Tencent COS | `join(record.path, record.filename)` in the source bucket |
|
||||
|
||||
Here, `record.path` and `record.filename` come from the file record itself, including the built-in `attachments` table and other file collections.
|
||||
|
||||
:::tip Perhatian
|
||||
|
||||
For Local Storage, `documentRoot` must come from `options.documentRoot` in the corresponding `storages` record. The default value is `storage/uploads`, but the final absolute path depends on the runtime storage directory of NocoBase.
|
||||
|
||||
:::
|
||||
|
||||
## Operation Steps
|
||||
|
||||
### Step 1: Stop writes or enter a maintenance window
|
||||
|
||||
Pause user uploads, updates, and deletions of attachments during the migration. You can use a maintenance window, temporarily disable entry points, or freeze related business flows.
|
||||
|
||||
The goal is to keep file records and physical files stable. If users upload or delete files during migration, the migration script may no longer be working against the latest records.
|
||||
|
||||
### Step 2: Back up the database and files
|
||||
|
||||
Prepare at least two backups:
|
||||
|
||||
1. Database backup
|
||||
2. File backup or object snapshot from the source storage
|
||||
|
||||
For Local Storage, back up the directory pointed to by `documentRoot`. Historical files are usually stored at:
|
||||
|
||||
```text
|
||||
<documentRoot>/<record.path>/<record.filename>
|
||||
```
|
||||
|
||||
For Amazon S3, Aliyun OSS, and Tencent COS, confirm that objects in the source bucket are still readable, and record the source storage engine's `id`, `name`, `type`, `path`, `baseUrl`, and `options`.
|
||||
|
||||
It is also recommended to export a migration manifest for rollback and manual review:
|
||||
|
||||
```text
|
||||
collection
|
||||
id
|
||||
oldStorageId
|
||||
oldPath
|
||||
oldFilename
|
||||
oldUrl
|
||||
newKey
|
||||
size
|
||||
mimetype
|
||||
```
|
||||
|
||||
### Step 3: Create and verify the S3 Pro storage
|
||||
|
||||
Create a new `s3-compatible` storage according to [S3 Pro](./s3-pro.md). At least verify these settings:
|
||||
|
||||
- bucket
|
||||
- endpoint
|
||||
- region
|
||||
- accessKey
|
||||
- secret
|
||||
- public / private
|
||||
- access endpoint
|
||||
- forcePathStyle
|
||||
- file size and MIME type rules
|
||||
|
||||
Upload a test file with the new storage and confirm upload, preview, download, and deletion all work. If the target is private access, also confirm that the access URL is a temporary signed URL and that its expiration is expected.
|
||||
|
||||
:::warning Perhatian
|
||||
|
||||
S3 Pro uses direct client upload. The target bucket must have CORS rules that allow uploads from the NocoBase site, otherwise new uploads will fail.
|
||||
|
||||
:::
|
||||
|
||||
### Step 4: Decide the object key mapping
|
||||
|
||||
For historical files migrated to S3 Pro, it is recommended to continue using the old relative path as the S3 object key:
|
||||
|
||||
```text
|
||||
oldKey = join(record.path, record.filename)
|
||||
```
|
||||
|
||||
For example:
|
||||
|
||||
```text
|
||||
record.path = "avatars"
|
||||
record.filename = "a-123.png"
|
||||
oldKey = "avatars/a-123.png"
|
||||
```
|
||||
|
||||
For persisted file records, S3 Pro uses `file.filename` directly as the full object key when accessing the file. It does not append `file.path`. Therefore, migrated records should be updated to:
|
||||
|
||||
```text
|
||||
storageId = <s3-pro-storage-id>
|
||||
filename = <oldKey>
|
||||
path = ""
|
||||
url = ""
|
||||
```
|
||||
|
||||
Do not migrate to this shape:
|
||||
|
||||
```text
|
||||
filename = "a-123.png"
|
||||
path = "avatars"
|
||||
```
|
||||
|
||||
Otherwise S3 Pro may treat only `a-123.png` as the object key, causing historical files to fail to load.
|
||||
|
||||
Use `/` as the separator when generating object keys. Do not use operating-system-specific path separators. If an old path starts with `/`, remove the leading slash.
|
||||
|
||||
### Step 5: Move physical files or confirm object location
|
||||
|
||||
Traverse all file records, including the built-in `attachments` table and other file collections. Only process records where `storageId = <old-storage-id>`.
|
||||
|
||||
If the source is Local Storage, upload the local files to the bucket used by S3 Pro. If the source is already Amazon S3, Aliyun OSS, or Tencent COS, and the new S3 Pro storage points to the same bucket and endpoint, with credentials that can read the same objects, you usually do not need to copy objects. In that case, confirm that the `oldKey` generated in Step 4 is accessible through S3 Pro.
|
||||
|
||||
| Source storage engine | Whether files need to be copied | Target object key |
|
||||
| --- | --- | --- |
|
||||
| Local Storage | Must be uploaded to the S3 Pro bucket | `join(record.path, record.filename)` |
|
||||
| Amazon S3 | Usually no copy if reusing the same bucket; copy is needed when changing bucket, account, or region | `join(record.path, record.filename)` |
|
||||
| Aliyun OSS | Usually no copy if reusing the same bucket; copy is needed when changing bucket, account, or region | `join(record.path, record.filename)` |
|
||||
| Tencent COS | Usually no copy if reusing the same bucket; copy is needed when changing bucket, account, or region | `join(record.path, record.filename)` |
|
||||
|
||||
Files or objects still need to be copied in these cases:
|
||||
|
||||
- Migrating from Local Storage to S3 Pro
|
||||
- Changing bucket, account, region, or cloud provider
|
||||
- Moving historical objects from a public bucket to a new private bucket
|
||||
- Renaming object keys or reorganizing directories
|
||||
- The source bucket policy is not suitable for S3 Pro signed access or direct client upload
|
||||
- S3 Pro cannot directly access the source objects with the current endpoint and credentials
|
||||
|
||||
Before the real migration, run a dry-run. At least output:
|
||||
|
||||
- Number of records to migrate
|
||||
- Total size to migrate
|
||||
- Number of missing local files or missing source objects
|
||||
- Number of duplicate object keys
|
||||
- Number of records with unrecognized `storageId`
|
||||
- List requiring manual handling
|
||||
|
||||
If duplicate object keys appear, do not overwrite directly. Compare file size, ETag, or hash to confirm whether they point to the same file. If they are different files, generate a new key for one record and use that new key when updating the record.
|
||||
|
||||
### Step 6: Verify target object accessibility
|
||||
|
||||
After files are copied, or after confirming that the source cloud bucket can be reused, run S3 HEAD Object or an equivalent check for every migrated record. Confirm that S3 Pro can access the object through the target object key.
|
||||
|
||||
Recommended output:
|
||||
|
||||
- Success count
|
||||
- Missing source file count
|
||||
- Upload or copy failure count
|
||||
- Missing target object count
|
||||
- Duplicate key count
|
||||
- List requiring manual handling
|
||||
|
||||
Do not rely only on the script exit code. Object storage can have partial failures, successful retries, same-key overwrites, or write-only-but-not-readable permission problems. The migration manifest and HEAD Object checks are more reliable.
|
||||
|
||||
### Step 7: Update file records
|
||||
|
||||
After confirming that all target objects exist, update the file records. Only update records in file collections, keep the original `id`, and do not update many-to-many junction tables generated by attachment fields.
|
||||
|
||||
Core fields:
|
||||
|
||||
| Field | Value after migration |
|
||||
| --- | --- |
|
||||
| `storageId` | New S3 Pro storage ID |
|
||||
| `filename` | `join(oldPath, oldFilename)` |
|
||||
| `path` | Empty string |
|
||||
| `url` | Empty string for private storage; public URL or empty string for public storage |
|
||||
|
||||
If S3 Pro is private, `url` must be empty. S3 Pro also clears `url` before saving non-public storage records. File access will dynamically generate a temporary signed URL.
|
||||
|
||||
:::warning Perhatian
|
||||
|
||||
Batch update records in a transaction if possible, and keep the migration manifest from Step 2 until after the migration is fully verified. Rollback needs that manifest to restore `storageId`, `path`, `filename`, and `url` to their old values.
|
||||
|
||||
:::
|
||||
|
||||
### Step 8: Switch future uploads to the new storage
|
||||
|
||||
After historical records are updated, switch the target storage for new uploads.
|
||||
|
||||
Check three types of configuration:
|
||||
|
||||
1. Default storage engine: if attachment fields or file collections do not specify storage, uploaded files use the default storage. Set the new S3 Pro storage as the default.
|
||||
2. Attachment fields: if an attachment field is configured with `options.storage = <old-storage-name>`, change it to the new S3 Pro storage `name`.
|
||||
3. File collections: if a file collection is configured with the old storage engine, change it to the new S3 Pro storage `name`.
|
||||
|
||||
Here, `name` is the storage engine name, not the storage ID. Historical file records use `storageId`.
|
||||
|
||||
### Step 9: Restart or refresh storage cache
|
||||
|
||||
The file manager caches `storages` configuration. If you update storage configuration through the NocoBase UI or resource API, `reloadStorages` is usually triggered. If you update the database directly, restart the application after migration or make sure the storage cache has been refreshed.
|
||||
|
||||
If you directly changed attachment field metadata or file collection metadata, it is also recommended to restart the application so collection configuration, field configuration, and frontend cache all use the new storage.
|
||||
|
||||
### Step 10: Sample-check and keep old files
|
||||
|
||||
After migration, verify at least these scenarios:
|
||||
|
||||
- View, preview, and download historical files in attachment fields
|
||||
- View, preview, and download historical files in file collections
|
||||
- Image thumbnails or preprocessing parameters
|
||||
- Office preview and other features that depend on external services reading file URLs
|
||||
- Save, preview, download, and delete newly uploaded files
|
||||
- Object deletion after deleting file records
|
||||
|
||||
Do not delete files from the old storage immediately. Keep them until the business side confirms the migration, backup cycles have covered the new state, and access logs show no abnormal behavior.
|
||||
|
||||
## Risks
|
||||
|
||||
### Data inconsistency during writes
|
||||
|
||||
If users can still upload or delete files during migration, objects may be copied after records are deleted, records may be updated while objects remain only in the old storage, or new uploads may not be included in the migration manifest. The default approach is to enter a maintenance window.
|
||||
|
||||
### Different object key rules in S3 Pro
|
||||
|
||||
After migration, `filename` should store the full object key and `path` should be empty. This differs from the historical record shape of Local Storage and built-in S3 / OSS / COS, and is the easiest place to make mistakes.
|
||||
|
||||
### URL accessibility changes
|
||||
|
||||
After migrating from public storage to private S3 Pro, historical public URLs may no longer remain valid. NocoBase dynamically generates temporary signed URLs for internal file access, but old URLs already saved in external systems are not automatically rewritten.
|
||||
|
||||
If third-party systems, email templates, exported files, or rich text content directly stores old URLs, evaluate a separate replacement strategy.
|
||||
|
||||
:::warning Perhatian
|
||||
|
||||
If Markdown (Vditor) field content already stores file URLs, handle those contents separately. Private S3 Pro access generates temporary signed URLs that expire, and currently these private links are not supported for long-term storage and use inside Markdown (Vditor) fields.
|
||||
|
||||
If these fields must continue referencing files, keep publicly accessible file URLs for now.
|
||||
|
||||
:::
|
||||
|
||||
### Preview services and private files
|
||||
|
||||
Some preview features depend on external services accessing file URLs. Private S3 Pro generates temporary signed URLs, which can usually be accessed, but they are affected by signature expiration, network reachability, bucket permissions, and server-side caching.
|
||||
|
||||
If files are very sensitive, reassess whether external preview services should be allowed to read them.
|
||||
|
||||
### Direct database changes do not trigger all hooks
|
||||
|
||||
If you update `storages`, collection configuration, field configuration, and file records directly through SQL or a custom script, some NocoBase cache refreshes and save hooks are not triggered automatically. Restart the application after the update and verify storage configuration again.
|
||||
|
||||
### Delete behavior points to the new storage
|
||||
|
||||
After a file record's `storageId` is updated to S3 Pro, deleting that file record later will make NocoBase try to delete the object from the new bucket. Objects in the old storage will not be deleted automatically and need to be cleaned up separately.
|
||||
@@ -9,6 +9,15 @@ keywords: "Tencent Cloud COS,Tencent Cloud Storage,SecretId,SecretKey,bucket,obj
|
||||
|
||||
Storage engine berdasarkan Tencent Cloud COS, perlu menyiapkan akun dan permission terkait sebelum digunakan.
|
||||
|
||||
|
||||
:::warning Perhatian
|
||||
|
||||
Engine ini tidak mendukung akses privat. Setelah file di-upload, NocoBase membuat URL yang dapat diakses langsung, dan siapa pun yang memiliki URL tersebut dapat mengakses file.
|
||||
|
||||
Meskipun bucket COS disetel privat, engine bawaan Tencent COS tidak membuat URL bertanda tangan sementara untuk akses file. Jika memerlukan akses privat, gunakan [S3 Pro](./s3-pro). Jika file historis sudah ada, lihat [Migrasi ke S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
:::
|
||||
|
||||
## Parameter Konfigurasi
|
||||
|
||||
|
||||
|
||||
@@ -44,6 +44,11 @@
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "custom-link",
|
||||
"label": "S3 Pro への移行",
|
||||
"link": "/file-manager/storage/migrate-to-s3-pro"
|
||||
},
|
||||
{
|
||||
"type": "custom-link",
|
||||
"label": "ファイルテーブル",
|
||||
|
||||
@@ -1,5 +1,11 @@
|
||||
# 添付ファイルフィールド
|
||||
|
||||
:::warning 注意
|
||||
|
||||
添付ファイルフィールドは将来のバージョンで廃止予定です。新規アプリケーションを作成する場合や既存業務を調整する場合は、できるだけ早くカスタムファイルテーブルへ移行し、関連ファイルをリレーションフィールドで管理してください。
|
||||
|
||||
:::
|
||||
|
||||
## はじめに
|
||||
|
||||
NocoBaseには、カスタムコレクションでファイルをアップロードできるように、「添付ファイル」タイプのフィールドが標準で組み込まれています。
|
||||
|
||||
@@ -2,6 +2,15 @@
|
||||
|
||||
Aliyun OSS をベースにしたストレージエンジンです。ご利用の前に、関連するアカウントと権限をご準備いただく必要があります。
|
||||
|
||||
|
||||
:::warning 注意
|
||||
|
||||
このエンジンはプライベートアクセスに対応していません。ファイルのアップロード後、NocoBase は直接アクセス可能な URL を生成し、その URL を知っている人は誰でもファイルにアクセスできます。
|
||||
|
||||
OSS bucket 自体をプライベートに設定していても、NocoBase 組み込みの Aliyun OSS エンジンはファイルアクセス用の一時署名 URL を生成しません。プライベートアクセスが必要な場合は [S3 Pro](./s3-pro) を使用してください。既存ファイルがある場合は、[S3 Pro への移行](./migrate-to-s3-pro.md)を参照してください。
|
||||
|
||||
:::
|
||||
|
||||
## 設定パラメーター
|
||||
|
||||
|
||||
|
||||
@@ -2,6 +2,15 @@
|
||||
|
||||
Amazon S3 をベースにしたストレージエンジンです。ご利用の前に、関連するアカウントと権限をご準備いただく必要があります。
|
||||
|
||||
|
||||
:::warning 注意
|
||||
|
||||
このエンジンはプライベートアクセスに対応していません。ファイルのアップロード後、NocoBase は直接アクセス可能な URL を生成し、その URL を知っている人は誰でもファイルにアクセスできます。
|
||||
|
||||
S3 bucket 自体をプライベートに設定していても、NocoBase 組み込みの Amazon S3 エンジンはファイルアクセス用の一時署名 URL を生成しません。プライベートアクセスが必要な場合は [S3 Pro](./s3-pro) を使用してください。既存ファイルがある場合は、[S3 Pro への移行](./migrate-to-s3-pro.md)を参照してください。
|
||||
|
||||
:::
|
||||
|
||||
## 設定パラメーター
|
||||
|
||||
|
||||
|
||||
@@ -18,6 +18,29 @@
|
||||
|
||||
システムインストール時にローカルストレージエンジンが自動的に追加され、すぐに使用できます。新しいエンジンを追加したり、既存のエンジンのパラメーターを編集したりすることも可能です。
|
||||
|
||||
|
||||
公開アクセスのみをサポートするストレージエンジンをすでに使用していて、既存ファイルを S3 Pro に移行したい場合は、[S3 Pro への移行](./migrate-to-s3-pro.md)を参照してください。
|
||||
|
||||
## ファイルのアクセス性
|
||||
|
||||
ストレージエンジンごとに、ファイルアクセス制御の対応状況が異なります。設定する前に、ファイルにプライベートアクセスが必要か確認してください。
|
||||
|
||||
| ストレージエンジン | ファイルのアクセス性 |
|
||||
| --- | --- |
|
||||
| [Local Storage](./local) | 公開アクセスのみ対応し、プライベートアクセスには対応していません |
|
||||
| [Amazon S3](./amazon-s3) | 公開アクセスのみ対応し、プライベートアクセスには対応していません |
|
||||
| [Aliyun OSS](./aliyun-oss) | 公開アクセスのみ対応し、プライベートアクセスには対応していません |
|
||||
| [Tencent COS](./tencent-cos) | 公開アクセスのみ対応し、プライベートアクセスには対応していません |
|
||||
| [S3 Pro](./s3-pro) | 一時署名 URL によるプライベートアクセスに対応しています |
|
||||
|
||||
:::warning 注意
|
||||
|
||||
ローカルストレージ、Amazon S3、Aliyun OSS、Tencent COS は、ファイルアクセス時にログイン認証を行わず、一時署名 URL も生成しません。ファイルのアップロード後、そのアクセス URL を知っている人は誰でも直接アクセスできます。
|
||||
|
||||
契約書、身分証明書、社内資料など公開すべきでないファイルを保存する場合は、[S3 Pro](./s3-pro) を使用してプライベートアクセスを有効にしてください。
|
||||
|
||||
:::
|
||||
|
||||
## 共通パラメーター
|
||||
|
||||
異なるエンジンの種類に固有のパラメーターに加えて、以下の項目は共通パラメーターです（例としてローカルストレージを使用します）。
|
||||
|
||||
@@ -2,6 +2,15 @@
|
||||
|
||||
アップロードされたファイルは、サーバーのローカルディスクのディレクトリに保存されます。この方法は、システムで管理するアップロードファイルの総量が少ない場合や、実験的な用途に適しています。
|
||||
|
||||
|
||||
:::warning 注意
|
||||
|
||||
ローカルストレージはプライベートアクセスに対応していません。ファイルのアップロード後、NocoBase は直接アクセス可能な URL を生成し、その URL を知っている人は誰でもファイルにアクセスできます。
|
||||
|
||||
契約書、身分証明書、社内資料など公開すべきでないファイルを保存する場合は、[S3 Pro](./s3-pro) を使用してください。既存ファイルがある場合は、[S3 Pro への移行](./migrate-to-s3-pro.md)を参照してください。
|
||||
|
||||
:::
|
||||
|
||||
## 設定パラメーター
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,270 @@
|
||||
---
|
||||
pkg: '@nocobase/plugin-file-storage-s3-pro'
|
||||
title: "S3 Pro への移行"
|
||||
description: "Migrate files from public-only storage engines to S3 Pro, including maintenance windows, backups, object key mapping, record updates, and risk checks."
|
||||
keywords: "S3 Pro,storage migration,private access,file migration,Local Storage,S3,OSS,COS,NocoBase"
|
||||
---
|
||||
|
||||
# S3 Pro への移行
|
||||
|
||||
If you already use [Local Storage](./local.md), [Amazon S3](./amazon-s3.md), [Aliyun OSS](./aliyun-oss.md), or [Tencent COS](./tencent-cos.md), and later need private file access, you can migrate historical files to [S3 Pro](./s3-pro.md).
|
||||
|
||||
This is not only a storage configuration change. A migration must handle three things: make the physical files or objects available from the new storage, update file records, and switch future uploads to the new storage.
|
||||
|
||||
:::warning 注意
|
||||
|
||||
Run the migration once in a test environment first, and prepare restorable database and file backups. If users continue to upload or delete attachments during the migration, records can be missed, overwritten, or become inconsistent.
|
||||
|
||||
:::
|
||||
|
||||
## Scope
|
||||
|
||||
This guide applies when migrating from these storage engines to S3 Pro:
|
||||
|
||||
| Source storage engine | Historical file location |
|
||||
| --- | --- |
|
||||
| Local Storage | `<documentRoot>/<record.path>/<record.filename>` |
|
||||
| Amazon S3 | `join(record.path, record.filename)` in the source bucket |
|
||||
| Aliyun OSS | `join(record.path, record.filename)` in the source bucket |
|
||||
| Tencent COS | `join(record.path, record.filename)` in the source bucket |
|
||||
|
||||
Here, `record.path` and `record.filename` come from the file record itself, including the built-in `attachments` table and other file collections.
|
||||
|
||||
:::tip 注意
|
||||
|
||||
For Local Storage, `documentRoot` must come from `options.documentRoot` in the corresponding `storages` record. The default value is `storage/uploads`, but the final absolute path depends on the runtime storage directory of NocoBase.
|
||||
|
||||
:::
|
||||
|
||||
## Operation Steps
|
||||
|
||||
### Step 1: Stop writes or enter a maintenance window
|
||||
|
||||
Pause user uploads, updates, and deletions of attachments during the migration. You can use a maintenance window, temporarily disable entry points, or freeze related business flows.
|
||||
|
||||
The goal is to keep file records and physical files stable. If users upload or delete files during migration, the migration script may no longer be working against the latest records.
|
||||
|
||||
### Step 2: Back up the database and files
|
||||
|
||||
Prepare at least two backups:
|
||||
|
||||
1. Database backup
|
||||
2. File backup or object snapshot from the source storage
|
||||
|
||||
For Local Storage, back up the directory pointed to by `documentRoot`. Historical files are usually stored at:
|
||||
|
||||
```text
|
||||
<documentRoot>/<record.path>/<record.filename>
|
||||
```
|
||||
|
||||
For Amazon S3, Aliyun OSS, and Tencent COS, confirm that objects in the source bucket are still readable, and record the source storage engine's `id`, `name`, `type`, `path`, `baseUrl`, and `options`.
|
||||
|
||||
It is also recommended to export a migration manifest for rollback and manual review:
|
||||
|
||||
```text
|
||||
collection
|
||||
id
|
||||
oldStorageId
|
||||
oldPath
|
||||
oldFilename
|
||||
oldUrl
|
||||
newKey
|
||||
size
|
||||
mimetype
|
||||
```
|
||||
|
||||
### Step 3: Create and verify the S3 Pro storage
|
||||
|
||||
Create a new `s3-compatible` storage according to [S3 Pro](./s3-pro.md). At least verify these settings:
|
||||
|
||||
- bucket
|
||||
- endpoint
|
||||
- region
|
||||
- accessKey
|
||||
- secret
|
||||
- public / private
|
||||
- access endpoint
|
||||
- forcePathStyle
|
||||
- file size and MIME type rules
|
||||
|
||||
Upload a test file with the new storage and confirm upload, preview, download, and deletion all work. If the target is private access, also confirm that the access URL is a temporary signed URL and that its expiration is expected.
|
||||
|
||||
:::warning 注意
|
||||
|
||||
S3 Pro uses direct client upload. The target bucket must have CORS rules that allow uploads from the NocoBase site, otherwise new uploads will fail.
|
||||
|
||||
:::
|
||||
|
||||
### Step 4: Decide the object key mapping
|
||||
|
||||
For historical files migrated to S3 Pro, it is recommended to continue using the old relative path as the S3 object key:
|
||||
|
||||
```text
|
||||
oldKey = join(record.path, record.filename)
|
||||
```
|
||||
|
||||
For example:
|
||||
|
||||
```text
|
||||
record.path = "avatars"
|
||||
record.filename = "a-123.png"
|
||||
oldKey = "avatars/a-123.png"
|
||||
```
|
||||
|
||||
For persisted file records, S3 Pro uses `file.filename` directly as the full object key when accessing the file. It does not append `file.path`. Therefore, migrated records should be updated to:
|
||||
|
||||
```text
|
||||
storageId = <s3-pro-storage-id>
|
||||
filename = <oldKey>
|
||||
path = ""
|
||||
url = ""
|
||||
```
|
||||
|
||||
Do not migrate to this shape:
|
||||
|
||||
```text
|
||||
filename = "a-123.png"
|
||||
path = "avatars"
|
||||
```
|
||||
|
||||
Otherwise S3 Pro may treat only `a-123.png` as the object key, causing historical files to fail to load.
|
||||
|
||||
Use `/` as the separator when generating object keys. Do not use operating-system-specific path separators. If an old path starts with `/`, remove the leading slash.
|
||||
|
||||
### Step 5: Move physical files or confirm object location
|
||||
|
||||
Traverse all file records, including the built-in `attachments` table and other file collections. Only process records where `storageId = <old-storage-id>`.
|
||||
|
||||
If the source is Local Storage, upload the local files to the bucket used by S3 Pro. If the source is already Amazon S3, Aliyun OSS, or Tencent COS, and the new S3 Pro storage points to the same bucket and endpoint, with credentials that can read the same objects, you usually do not need to copy objects. In that case, confirm that the `oldKey` generated in Step 4 is accessible through S3 Pro.
|
||||
|
||||
| Source storage engine | Whether files need to be copied | Target object key |
|
||||
| --- | --- | --- |
|
||||
| Local Storage | Must be uploaded to the S3 Pro bucket | `join(record.path, record.filename)` |
|
||||
| Amazon S3 | Usually no copy if reusing the same bucket; copy is needed when changing bucket, account, or region | `join(record.path, record.filename)` |
|
||||
| Aliyun OSS | Usually no copy if reusing the same bucket; copy is needed when changing bucket, account, or region | `join(record.path, record.filename)` |
|
||||
| Tencent COS | Usually no copy if reusing the same bucket; copy is needed when changing bucket, account, or region | `join(record.path, record.filename)` |
|
||||
|
||||
Files or objects still need to be copied in these cases:
|
||||
|
||||
- Migrating from Local Storage to S3 Pro
|
||||
- Changing bucket, account, region, or cloud provider
|
||||
- Moving historical objects from a public bucket to a new private bucket
|
||||
- Renaming object keys or reorganizing directories
|
||||
- The source bucket policy is not suitable for S3 Pro signed access or direct client upload
|
||||
- S3 Pro cannot directly access the source objects with the current endpoint and credentials
|
||||
|
||||
Before the real migration, run a dry-run. At least output:
|
||||
|
||||
- Number of records to migrate
|
||||
- Total size to migrate
|
||||
- Number of missing local files or missing source objects
|
||||
- Number of duplicate object keys
|
||||
- Number of records with unrecognized `storageId`
|
||||
- List requiring manual handling
|
||||
|
||||
If duplicate object keys appear, do not overwrite directly. Compare file size, ETag, or hash to confirm whether they point to the same file. If they are different files, generate a new key for one record and use that new key when updating the record.
|
||||
|
||||
### Step 6: Verify target object accessibility
|
||||
|
||||
After files are copied, or after confirming that the source cloud bucket can be reused, run S3 HEAD Object or an equivalent check for every migrated record. Confirm that S3 Pro can access the object through the target object key.
|
||||
|
||||
Recommended output:
|
||||
|
||||
- Success count
|
||||
- Missing source file count
|
||||
- Upload or copy failure count
|
||||
- Missing target object count
|
||||
- Duplicate key count
|
||||
- List requiring manual handling
|
||||
|
||||
Do not rely only on the script exit code. Object storage can have partial failures, successful retries, same-key overwrites, or write-only-but-not-readable permission problems. The migration manifest and HEAD Object checks are more reliable.
|
||||
|
||||
### Step 7: Update file records
|
||||
|
||||
After confirming that all target objects exist, update the file records. Only update records in file collections, keep the original `id`, and do not update many-to-many junction tables generated by attachment fields.
|
||||
|
||||
Core fields:
|
||||
|
||||
| Field | Value after migration |
|
||||
| --- | --- |
|
||||
| `storageId` | New S3 Pro storage ID |
|
||||
| `filename` | `join(oldPath, oldFilename)` |
|
||||
| `path` | Empty string |
|
||||
| `url` | Empty string for private storage; public URL or empty string for public storage |
|
||||
|
||||
If S3 Pro is private, `url` must be empty. S3 Pro also clears `url` before saving non-public storage records. File access will dynamically generate a temporary signed URL.
|
||||
|
||||
:::warning 注意
|
||||
|
||||
Batch update records in a transaction if possible, and keep the migration manifest from Step 2 until after the migration is fully verified. Rollback needs that manifest to restore `storageId`, `path`, `filename`, and `url` to their old values.
|
||||
|
||||
:::
|
||||
|
||||
### Step 8: Switch future uploads to the new storage
|
||||
|
||||
After historical records are updated, switch the target storage for new uploads.
|
||||
|
||||
Check three types of configuration:
|
||||
|
||||
1. Default storage engine: if attachment fields or file collections do not specify storage, uploaded files use the default storage. Set the new S3 Pro storage as the default.
|
||||
2. Attachment fields: if an attachment field is configured with `options.storage = <old-storage-name>`, change it to the new S3 Pro storage `name`.
|
||||
3. File collections: if a file collection is configured with the old storage engine, change it to the new S3 Pro storage `name`.
|
||||
|
||||
Here, `name` is the storage engine name, not the storage ID. Historical file records use `storageId`.
|
||||
|
||||
### Step 9: Restart or refresh storage cache
|
||||
|
||||
The file manager caches `storages` configuration. If you update storage configuration through the NocoBase UI or resource API, `reloadStorages` is usually triggered. If you update the database directly, restart the application after migration or make sure the storage cache has been refreshed.
|
||||
|
||||
If you directly changed attachment field metadata or file collection metadata, it is also recommended to restart the application so collection configuration, field configuration, and frontend cache all use the new storage.
|
||||
|
||||
### Step 10: Sample-check and keep old files
|
||||
|
||||
After migration, verify at least these scenarios:
|
||||
|
||||
- View, preview, and download historical files in attachment fields
|
||||
- View, preview, and download historical files in file collections
|
||||
- Image thumbnails or preprocessing parameters
|
||||
- Office preview and other features that depend on external services reading file URLs
|
||||
- Save, preview, download, and delete newly uploaded files
|
||||
- Object deletion after deleting file records
|
||||
|
||||
Do not delete files from the old storage immediately. Keep them until the business side confirms the migration, backup cycles have covered the new state, and access logs show no abnormal behavior.
|
||||
|
||||
## Risks
|
||||
|
||||
### Data inconsistency during writes
|
||||
|
||||
If users can still upload or delete files during migration, objects may be copied after records are deleted, records may be updated while objects remain only in the old storage, or new uploads may not be included in the migration manifest. The default approach is to enter a maintenance window.
|
||||
|
||||
### Different object key rules in S3 Pro
|
||||
|
||||
After migration, `filename` should store the full object key and `path` should be empty. This differs from the historical record shape of Local Storage and built-in S3 / OSS / COS, and is the easiest place to make mistakes.
|
||||
|
||||
### URL accessibility changes
|
||||
|
||||
After migrating from public storage to private S3 Pro, historical public URLs may no longer remain valid. NocoBase dynamically generates temporary signed URLs for internal file access, but old URLs already saved in external systems are not automatically rewritten.
|
||||
|
||||
If third-party systems, email templates, exported files, or rich text content directly stores old URLs, evaluate a separate replacement strategy.
|
||||
|
||||
:::warning 注意
|
||||
|
||||
If Markdown (Vditor) field content already stores file URLs, handle those contents separately. Private S3 Pro access generates temporary signed URLs that expire, and currently these private links are not supported for long-term storage and use inside Markdown (Vditor) fields.
|
||||
|
||||
If these fields must continue referencing files, keep publicly accessible file URLs for now.
|
||||
|
||||
:::
|
||||
|
||||
### Preview services and private files
|
||||
|
||||
Some preview features depend on external services accessing file URLs. Private S3 Pro generates temporary signed URLs, which can usually be accessed, but they are affected by signature expiration, network reachability, bucket permissions, and server-side caching.
|
||||
|
||||
If files are very sensitive, reassess whether external preview services should be allowed to read them.
|
||||
|
||||
### Direct database changes do not trigger all hooks
|
||||
|
||||
If you update `storages`, collection configuration, field configuration, and file records directly through SQL or a custom script, some NocoBase cache refreshes and save hooks are not triggered automatically. Restart the application after the update and verify storage configuration again.
|
||||
|
||||
### Delete behavior points to the new storage
|
||||
|
||||
After a file record's `storageId` is updated to S3 Pro, deleting that file record later will make NocoBase try to delete the object from the new bucket. Objects in the old storage will not be deleted automatically and need to be cleaned up separately.
|
||||
@@ -2,6 +2,15 @@
|
||||
|
||||
テンセントクラウド COS をベースにしたストレージエンジンです。ご利用の前に、関連するアカウントと権限をご準備いただく必要があります。
|
||||
|
||||
|
||||
:::warning 注意
|
||||
|
||||
このエンジンはプライベートアクセスに対応していません。ファイルのアップロード後、NocoBase は直接アクセス可能な URL を生成し、その URL を知っている人は誰でもファイルにアクセスできます。
|
||||
|
||||
COS bucket 自体をプライベートに設定していても、NocoBase 組み込みの Tencent COS エンジンはファイルアクセス用の一時署名 URL を生成しません。プライベートアクセスが必要な場合は [S3 Pro](./s3-pro) を使用してください。既存ファイルがある場合は、[S3 Pro への移行](./migrate-to-s3-pro.md)を参照してください。
|
||||
|
||||
:::
|
||||
|
||||
## 設定パラメーター
|
||||
|
||||
|
||||
|
||||
@@ -44,6 +44,11 @@
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "custom-link",
|
||||
"label": "Migrar para S3 Pro",
|
||||
"link": "/file-manager/storage/migrate-to-s3-pro"
|
||||
},
|
||||
{
|
||||
"type": "custom-link",
|
||||
"label": "Tabela de Arquivos",
|
||||
|
||||
@@ -1,5 +1,11 @@
|
||||
# Campo de Anexo
|
||||
|
||||
:::warning Observação
|
||||
|
||||
O campo de anexo será descontinuado em uma versão futura. Para novas aplicações ou ao ajustar fluxos existentes, migre o quanto antes para uma tabela de arquivos personalizada e gerencie os arquivos relacionados com campos de relação.
|
||||
|
||||
:::
|
||||
|
||||
## Introdução
|
||||
|
||||
O sistema possui um tipo de campo "Anexo" integrado, que permite aos usuários fazer upload de arquivos em suas **coleções** personalizadas.
|
||||
|
||||
@@ -2,6 +2,15 @@
|
||||
|
||||
Um motor de armazenamento baseado no Aliyun OSS. Antes de usá-lo, você precisará preparar a conta e as permissões necessárias.
|
||||
|
||||
|
||||
:::warning Observação
|
||||
|
||||
Este mecanismo não oferece acesso privado. Depois que um arquivo é enviado, o NocoBase gera uma URL diretamente acessível, e qualquer pessoa com essa URL pode acessar o arquivo.
|
||||
|
||||
Mesmo que o bucket OSS seja privado, o mecanismo integrado Aliyun OSS não gera URLs assinadas temporárias para acesso a arquivos. Se precisar de acesso privado, use [S3 Pro](./s3-pro). Se já houver arquivos históricos, consulte [Migrar para S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
:::
|
||||
|
||||
## Parâmetros de Configuração
|
||||
|
||||
|
||||
|
||||
@@ -2,6 +2,15 @@
|
||||
|
||||
Este é um motor de armazenamento baseado no Amazon S3. Antes de usá-lo, você precisará preparar a conta e as permissões necessárias.
|
||||
|
||||
|
||||
:::warning Observação
|
||||
|
||||
Este mecanismo não oferece acesso privado. Depois que um arquivo é enviado, o NocoBase gera uma URL diretamente acessível, e qualquer pessoa com essa URL pode acessar o arquivo.
|
||||
|
||||
Mesmo que o bucket S3 seja privado, o mecanismo integrado Amazon S3 não gera URLs assinadas temporárias para acesso a arquivos. Se precisar de acesso privado, use [S3 Pro](./s3-pro). Se já houver arquivos históricos, consulte [Migrar para S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
:::
|
||||
|
||||
## Parâmetros de Configuração
|
||||
|
||||
|
||||
|
||||
@@ -18,6 +18,8 @@ Atualmente, o NocoBase oferece suporte integrado para os seguintes tipos de meca
|
||||
|
||||
O sistema adiciona automaticamente um mecanismo de armazenamento local durante a instalação, que pode ser usado diretamente. Você também pode adicionar novos mecanismos ou editar os parâmetros dos existentes.
|
||||
|
||||
|
||||
Se você já usa um mecanismo de armazenamento que oferece apenas acesso público e deseja migrar arquivos históricos para o S3 Pro, consulte [Migrar para S3 Pro](./migrate-to-s3-pro.md).
|
||||
## Parâmetros Comuns
|
||||
|
||||
Além dos parâmetros específicos para diferentes tipos de mecanismos, os seguintes são parâmetros comuns (usando o armazenamento local como exemplo):
|
||||
@@ -64,4 +66,24 @@ Após um arquivo ser enviado, o caminho de acesso final é construído concatena
|
||||
```
|
||||
|
||||
Por exemplo: `https://cdn.nocobase.com/app/user/avatar/20240529115151.png`.
|
||||
:::
|
||||
:::
|
||||
|
||||
## Acessibilidade dos arquivos
|
||||
|
||||
Cada mecanismo de armazenamento oferece controles de acesso diferentes. Antes de configurar, confirme se os arquivos precisam de acesso privado:
|
||||
|
||||
| Mecanismo de armazenamento | Acessibilidade dos arquivos |
|
||||
| --- | --- |
|
||||
| [Local Storage](./local) | Somente acesso público é suportado; acesso privado não é suportado |
|
||||
| [Amazon S3](./amazon-s3) | Somente acesso público é suportado; acesso privado não é suportado |
|
||||
| [Aliyun OSS](./aliyun-oss) | Somente acesso público é suportado; acesso privado não é suportado |
|
||||
| [Tencent COS](./tencent-cos) | Somente acesso público é suportado; acesso privado não é suportado |
|
||||
| [S3 Pro](./s3-pro) | Acesso privado é suportado por URLs assinadas temporárias |
|
||||
|
||||
:::warning Observação
|
||||
|
||||
Armazenamento local, Amazon S3, Aliyun OSS e Tencent COS não fazem autenticação de login para acesso a arquivos e não geram URLs assinadas temporárias. Depois que um arquivo é enviado, qualquer pessoa com a URL de acesso pode acessá-lo diretamente.
|
||||
|
||||
Se você precisa armazenar contratos, documentos de identidade, materiais internos ou outros arquivos que não devem ser públicos, use [S3 Pro](./s3-pro) e ative o acesso privado.
|
||||
|
||||
:::
|
||||
|
||||
@@ -2,6 +2,15 @@
|
||||
|
||||
Os arquivos enviados serão salvos em um diretório local no disco rígido do servidor. Isso é ideal para cenários onde o volume total de arquivos enviados gerenciados pelo sistema é pequeno ou para fins experimentais.
|
||||
|
||||
|
||||
:::warning Observação
|
||||
|
||||
O armazenamento local não oferece acesso privado. Depois que um arquivo é enviado, o NocoBase gera uma URL diretamente acessível, e qualquer pessoa com essa URL pode acessar o arquivo.
|
||||
|
||||
Se você precisa armazenar contratos, documentos de identidade, materiais internos ou outros arquivos que não devem ser públicos, use [S3 Pro](./s3-pro). Se já houver arquivos históricos, consulte [Migrar para S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
:::
|
||||
|
||||
## Parâmetros de Configuração
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,270 @@
|
||||
---
|
||||
pkg: '@nocobase/plugin-file-storage-s3-pro'
|
||||
title: "Migrar para S3 Pro"
|
||||
description: "Migrate files from public-only storage engines to S3 Pro, including maintenance windows, backups, object key mapping, record updates, and risk checks."
|
||||
keywords: "S3 Pro,storage migration,private access,file migration,Local Storage,S3,OSS,COS,NocoBase"
|
||||
---
|
||||
|
||||
# Migrar para S3 Pro
|
||||
|
||||
If you already use [Local Storage](./local.md), [Amazon S3](./amazon-s3.md), [Aliyun OSS](./aliyun-oss.md), or [Tencent COS](./tencent-cos.md), and later need private file access, you can migrate historical files to [S3 Pro](./s3-pro.md).
|
||||
|
||||
This is not only a storage configuration change. A migration must handle three things: make the physical files or objects available from the new storage, update file records, and switch future uploads to the new storage.
|
||||
|
||||
:::warning Observação
|
||||
|
||||
Run the migration once in a test environment first, and prepare restorable database and file backups. If users continue to upload or delete attachments during the migration, records can be missed, overwritten, or become inconsistent.
|
||||
|
||||
:::
|
||||
|
||||
## Scope
|
||||
|
||||
This guide applies when migrating from these storage engines to S3 Pro:
|
||||
|
||||
| Source storage engine | Historical file location |
|
||||
| --- | --- |
|
||||
| Local Storage | `<documentRoot>/<record.path>/<record.filename>` |
|
||||
| Amazon S3 | `join(record.path, record.filename)` in the source bucket |
|
||||
| Aliyun OSS | `join(record.path, record.filename)` in the source bucket |
|
||||
| Tencent COS | `join(record.path, record.filename)` in the source bucket |
|
||||
|
||||
Here, `record.path` and `record.filename` come from the file record itself, including the built-in `attachments` table and other file collections.
|
||||
|
||||
:::tip Observação
|
||||
|
||||
For Local Storage, `documentRoot` must come from `options.documentRoot` in the corresponding `storages` record. The default value is `storage/uploads`, but the final absolute path depends on the runtime storage directory of NocoBase.
|
||||
|
||||
:::
|
||||
|
||||
## Operation Steps
|
||||
|
||||
### Step 1: Stop writes or enter a maintenance window
|
||||
|
||||
Pause user uploads, updates, and deletions of attachments during the migration. You can use a maintenance window, temporarily disable entry points, or freeze related business flows.
|
||||
|
||||
The goal is to keep file records and physical files stable. If users upload or delete files during migration, the migration script may no longer be working against the latest records.
|
||||
|
||||
### Step 2: Back up the database and files
|
||||
|
||||
Prepare at least two backups:
|
||||
|
||||
1. Database backup
|
||||
2. File backup or object snapshot from the source storage
|
||||
|
||||
For Local Storage, back up the directory pointed to by `documentRoot`. Historical files are usually stored at:
|
||||
|
||||
```text
|
||||
<documentRoot>/<record.path>/<record.filename>
|
||||
```
|
||||
|
||||
For Amazon S3, Aliyun OSS, and Tencent COS, confirm that objects in the source bucket are still readable, and record the source storage engine's `id`, `name`, `type`, `path`, `baseUrl`, and `options`.
|
||||
|
||||
It is also recommended to export a migration manifest for rollback and manual review:
|
||||
|
||||
```text
|
||||
collection
|
||||
id
|
||||
oldStorageId
|
||||
oldPath
|
||||
oldFilename
|
||||
oldUrl
|
||||
newKey
|
||||
size
|
||||
mimetype
|
||||
```
|
||||
|
||||
### Step 3: Create and verify the S3 Pro storage
|
||||
|
||||
Create a new `s3-compatible` storage according to [S3 Pro](./s3-pro.md). At least verify these settings:
|
||||
|
||||
- bucket
|
||||
- endpoint
|
||||
- region
|
||||
- accessKey
|
||||
- secret
|
||||
- public / private
|
||||
- access endpoint
|
||||
- forcePathStyle
|
||||
- file size and MIME type rules
|
||||
|
||||
Upload a test file with the new storage and confirm upload, preview, download, and deletion all work. If the target is private access, also confirm that the access URL is a temporary signed URL and that its expiration is expected.
|
||||
|
||||
:::warning Observação
|
||||
|
||||
S3 Pro uses direct client upload. The target bucket must have CORS rules that allow uploads from the NocoBase site, otherwise new uploads will fail.
|
||||
|
||||
:::
|
||||
|
||||
### Step 4: Decide the object key mapping
|
||||
|
||||
For historical files migrated to S3 Pro, it is recommended to continue using the old relative path as the S3 object key:
|
||||
|
||||
```text
|
||||
oldKey = join(record.path, record.filename)
|
||||
```
|
||||
|
||||
For example:
|
||||
|
||||
```text
|
||||
record.path = "avatars"
|
||||
record.filename = "a-123.png"
|
||||
oldKey = "avatars/a-123.png"
|
||||
```
|
||||
|
||||
For persisted file records, S3 Pro uses `file.filename` directly as the full object key when accessing the file. It does not append `file.path`. Therefore, migrated records should be updated to:
|
||||
|
||||
```text
|
||||
storageId = <s3-pro-storage-id>
|
||||
filename = <oldKey>
|
||||
path = ""
|
||||
url = ""
|
||||
```
|
||||
|
||||
Do not migrate to this shape:
|
||||
|
||||
```text
|
||||
filename = "a-123.png"
|
||||
path = "avatars"
|
||||
```
|
||||
|
||||
Otherwise S3 Pro may treat only `a-123.png` as the object key, causing historical files to fail to load.
|
||||
|
||||
Use `/` as the separator when generating object keys. Do not use operating-system-specific path separators. If an old path starts with `/`, remove the leading slash.
|
||||
|
||||
### Step 5: Move physical files or confirm object location
|
||||
|
||||
Traverse all file records, including the built-in `attachments` table and other file collections. Only process records where `storageId = <old-storage-id>`.
|
||||
|
||||
If the source is Local Storage, upload the local files to the bucket used by S3 Pro. If the source is already Amazon S3, Aliyun OSS, or Tencent COS, and the new S3 Pro storage points to the same bucket and endpoint, with credentials that can read the same objects, you usually do not need to copy objects. In that case, confirm that the `oldKey` generated in Step 4 is accessible through S3 Pro.
|
||||
|
||||
| Source storage engine | Whether files need to be copied | Target object key |
|
||||
| --- | --- | --- |
|
||||
| Local Storage | Must be uploaded to the S3 Pro bucket | `join(record.path, record.filename)` |
|
||||
| Amazon S3 | Usually no copy if reusing the same bucket; copy is needed when changing bucket, account, or region | `join(record.path, record.filename)` |
|
||||
| Aliyun OSS | Usually no copy if reusing the same bucket; copy is needed when changing bucket, account, or region | `join(record.path, record.filename)` |
|
||||
| Tencent COS | Usually no copy if reusing the same bucket; copy is needed when changing bucket, account, or region | `join(record.path, record.filename)` |
|
||||
|
||||
Files or objects still need to be copied in these cases:
|
||||
|
||||
- Migrating from Local Storage to S3 Pro
|
||||
- Changing bucket, account, region, or cloud provider
|
||||
- Moving historical objects from a public bucket to a new private bucket
|
||||
- Renaming object keys or reorganizing directories
|
||||
- The source bucket policy is not suitable for S3 Pro signed access or direct client upload
|
||||
- S3 Pro cannot directly access the source objects with the current endpoint and credentials
|
||||
|
||||
Before the real migration, run a dry-run. At least output:
|
||||
|
||||
- Number of records to migrate
|
||||
- Total size to migrate
|
||||
- Number of missing local files or missing source objects
|
||||
- Number of duplicate object keys
|
||||
- Number of records with unrecognized `storageId`
|
||||
- List requiring manual handling
|
||||
|
||||
If duplicate object keys appear, do not overwrite directly. Compare file size, ETag, or hash to confirm whether they point to the same file. If they are different files, generate a new key for one record and use that new key when updating the record.
|
||||
|
||||
### Step 6: Verify target object accessibility
|
||||
|
||||
After files are copied, or after confirming that the source cloud bucket can be reused, run S3 HEAD Object or an equivalent check for every migrated record. Confirm that S3 Pro can access the object through the target object key.
|
||||
|
||||
Recommended output:
|
||||
|
||||
- Success count
|
||||
- Missing source file count
|
||||
- Upload or copy failure count
|
||||
- Missing target object count
|
||||
- Duplicate key count
|
||||
- List requiring manual handling
|
||||
|
||||
Do not rely only on the script exit code. Object storage can have partial failures, successful retries, same-key overwrites, or write-only-but-not-readable permission problems. The migration manifest and HEAD Object checks are more reliable.
|
||||
|
||||
### Step 7: Update file records
|
||||
|
||||
After confirming that all target objects exist, update the file records. Only update records in file collections, keep the original `id`, and do not update many-to-many junction tables generated by attachment fields.
|
||||
|
||||
Core fields:
|
||||
|
||||
| Field | Value after migration |
|
||||
| --- | --- |
|
||||
| `storageId` | New S3 Pro storage ID |
|
||||
| `filename` | `join(oldPath, oldFilename)` |
|
||||
| `path` | Empty string |
|
||||
| `url` | Empty string for private storage; public URL or empty string for public storage |
|
||||
|
||||
If S3 Pro is private, `url` must be empty. S3 Pro also clears `url` before saving non-public storage records. File access will dynamically generate a temporary signed URL.
|
||||
|
||||
:::warning Observação
|
||||
|
||||
Batch update records in a transaction if possible, and keep the migration manifest from Step 2 until after the migration is fully verified. Rollback needs that manifest to restore `storageId`, `path`, `filename`, and `url` to their old values.
|
||||
|
||||
:::
|
||||
|
||||
### Step 8: Switch future uploads to the new storage
|
||||
|
||||
After historical records are updated, switch the target storage for new uploads.
|
||||
|
||||
Check three types of configuration:
|
||||
|
||||
1. Default storage engine: if attachment fields or file collections do not specify storage, uploaded files use the default storage. Set the new S3 Pro storage as the default.
|
||||
2. Attachment fields: if an attachment field is configured with `options.storage = <old-storage-name>`, change it to the new S3 Pro storage `name`.
|
||||
3. File collections: if a file collection is configured with the old storage engine, change it to the new S3 Pro storage `name`.
|
||||
|
||||
Here, `name` is the storage engine name, not the storage ID. Historical file records use `storageId`.
|
||||
|
||||
### Step 9: Restart or refresh storage cache
|
||||
|
||||
The file manager caches `storages` configuration. If you update storage configuration through the NocoBase UI or resource API, `reloadStorages` is usually triggered. If you update the database directly, restart the application after migration or make sure the storage cache has been refreshed.
|
||||
|
||||
If you directly changed attachment field metadata or file collection metadata, it is also recommended to restart the application so collection configuration, field configuration, and frontend cache all use the new storage.
|
||||
|
||||
### Step 10: Sample-check and keep old files
|
||||
|
||||
After migration, verify at least these scenarios:
|
||||
|
||||
- View, preview, and download historical files in attachment fields
|
||||
- View, preview, and download historical files in file collections
|
||||
- Image thumbnails or preprocessing parameters
|
||||
- Office preview and other features that depend on external services reading file URLs
|
||||
- Save, preview, download, and delete newly uploaded files
|
||||
- Object deletion after deleting file records
|
||||
|
||||
Do not delete files from the old storage immediately. Keep them until the business side confirms the migration, backup cycles have covered the new state, and access logs show no abnormal behavior.
|
||||
|
||||
## Risks
|
||||
|
||||
### Data inconsistency during writes
|
||||
|
||||
If users can still upload or delete files during migration, objects may be copied after records are deleted, records may be updated while objects remain only in the old storage, or new uploads may not be included in the migration manifest. The default approach is to enter a maintenance window.
|
||||
|
||||
### Different object key rules in S3 Pro
|
||||
|
||||
After migration, `filename` should store the full object key and `path` should be empty. This differs from the historical record shape of Local Storage and built-in S3 / OSS / COS, and is the easiest place to make mistakes.
|
||||
|
||||
### URL accessibility changes
|
||||
|
||||
After migrating from public storage to private S3 Pro, historical public URLs may no longer remain valid. NocoBase dynamically generates temporary signed URLs for internal file access, but old URLs already saved in external systems are not automatically rewritten.
|
||||
|
||||
If third-party systems, email templates, exported files, or rich text content directly stores old URLs, evaluate a separate replacement strategy.
|
||||
|
||||
:::warning Observação
|
||||
|
||||
If Markdown (Vditor) field content already stores file URLs, handle those contents separately. Private S3 Pro access generates temporary signed URLs that expire, and currently these private links are not supported for long-term storage and use inside Markdown (Vditor) fields.
|
||||
|
||||
If these fields must continue referencing files, keep publicly accessible file URLs for now.
|
||||
|
||||
:::
|
||||
|
||||
### Preview services and private files
|
||||
|
||||
Some preview features depend on external services accessing file URLs. Private S3 Pro generates temporary signed URLs, which can usually be accessed, but they are affected by signature expiration, network reachability, bucket permissions, and server-side caching.
|
||||
|
||||
If files are very sensitive, reassess whether external preview services should be allowed to read them.
|
||||
|
||||
### Direct database changes do not trigger all hooks
|
||||
|
||||
If you update `storages`, collection configuration, field configuration, and file records directly through SQL or a custom script, some NocoBase cache refreshes and save hooks are not triggered automatically. Restart the application after the update and verify storage configuration again.
|
||||
|
||||
### Delete behavior points to the new storage
|
||||
|
||||
After a file record's `storageId` is updated to S3 Pro, deleting that file record later will make NocoBase try to delete the object from the new bucket. Objects in the old storage will not be deleted automatically and need to be cleaned up separately.
|
||||
@@ -2,6 +2,15 @@
|
||||
|
||||
Um motor de armazenamento baseado no Tencent Cloud COS. Antes de usar, você precisa preparar a conta e as permissões relevantes.
|
||||
|
||||
|
||||
:::warning Observação
|
||||
|
||||
Este mecanismo não oferece acesso privado. Depois que um arquivo é enviado, o NocoBase gera uma URL diretamente acessível, e qualquer pessoa com essa URL pode acessar o arquivo.
|
||||
|
||||
Mesmo que o bucket COS seja privado, o mecanismo integrado Tencent COS não gera URLs assinadas temporárias para acesso a arquivos. Se precisar de acesso privado, use [S3 Pro](./s3-pro). Se já houver arquivos históricos, consulte [Migrar para S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
:::
|
||||
|
||||
## Parâmetros de Configuração
|
||||
|
||||
|
||||
|
||||
@@ -44,6 +44,11 @@
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "custom-link",
|
||||
"label": "Миграция на S3 Pro",
|
||||
"link": "/file-manager/storage/migrate-to-s3-pro"
|
||||
},
|
||||
{
|
||||
"type": "custom-link",
|
||||
"label": "Таблица файлов",
|
||||
|
||||
@@ -1,5 +1,11 @@
|
||||
# Поле "Вложение"
|
||||
|
||||
:::warning Примечание
|
||||
|
||||
Поле вложения будет устаревшим в будущей версии. Для новых приложений или при изменении существующих процессов как можно раньше перейдите на пользовательскую таблицу файлов и управляйте связанными файлами через поля отношений.
|
||||
|
||||
:::
|
||||
|
||||
## Введение
|
||||
|
||||
В системе предусмотрен встроенный тип поля "Вложение", который позволяет пользователям загружать файлы в пользовательские коллекции.
|
||||
|
||||
@@ -2,6 +2,15 @@
|
||||
|
||||
Это механизм хранения, основанный на Aliyun OSS. Перед использованием необходимо подготовить соответствующую учетную запись и предоставить необходимые разрешения.
|
||||
|
||||
|
||||
:::warning Примечание
|
||||
|
||||
Этот движок не поддерживает приватный доступ. После загрузки файла NocoBase создает URL прямого доступа, и любой, у кого есть этот URL, может открыть файл.
|
||||
|
||||
Даже если bucket OSS настроен как приватный, встроенный движок Aliyun OSS не создает временные подписанные URL для доступа к файлам. Если нужен приватный доступ, используйте [S3 Pro](./s3-pro). Если исторические файлы уже есть, см. [Миграция на S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
:::
|
||||
|
||||
## Параметры конфигурации
|
||||
|
||||
|
||||
|
||||
@@ -2,6 +2,15 @@
|
||||
|
||||
Движок хранения на базе Amazon S3. Перед использованием вам потребуется подготовить соответствующую учетную запись и разрешения.
|
||||
|
||||
|
||||
:::warning Примечание
|
||||
|
||||
Этот движок не поддерживает приватный доступ. После загрузки файла NocoBase создает URL прямого доступа, и любой, у кого есть этот URL, может открыть файл.
|
||||
|
||||
Даже если bucket S3 настроен как приватный, встроенный движок Amazon S3 не создает временные подписанные URL для доступа к файлам. Если нужен приватный доступ, используйте [S3 Pro](./s3-pro). Если исторические файлы уже есть, см. [Миграция на S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
:::
|
||||
|
||||
## Параметры конфигурации
|
||||
|
||||
|
||||
|
||||
@@ -18,6 +18,29 @@
|
||||
|
||||
При установке система автоматически добавляет движок локального хранения, который можно использовать сразу. Вы также можете добавить новые движки или изменить параметры существующих.
|
||||
|
||||
|
||||
Если вы уже используете движок хранения, который поддерживает только публичный доступ, и хотите перенести исторические файлы на S3 Pro, см. [Миграция на S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
## Доступность файлов
|
||||
|
||||
Разные движки хранения поддерживают разные механизмы контроля доступа к файлам. Перед настройкой проверьте, нужен ли файлам приватный доступ:
|
||||
|
||||
| Движок хранения | Доступность файлов |
|
||||
| --- | --- |
|
||||
| [Local Storage](./local) | Поддерживается только публичный доступ; приватный доступ не поддерживается |
|
||||
| [Amazon S3](./amazon-s3) | Поддерживается только публичный доступ; приватный доступ не поддерживается |
|
||||
| [Aliyun OSS](./aliyun-oss) | Поддерживается только публичный доступ; приватный доступ не поддерживается |
|
||||
| [Tencent COS](./tencent-cos) | Поддерживается только публичный доступ; приватный доступ не поддерживается |
|
||||
| [S3 Pro](./s3-pro) | Приватный доступ поддерживается через временные подписанные URL |
|
||||
|
||||
:::warning Примечание
|
||||
|
||||
Локальное хранилище, Amazon S3, Aliyun OSS и Tencent COS не выполняют проверку входа для доступа к файлам и не создают временные подписанные URL. После загрузки файла любой, у кого есть URL доступа, может открыть файл напрямую.
|
||||
|
||||
Если нужно хранить договоры, документы, внутренние материалы или другие непубличные файлы, используйте [S3 Pro](./s3-pro) и включите приватный доступ.
|
||||
|
||||
:::
|
||||
|
||||
## Общие параметры
|
||||
|
||||
Помимо специфических параметров для каждого типа движка, ниже представлены общие параметры (на примере локального хранилища):
|
||||
|
||||
@@ -2,6 +2,15 @@
|
||||
|
||||
Загруженные файлы будут сохраняться в локальном каталоге на жёстком диске сервера. Этот вариант подходит для сценариев с небольшим общим объёмом загружаемых файлов, управляемых системой, или для экспериментальных целей.
|
||||
|
||||
|
||||
:::warning Примечание
|
||||
|
||||
Локальное хранилище не поддерживает приватный доступ. После загрузки файла NocoBase создает URL прямого доступа, и любой, у кого есть этот URL, может открыть файл.
|
||||
|
||||
Если нужно хранить договоры, документы, внутренние материалы или другие непубличные файлы, используйте [S3 Pro](./s3-pro). Если исторические файлы уже есть, см. [Миграция на S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
:::
|
||||
|
||||
## Параметры конфигурации
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,270 @@
|
||||
---
|
||||
pkg: '@nocobase/plugin-file-storage-s3-pro'
|
||||
title: "Миграция на S3 Pro"
|
||||
description: "Migrate files from public-only storage engines to S3 Pro, including maintenance windows, backups, object key mapping, record updates, and risk checks."
|
||||
keywords: "S3 Pro,storage migration,private access,file migration,Local Storage,S3,OSS,COS,NocoBase"
|
||||
---
|
||||
|
||||
# Миграция на S3 Pro
|
||||
|
||||
If you already use [Local Storage](./local.md), [Amazon S3](./amazon-s3.md), [Aliyun OSS](./aliyun-oss.md), or [Tencent COS](./tencent-cos.md), and later need private file access, you can migrate historical files to [S3 Pro](./s3-pro.md).
|
||||
|
||||
This is not only a storage configuration change. A migration must handle three things: make the physical files or objects available from the new storage, update file records, and switch future uploads to the new storage.
|
||||
|
||||
:::warning Примечание
|
||||
|
||||
Run the migration once in a test environment first, and prepare restorable database and file backups. If users continue to upload or delete attachments during the migration, records can be missed, overwritten, or become inconsistent.
|
||||
|
||||
:::
|
||||
|
||||
## Scope
|
||||
|
||||
This guide applies when migrating from these storage engines to S3 Pro:
|
||||
|
||||
| Source storage engine | Historical file location |
|
||||
| --- | --- |
|
||||
| Local Storage | `<documentRoot>/<record.path>/<record.filename>` |
|
||||
| Amazon S3 | `join(record.path, record.filename)` in the source bucket |
|
||||
| Aliyun OSS | `join(record.path, record.filename)` in the source bucket |
|
||||
| Tencent COS | `join(record.path, record.filename)` in the source bucket |
|
||||
|
||||
Here, `record.path` and `record.filename` come from the file record itself, including the built-in `attachments` table and other file collections.
|
||||
|
||||
:::tip Примечание
|
||||
|
||||
For Local Storage, `documentRoot` must come from `options.documentRoot` in the corresponding `storages` record. The default value is `storage/uploads`, but the final absolute path depends on the runtime storage directory of NocoBase.
|
||||
|
||||
:::
|
||||
|
||||
## Operation Steps
|
||||
|
||||
### Step 1: Stop writes or enter a maintenance window
|
||||
|
||||
Pause user uploads, updates, and deletions of attachments during the migration. You can use a maintenance window, temporarily disable entry points, or freeze related business flows.
|
||||
|
||||
The goal is to keep file records and physical files stable. If users upload or delete files during migration, the migration script may no longer be working against the latest records.
|
||||
|
||||
### Step 2: Back up the database and files
|
||||
|
||||
Prepare at least two backups:
|
||||
|
||||
1. Database backup
|
||||
2. File backup or object snapshot from the source storage
|
||||
|
||||
For Local Storage, back up the directory pointed to by `documentRoot`. Historical files are usually stored at:
|
||||
|
||||
```text
|
||||
<documentRoot>/<record.path>/<record.filename>
|
||||
```
|
||||
|
||||
For Amazon S3, Aliyun OSS, and Tencent COS, confirm that objects in the source bucket are still readable, and record the source storage engine's `id`, `name`, `type`, `path`, `baseUrl`, and `options`.
|
||||
|
||||
It is also recommended to export a migration manifest for rollback and manual review:
|
||||
|
||||
```text
|
||||
collection
|
||||
id
|
||||
oldStorageId
|
||||
oldPath
|
||||
oldFilename
|
||||
oldUrl
|
||||
newKey
|
||||
size
|
||||
mimetype
|
||||
```
|
||||
|
||||
### Step 3: Create and verify the S3 Pro storage
|
||||
|
||||
Create a new `s3-compatible` storage according to [S3 Pro](./s3-pro.md). At least verify these settings:
|
||||
|
||||
- bucket
|
||||
- endpoint
|
||||
- region
|
||||
- accessKey
|
||||
- secret
|
||||
- public / private
|
||||
- access endpoint
|
||||
- forcePathStyle
|
||||
- file size and MIME type rules
|
||||
|
||||
Upload a test file with the new storage and confirm upload, preview, download, and deletion all work. If the target is private access, also confirm that the access URL is a temporary signed URL and that its expiration is expected.
|
||||
|
||||
:::warning Примечание
|
||||
|
||||
S3 Pro uses direct client upload. The target bucket must have CORS rules that allow uploads from the NocoBase site, otherwise new uploads will fail.
|
||||
|
||||
:::
|
||||
|
||||
### Step 4: Decide the object key mapping
|
||||
|
||||
For historical files migrated to S3 Pro, it is recommended to continue using the old relative path as the S3 object key:
|
||||
|
||||
```text
|
||||
oldKey = join(record.path, record.filename)
|
||||
```
|
||||
|
||||
For example:
|
||||
|
||||
```text
|
||||
record.path = "avatars"
|
||||
record.filename = "a-123.png"
|
||||
oldKey = "avatars/a-123.png"
|
||||
```
|
||||
|
||||
For persisted file records, S3 Pro uses `file.filename` directly as the full object key when accessing the file. It does not append `file.path`. Therefore, migrated records should be updated to:
|
||||
|
||||
```text
|
||||
storageId = <s3-pro-storage-id>
|
||||
filename = <oldKey>
|
||||
path = ""
|
||||
url = ""
|
||||
```
|
||||
|
||||
Do not migrate to this shape:
|
||||
|
||||
```text
|
||||
filename = "a-123.png"
|
||||
path = "avatars"
|
||||
```
|
||||
|
||||
Otherwise S3 Pro may treat only `a-123.png` as the object key, causing historical files to fail to load.
|
||||
|
||||
Use `/` as the separator when generating object keys. Do not use operating-system-specific path separators. If an old path starts with `/`, remove the leading slash.
|
||||
|
||||
### Step 5: Move physical files or confirm object location
|
||||
|
||||
Traverse all file records, including the built-in `attachments` table and other file collections. Only process records where `storageId = <old-storage-id>`.
|
||||
|
||||
If the source is Local Storage, upload the local files to the bucket used by S3 Pro. If the source is already Amazon S3, Aliyun OSS, or Tencent COS, and the new S3 Pro storage points to the same bucket and endpoint, with credentials that can read the same objects, you usually do not need to copy objects. In that case, confirm that the `oldKey` generated in Step 4 is accessible through S3 Pro.
|
||||
|
||||
| Source storage engine | Whether files need to be copied | Target object key |
|
||||
| --- | --- | --- |
|
||||
| Local Storage | Must be uploaded to the S3 Pro bucket | `join(record.path, record.filename)` |
|
||||
| Amazon S3 | Usually no copy if reusing the same bucket; copy is needed when changing bucket, account, or region | `join(record.path, record.filename)` |
|
||||
| Aliyun OSS | Usually no copy if reusing the same bucket; copy is needed when changing bucket, account, or region | `join(record.path, record.filename)` |
|
||||
| Tencent COS | Usually no copy if reusing the same bucket; copy is needed when changing bucket, account, or region | `join(record.path, record.filename)` |
|
||||
|
||||
Files or objects still need to be copied in these cases:
|
||||
|
||||
- Migrating from Local Storage to S3 Pro
|
||||
- Changing bucket, account, region, or cloud provider
|
||||
- Moving historical objects from a public bucket to a new private bucket
|
||||
- Renaming object keys or reorganizing directories
|
||||
- The source bucket policy is not suitable for S3 Pro signed access or direct client upload
|
||||
- S3 Pro cannot directly access the source objects with the current endpoint and credentials
|
||||
|
||||
Before the real migration, run a dry-run. At least output:
|
||||
|
||||
- Number of records to migrate
|
||||
- Total size to migrate
|
||||
- Number of missing local files or missing source objects
|
||||
- Number of duplicate object keys
|
||||
- Number of records with unrecognized `storageId`
|
||||
- List requiring manual handling
|
||||
|
||||
If duplicate object keys appear, do not overwrite directly. Compare file size, ETag, or hash to confirm whether they point to the same file. If they are different files, generate a new key for one record and use that new key when updating the record.
|
||||
|
||||
### Step 6: Verify target object accessibility
|
||||
|
||||
After files are copied, or after confirming that the source cloud bucket can be reused, run S3 HEAD Object or an equivalent check for every migrated record. Confirm that S3 Pro can access the object through the target object key.
|
||||
|
||||
Recommended output:
|
||||
|
||||
- Success count
|
||||
- Missing source file count
|
||||
- Upload or copy failure count
|
||||
- Missing target object count
|
||||
- Duplicate key count
|
||||
- List requiring manual handling
|
||||
|
||||
Do not rely only on the script exit code. Object storage can have partial failures, successful retries, same-key overwrites, or write-only-but-not-readable permission problems. The migration manifest and HEAD Object checks are more reliable.
|
||||
|
||||
### Step 7: Update file records
|
||||
|
||||
After confirming that all target objects exist, update the file records. Only update records in file collections, keep the original `id`, and do not update many-to-many junction tables generated by attachment fields.
|
||||
|
||||
Core fields:
|
||||
|
||||
| Field | Value after migration |
|
||||
| --- | --- |
|
||||
| `storageId` | New S3 Pro storage ID |
|
||||
| `filename` | `join(oldPath, oldFilename)` |
|
||||
| `path` | Empty string |
|
||||
| `url` | Empty string for private storage; public URL or empty string for public storage |
|
||||
|
||||
If S3 Pro is private, `url` must be empty. S3 Pro also clears `url` before saving non-public storage records. File access will dynamically generate a temporary signed URL.
|
||||
|
||||
:::warning Примечание
|
||||
|
||||
Batch update records in a transaction if possible, and keep the migration manifest from Step 2 until after the migration is fully verified. Rollback needs that manifest to restore `storageId`, `path`, `filename`, and `url` to their old values.
|
||||
|
||||
:::
|
||||
|
||||
### Step 8: Switch future uploads to the new storage
|
||||
|
||||
After historical records are updated, switch the target storage for new uploads.
|
||||
|
||||
Check three types of configuration:
|
||||
|
||||
1. Default storage engine: if attachment fields or file collections do not specify storage, uploaded files use the default storage. Set the new S3 Pro storage as the default.
|
||||
2. Attachment fields: if an attachment field is configured with `options.storage = <old-storage-name>`, change it to the new S3 Pro storage `name`.
|
||||
3. File collections: if a file collection is configured with the old storage engine, change it to the new S3 Pro storage `name`.
|
||||
|
||||
Here, `name` is the storage engine name, not the storage ID. Historical file records use `storageId`.
|
||||
|
||||
### Step 9: Restart or refresh storage cache
|
||||
|
||||
The file manager caches `storages` configuration. If you update storage configuration through the NocoBase UI or resource API, `reloadStorages` is usually triggered. If you update the database directly, restart the application after migration or make sure the storage cache has been refreshed.
|
||||
|
||||
If you directly changed attachment field metadata or file collection metadata, it is also recommended to restart the application so collection configuration, field configuration, and frontend cache all use the new storage.
|
||||
|
||||
### Step 10: Sample-check and keep old files
|
||||
|
||||
After migration, verify at least these scenarios:
|
||||
|
||||
- View, preview, and download historical files in attachment fields
|
||||
- View, preview, and download historical files in file collections
|
||||
- Image thumbnails or preprocessing parameters
|
||||
- Office preview and other features that depend on external services reading file URLs
|
||||
- Save, preview, download, and delete newly uploaded files
|
||||
- Object deletion after deleting file records
|
||||
|
||||
Do not delete files from the old storage immediately. Keep them until the business side confirms the migration, backup cycles have covered the new state, and access logs show no abnormal behavior.
|
||||
|
||||
## Risks
|
||||
|
||||
### Data inconsistency during writes
|
||||
|
||||
If users can still upload or delete files during migration, objects may be copied after records are deleted, records may be updated while objects remain only in the old storage, or new uploads may not be included in the migration manifest. The default approach is to enter a maintenance window.
|
||||
|
||||
### Different object key rules in S3 Pro
|
||||
|
||||
After migration, `filename` should store the full object key and `path` should be empty. This differs from the historical record shape of Local Storage and built-in S3 / OSS / COS, and is the easiest place to make mistakes.
|
||||
|
||||
### URL accessibility changes
|
||||
|
||||
After migrating from public storage to private S3 Pro, historical public URLs may no longer remain valid. NocoBase dynamically generates temporary signed URLs for internal file access, but old URLs already saved in external systems are not automatically rewritten.
|
||||
|
||||
If third-party systems, email templates, exported files, or rich text content directly stores old URLs, evaluate a separate replacement strategy.
|
||||
|
||||
:::warning Примечание
|
||||
|
||||
If Markdown (Vditor) field content already stores file URLs, handle those contents separately. Private S3 Pro access generates temporary signed URLs that expire, and currently these private links are not supported for long-term storage and use inside Markdown (Vditor) fields.
|
||||
|
||||
If these fields must continue referencing files, keep publicly accessible file URLs for now.
|
||||
|
||||
:::
|
||||
|
||||
### Preview services and private files
|
||||
|
||||
Some preview features depend on external services accessing file URLs. Private S3 Pro generates temporary signed URLs, which can usually be accessed, but they are affected by signature expiration, network reachability, bucket permissions, and server-side caching.
|
||||
|
||||
If files are very sensitive, reassess whether external preview services should be allowed to read them.
|
||||
|
||||
### Direct database changes do not trigger all hooks
|
||||
|
||||
If you update `storages`, collection configuration, field configuration, and file records directly through SQL or a custom script, some NocoBase cache refreshes and save hooks are not triggered automatically. Restart the application after the update and verify storage configuration again.
|
||||
|
||||
### Delete behavior points to the new storage
|
||||
|
||||
After a file record's `storageId` is updated to S3 Pro, deleting that file record later will make NocoBase try to delete the object from the new bucket. Objects in the old storage will not be deleted automatically and need to be cleaned up separately.
|
||||
@@ -2,6 +2,15 @@
|
||||
|
||||
Механизм хранения, основанный на Tencent Cloud COS. Перед использованием вам необходимо подготовить соответствующую учетную запись и разрешения.
|
||||
|
||||
|
||||
:::warning Примечание
|
||||
|
||||
Этот движок не поддерживает приватный доступ. После загрузки файла NocoBase создает URL прямого доступа, и любой, у кого есть этот URL, может открыть файл.
|
||||
|
||||
Даже если bucket COS настроен как приватный, встроенный движок Tencent COS не создает временные подписанные URL для доступа к файлам. Если нужен приватный доступ, используйте [S3 Pro](./s3-pro). Если исторические файлы уже есть, см. [Миграция на S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
:::
|
||||
|
||||
## Параметры конфигурации
|
||||
|
||||
|
||||
|
||||
@@ -44,6 +44,11 @@
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "custom-link",
|
||||
"label": "Di chuyển sang S3 Pro",
|
||||
"link": "/file-manager/storage/migrate-to-s3-pro"
|
||||
},
|
||||
{
|
||||
"type": "custom-link",
|
||||
"label": "File Collection",
|
||||
|
||||
@@ -7,6 +7,12 @@ keywords: "field attachment,field attachment,upload file,loại MIME,storage eng
|
||||
|
||||
# Field Attachment
|
||||
|
||||
:::warning Lưu ý
|
||||
|
||||
Field Attachment sẽ bị loại bỏ trong phiên bản tương lai. Với ứng dụng mới hoặc khi điều chỉnh nghiệp vụ hiện có, hãy sớm chuyển sang file collection tùy chỉnh và quản lý file liên quan bằng field quan hệ.
|
||||
|
||||
:::
|
||||
|
||||
## Giới thiệu
|
||||
|
||||
Hệ thống có sẵn loại field "Attachment" để hỗ trợ người dùng upload file trong các bảng dữ liệu tùy chỉnh.
|
||||
|
||||
@@ -9,6 +9,15 @@ keywords: "Aliyun OSS,Aliyun,AccessKey,Bucket,Object Storage,Cấu hình OSS,Noc
|
||||
|
||||
Storage engine dựa trên Aliyun OSS, cần chuẩn bị tài khoản và quyền liên quan trước khi sử dụng.
|
||||
|
||||
|
||||
:::warning Lưu ý
|
||||
|
||||
Engine này không hỗ trợ truy cập riêng tư. Sau khi file được upload, NocoBase tạo URL có thể truy cập trực tiếp, và bất kỳ ai có URL đó đều có thể truy cập file.
|
||||
|
||||
Ngay cả khi bucket OSS được cấu hình riêng tư, engine Aliyun OSS tích hợp sẵn cũng không tạo URL ký tạm thời để truy cập file. Nếu cần truy cập riêng tư, hãy dùng [S3 Pro](./s3-pro). Nếu đã có file lịch sử, hãy xem [Di chuyển sang S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
:::
|
||||
|
||||
## Tham số cấu hình
|
||||
|
||||
|
||||
|
||||
@@ -9,6 +9,15 @@ keywords: "Amazon S3,AWS,Bucket,AccessKey,Cloud Storage,Cấu hình S3,NocoBase"
|
||||
|
||||
Storage engine dựa trên Amazon S3, cần chuẩn bị tài khoản và quyền liên quan trước khi sử dụng.
|
||||
|
||||
|
||||
:::warning Lưu ý
|
||||
|
||||
Engine này không hỗ trợ truy cập riêng tư. Sau khi file được upload, NocoBase tạo URL có thể truy cập trực tiếp, và bất kỳ ai có URL đó đều có thể truy cập file.
|
||||
|
||||
Ngay cả khi bucket S3 được cấu hình riêng tư, engine Amazon S3 tích hợp sẵn cũng không tạo URL ký tạm thời để truy cập file. Nếu cần truy cập riêng tư, hãy dùng [S3 Pro](./s3-pro). Nếu đã có file lịch sử, hãy xem [Di chuyển sang S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
:::
|
||||
|
||||
## Tham số cấu hình
|
||||
|
||||
|
||||
|
||||
@@ -25,6 +25,29 @@ Hiện tại NocoBase tích hợp sẵn các loại engine sau:
|
||||
|
||||
Khi cài đặt hệ thống, một Local Storage engine sẽ được tự động thêm và có thể sử dụng ngay. Bạn cũng có thể thêm engine mới hoặc chỉnh sửa các tham số của engine đã có.
|
||||
|
||||
|
||||
Nếu bạn đang dùng storage engine chỉ hỗ trợ truy cập công khai và muốn di chuyển file lịch sử sang S3 Pro, hãy xem [Di chuyển sang S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
## Khả năng truy cập file
|
||||
|
||||
Mỗi storage engine hỗ trợ kiểm soát truy cập file khác nhau. Trước khi cấu hình, hãy xác nhận file có cần truy cập riêng tư hay không:
|
||||
|
||||
| Storage engine | Khả năng truy cập file |
|
||||
| --- | --- |
|
||||
| [Local Storage](./local) | Chỉ hỗ trợ truy cập công khai; không hỗ trợ truy cập riêng tư |
|
||||
| [Amazon S3](./amazon-s3) | Chỉ hỗ trợ truy cập công khai; không hỗ trợ truy cập riêng tư |
|
||||
| [Aliyun OSS](./aliyun-oss) | Chỉ hỗ trợ truy cập công khai; không hỗ trợ truy cập riêng tư |
|
||||
| [Tencent COS](./tencent-cos) | Chỉ hỗ trợ truy cập công khai; không hỗ trợ truy cập riêng tư |
|
||||
| [S3 Pro](./s3-pro) | Hỗ trợ truy cập riêng tư qua URL ký tạm thời |
|
||||
|
||||
:::warning Lưu ý
|
||||
|
||||
Local Storage, Amazon S3, Aliyun OSS và Tencent COS không kiểm tra đăng nhập khi truy cập file và không tạo URL ký tạm thời. Sau khi file được upload, bất kỳ ai có URL truy cập file đều có thể truy cập trực tiếp.
|
||||
|
||||
Nếu cần lưu hợp đồng, giấy tờ định danh, tài liệu nội bộ hoặc các file không nên công khai, hãy dùng [S3 Pro](./s3-pro) và bật truy cập riêng tư.
|
||||
|
||||
:::
|
||||
|
||||
## Tham số chung
|
||||
|
||||
Ngoài các tham số riêng của từng loại engine, các phần dưới đây là tham số chung (lấy Local Storage làm ví dụ):
|
||||
|
||||
@@ -9,6 +9,15 @@ keywords: "Local Storage,ổ cứng server,đường dẫn lưu trữ,lưu trữ
|
||||
|
||||
File được upload sẽ được lưu trong thư mục ổ cứng cục bộ của server, phù hợp với các tình huống tổng lượng file upload mà hệ thống quản lý ít hoặc thử nghiệm.
|
||||
|
||||
|
||||
:::warning Lưu ý
|
||||
|
||||
Local Storage không hỗ trợ truy cập riêng tư. Sau khi file được upload, NocoBase tạo URL có thể truy cập trực tiếp, và bất kỳ ai có URL đó đều có thể truy cập file.
|
||||
|
||||
Nếu cần lưu hợp đồng, giấy tờ định danh, tài liệu nội bộ hoặc các file không nên công khai, hãy dùng [S3 Pro](./s3-pro). Nếu đã có file lịch sử, hãy xem [Di chuyển sang S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
:::
|
||||
|
||||
## Tham số cấu hình
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,270 @@
|
||||
---
|
||||
pkg: '@nocobase/plugin-file-storage-s3-pro'
|
||||
title: "Di chuyển sang S3 Pro"
|
||||
description: "Migrate files from public-only storage engines to S3 Pro, including maintenance windows, backups, object key mapping, record updates, and risk checks."
|
||||
keywords: "S3 Pro,storage migration,private access,file migration,Local Storage,S3,OSS,COS,NocoBase"
|
||||
---
|
||||
|
||||
# Di chuyển sang S3 Pro
|
||||
|
||||
If you already use [Local Storage](./local.md), [Amazon S3](./amazon-s3.md), [Aliyun OSS](./aliyun-oss.md), or [Tencent COS](./tencent-cos.md), and later need private file access, you can migrate historical files to [S3 Pro](./s3-pro.md).
|
||||
|
||||
This is not only a storage configuration change. A migration must handle three things: make the physical files or objects available from the new storage, update file records, and switch future uploads to the new storage.
|
||||
|
||||
:::warning Lưu ý
|
||||
|
||||
Run the migration once in a test environment first, and prepare restorable database and file backups. If users continue to upload or delete attachments during the migration, records can be missed, overwritten, or become inconsistent.
|
||||
|
||||
:::
|
||||
|
||||
## Scope
|
||||
|
||||
This guide applies when migrating from these storage engines to S3 Pro:
|
||||
|
||||
| Source storage engine | Historical file location |
|
||||
| --- | --- |
|
||||
| Local Storage | `<documentRoot>/<record.path>/<record.filename>` |
|
||||
| Amazon S3 | `join(record.path, record.filename)` in the source bucket |
|
||||
| Aliyun OSS | `join(record.path, record.filename)` in the source bucket |
|
||||
| Tencent COS | `join(record.path, record.filename)` in the source bucket |
|
||||
|
||||
Here, `record.path` and `record.filename` come from the file record itself, including the built-in `attachments` table and other file collections.
|
||||
|
||||
:::tip Lưu ý
|
||||
|
||||
For Local Storage, `documentRoot` must come from `options.documentRoot` in the corresponding `storages` record. The default value is `storage/uploads`, but the final absolute path depends on the runtime storage directory of NocoBase.
|
||||
|
||||
:::
|
||||
|
||||
## Operation Steps
|
||||
|
||||
### Step 1: Stop writes or enter a maintenance window
|
||||
|
||||
Pause user uploads, updates, and deletions of attachments during the migration. You can use a maintenance window, temporarily disable entry points, or freeze related business flows.
|
||||
|
||||
The goal is to keep file records and physical files stable. If users upload or delete files during migration, the migration script may no longer be working against the latest records.
|
||||
|
||||
### Step 2: Back up the database and files
|
||||
|
||||
Prepare at least two backups:
|
||||
|
||||
1. Database backup
|
||||
2. File backup or object snapshot from the source storage
|
||||
|
||||
For Local Storage, back up the directory pointed to by `documentRoot`. Historical files are usually stored at:
|
||||
|
||||
```text
|
||||
<documentRoot>/<record.path>/<record.filename>
|
||||
```
|
||||
|
||||
For Amazon S3, Aliyun OSS, and Tencent COS, confirm that objects in the source bucket are still readable, and record the source storage engine's `id`, `name`, `type`, `path`, `baseUrl`, and `options`.
|
||||
|
||||
It is also recommended to export a migration manifest for rollback and manual review:
|
||||
|
||||
```text
|
||||
collection
|
||||
id
|
||||
oldStorageId
|
||||
oldPath
|
||||
oldFilename
|
||||
oldUrl
|
||||
newKey
|
||||
size
|
||||
mimetype
|
||||
```
|
||||
|
||||
### Step 3: Create and verify the S3 Pro storage
|
||||
|
||||
Create a new `s3-compatible` storage according to [S3 Pro](./s3-pro.md). At least verify these settings:
|
||||
|
||||
- bucket
|
||||
- endpoint
|
||||
- region
|
||||
- accessKey
|
||||
- secret
|
||||
- public / private
|
||||
- access endpoint
|
||||
- forcePathStyle
|
||||
- file size and MIME type rules
|
||||
|
||||
Upload a test file with the new storage and confirm upload, preview, download, and deletion all work. If the target is private access, also confirm that the access URL is a temporary signed URL and that its expiration is expected.
|
||||
|
||||
:::warning Lưu ý
|
||||
|
||||
S3 Pro uses direct client upload. The target bucket must have CORS rules that allow uploads from the NocoBase site, otherwise new uploads will fail.
|
||||
|
||||
:::
|
||||
|
||||
### Step 4: Decide the object key mapping
|
||||
|
||||
For historical files migrated to S3 Pro, it is recommended to continue using the old relative path as the S3 object key:
|
||||
|
||||
```text
|
||||
oldKey = join(record.path, record.filename)
|
||||
```
|
||||
|
||||
For example:
|
||||
|
||||
```text
|
||||
record.path = "avatars"
|
||||
record.filename = "a-123.png"
|
||||
oldKey = "avatars/a-123.png"
|
||||
```
|
||||
|
||||
For persisted file records, S3 Pro uses `file.filename` directly as the full object key when accessing the file. It does not append `file.path`. Therefore, migrated records should be updated to:
|
||||
|
||||
```text
|
||||
storageId = <s3-pro-storage-id>
|
||||
filename = <oldKey>
|
||||
path = ""
|
||||
url = ""
|
||||
```
|
||||
|
||||
Do not migrate to this shape:
|
||||
|
||||
```text
|
||||
filename = "a-123.png"
|
||||
path = "avatars"
|
||||
```
|
||||
|
||||
Otherwise S3 Pro may treat only `a-123.png` as the object key, causing historical files to fail to load.
|
||||
|
||||
Use `/` as the separator when generating object keys. Do not use operating-system-specific path separators. If an old path starts with `/`, remove the leading slash.
|
||||
|
||||
### Step 5: Move physical files or confirm object location
|
||||
|
||||
Traverse all file records, including the built-in `attachments` table and other file collections. Only process records where `storageId = <old-storage-id>`.
|
||||
|
||||
If the source is Local Storage, upload the local files to the bucket used by S3 Pro. If the source is already Amazon S3, Aliyun OSS, or Tencent COS, and the new S3 Pro storage points to the same bucket and endpoint, with credentials that can read the same objects, you usually do not need to copy objects. In that case, confirm that the `oldKey` generated in Step 4 is accessible through S3 Pro.
|
||||
|
||||
| Source storage engine | Whether files need to be copied | Target object key |
|
||||
| --- | --- | --- |
|
||||
| Local Storage | Must be uploaded to the S3 Pro bucket | `join(record.path, record.filename)` |
|
||||
| Amazon S3 | Usually no copy if reusing the same bucket; copy is needed when changing bucket, account, or region | `join(record.path, record.filename)` |
|
||||
| Aliyun OSS | Usually no copy if reusing the same bucket; copy is needed when changing bucket, account, or region | `join(record.path, record.filename)` |
|
||||
| Tencent COS | Usually no copy if reusing the same bucket; copy is needed when changing bucket, account, or region | `join(record.path, record.filename)` |
|
||||
|
||||
Files or objects still need to be copied in these cases:
|
||||
|
||||
- Migrating from Local Storage to S3 Pro
|
||||
- Changing bucket, account, region, or cloud provider
|
||||
- Moving historical objects from a public bucket to a new private bucket
|
||||
- Renaming object keys or reorganizing directories
|
||||
- The source bucket policy is not suitable for S3 Pro signed access or direct client upload
|
||||
- S3 Pro cannot directly access the source objects with the current endpoint and credentials
|
||||
|
||||
Before the real migration, run a dry-run. At least output:
|
||||
|
||||
- Number of records to migrate
|
||||
- Total size to migrate
|
||||
- Number of missing local files or missing source objects
|
||||
- Number of duplicate object keys
|
||||
- Number of records with unrecognized `storageId`
|
||||
- List requiring manual handling
|
||||
|
||||
If duplicate object keys appear, do not overwrite directly. Compare file size, ETag, or hash to confirm whether they point to the same file. If they are different files, generate a new key for one record and use that new key when updating the record.
|
||||
|
||||
### Step 6: Verify target object accessibility
|
||||
|
||||
After files are copied, or after confirming that the source cloud bucket can be reused, run S3 HEAD Object or an equivalent check for every migrated record. Confirm that S3 Pro can access the object through the target object key.
|
||||
|
||||
Recommended output:
|
||||
|
||||
- Success count
|
||||
- Missing source file count
|
||||
- Upload or copy failure count
|
||||
- Missing target object count
|
||||
- Duplicate key count
|
||||
- List requiring manual handling
|
||||
|
||||
Do not rely only on the script exit code. Object storage can have partial failures, successful retries, same-key overwrites, or write-only-but-not-readable permission problems. The migration manifest and HEAD Object checks are more reliable.
|
||||
|
||||
### Step 7: Update file records
|
||||
|
||||
After confirming that all target objects exist, update the file records. Only update records in file collections, keep the original `id`, and do not update many-to-many junction tables generated by attachment fields.
|
||||
|
||||
Core fields:
|
||||
|
||||
| Field | Value after migration |
|
||||
| --- | --- |
|
||||
| `storageId` | New S3 Pro storage ID |
|
||||
| `filename` | `join(oldPath, oldFilename)` |
|
||||
| `path` | Empty string |
|
||||
| `url` | Empty string for private storage; public URL or empty string for public storage |
|
||||
|
||||
If S3 Pro is private, `url` must be empty. S3 Pro also clears `url` before saving non-public storage records. File access will dynamically generate a temporary signed URL.
|
||||
|
||||
:::warning Lưu ý
|
||||
|
||||
Batch update records in a transaction if possible, and keep the migration manifest from Step 2 until after the migration is fully verified. Rollback needs that manifest to restore `storageId`, `path`, `filename`, and `url` to their old values.
|
||||
|
||||
:::
|
||||
|
||||
### Step 8: Switch future uploads to the new storage
|
||||
|
||||
After historical records are updated, switch the target storage for new uploads.
|
||||
|
||||
Check three types of configuration:
|
||||
|
||||
1. Default storage engine: if attachment fields or file collections do not specify storage, uploaded files use the default storage. Set the new S3 Pro storage as the default.
|
||||
2. Attachment fields: if an attachment field is configured with `options.storage = <old-storage-name>`, change it to the new S3 Pro storage `name`.
|
||||
3. File collections: if a file collection is configured with the old storage engine, change it to the new S3 Pro storage `name`.
|
||||
|
||||
Here, `name` is the storage engine name, not the storage ID. Historical file records use `storageId`.
|
||||
|
||||
### Step 9: Restart or refresh storage cache
|
||||
|
||||
The file manager caches `storages` configuration. If you update storage configuration through the NocoBase UI or resource API, `reloadStorages` is usually triggered. If you update the database directly, restart the application after migration or make sure the storage cache has been refreshed.
|
||||
|
||||
If you directly changed attachment field metadata or file collection metadata, it is also recommended to restart the application so collection configuration, field configuration, and frontend cache all use the new storage.
|
||||
|
||||
### Step 10: Sample-check and keep old files
|
||||
|
||||
After migration, verify at least these scenarios:
|
||||
|
||||
- View, preview, and download historical files in attachment fields
|
||||
- View, preview, and download historical files in file collections
|
||||
- Image thumbnails or preprocessing parameters
|
||||
- Office preview and other features that depend on external services reading file URLs
|
||||
- Save, preview, download, and delete newly uploaded files
|
||||
- Object deletion after deleting file records
|
||||
|
||||
Do not delete files from the old storage immediately. Keep them until the business side confirms the migration, backup cycles have covered the new state, and access logs show no abnormal behavior.
|
||||
|
||||
## Risks
|
||||
|
||||
### Data inconsistency during writes
|
||||
|
||||
If users can still upload or delete files during migration, objects may be copied after records are deleted, records may be updated while objects remain only in the old storage, or new uploads may not be included in the migration manifest. The default approach is to enter a maintenance window.
|
||||
|
||||
### Different object key rules in S3 Pro
|
||||
|
||||
After migration, `filename` should store the full object key and `path` should be empty. This differs from the historical record shape of Local Storage and built-in S3 / OSS / COS, and is the easiest place to make mistakes.
|
||||
|
||||
### URL accessibility changes
|
||||
|
||||
After migrating from public storage to private S3 Pro, historical public URLs may no longer remain valid. NocoBase dynamically generates temporary signed URLs for internal file access, but old URLs already saved in external systems are not automatically rewritten.
|
||||
|
||||
If third-party systems, email templates, exported files, or rich text content directly stores old URLs, evaluate a separate replacement strategy.
|
||||
|
||||
:::warning Lưu ý
|
||||
|
||||
If Markdown (Vditor) field content already stores file URLs, handle those contents separately. Private S3 Pro access generates temporary signed URLs that expire, and currently these private links are not supported for long-term storage and use inside Markdown (Vditor) fields.
|
||||
|
||||
If these fields must continue referencing files, keep publicly accessible file URLs for now.
|
||||
|
||||
:::
|
||||
|
||||
### Preview services and private files
|
||||
|
||||
Some preview features depend on external services accessing file URLs. Private S3 Pro generates temporary signed URLs, which can usually be accessed, but they are affected by signature expiration, network reachability, bucket permissions, and server-side caching.
|
||||
|
||||
If files are very sensitive, reassess whether external preview services should be allowed to read them.
|
||||
|
||||
### Direct database changes do not trigger all hooks
|
||||
|
||||
If you update `storages`, collection configuration, field configuration, and file records directly through SQL or a custom script, some NocoBase cache refreshes and save hooks are not triggered automatically. Restart the application after the update and verify storage configuration again.
|
||||
|
||||
### Delete behavior points to the new storage
|
||||
|
||||
After a file record's `storageId` is updated to S3 Pro, deleting that file record later will make NocoBase try to delete the object from the new bucket. Objects in the old storage will not be deleted automatically and need to be cleaned up separately.
|
||||
@@ -9,6 +9,15 @@ keywords: "Tencent COS,Tencent Cloud,SecretId,SecretKey,Bucket,Object Storage,No
|
||||
|
||||
Storage engine dựa trên Tencent COS, cần chuẩn bị tài khoản và quyền liên quan trước khi sử dụng.
|
||||
|
||||
|
||||
:::warning Lưu ý
|
||||
|
||||
Engine này không hỗ trợ truy cập riêng tư. Sau khi file được upload, NocoBase tạo URL có thể truy cập trực tiếp, và bất kỳ ai có URL đó đều có thể truy cập file.
|
||||
|
||||
Ngay cả khi bucket COS được cấu hình riêng tư, engine Tencent COS tích hợp sẵn cũng không tạo URL ký tạm thời để truy cập file. Nếu cần truy cập riêng tư, hãy dùng [S3 Pro](./s3-pro). Nếu đã có file lịch sử, hãy xem [Di chuyển sang S3 Pro](./migrate-to-s3-pro.md).
|
||||
|
||||
:::
|
||||
|
||||
## Tham số cấu hình
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user