opensource/Matrix-3D
3
0
Fork 0
mirror of https://github.com/SkyworkAI/Matrix-3D.git synced 2026-08-30 18:11:35 +08:00
Code Issues Packages Projects Releases Wiki Activity

modify filenames

Browse Source
This commit is contained in:
Inn1917
2025-09-02 15:47:16 +08:00
parent ac979f5e5b
commit 814f68f2a7
3 changed files with 15 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+13 -12
View File
@@ -202,16 +202,7 @@ chmod +x install.sh
./install.sh
```
## 📊 GPU vram requirement
The minimum GPU vram requirement to run our pipeline is 16g. The specific amount of GPU vram occupation under different model settings are listed below.
| Model Name | VRAM |
| :---------: | :----------: |
| PanoVideoGen-480p w.o. vram management | ~40g |
| PanoVideoGen-720p w.o. vram management | ~60g |
| PanoVideoGen-720p-5b w.o. vram management | ~19g |
| PanoVideoGen-480p w. vram management | ~15g |
| PanoVideoGen-720p w. vram management | ~19g |
| PanoVideoGen-720p-5b w. vram management | ~12g |
## 💫 Pretrained Models
| Model Name | Description | Download |
@@ -222,6 +213,16 @@ The minimum GPU vram requirement to run our pipeline is 16g. The specific amount
|PanoVideoGen-720p-5b|lora_5b.safetensors|[Link](https://huggingface.co/Skywork/Matrix-3D)|
|PanoLRM-480p|pano_lrm_480p.pt|[Link](https://huggingface.co/Skywork/Matrix-3D)|
## 📊 GPU vram requirement
The minimum GPU vram requirement to run our pipeline is 16g. The specific amount of GPU vram occupation under different model settings are listed below.
| Model Name | vram management on/off |VRAM |
| :---------: | :----------: | :----------: |
| PanoVideoGen-480p| off | ~40g |
| PanoVideoGen-720p| off | ~60g |
| PanoVideoGen-720p-5b| off | ~19g |
| PanoVideoGen-480p| on | ~15g |
| PanoVideoGen-720p| on | ~19g |
| PanoVideoGen-720p-5b| on | ~12g |
<!-- | Model Name | Drop Location |
| :---------: | :----------: |
@@ -301,13 +302,13 @@ torchrun --nproc_per_node ${VISIBLE_GPU_NUM} code/panoramic_image_to_video.py \
--enable_vram_management # enable this to allow model to run on devices with 19G vram.
```
<span id="5b">**5b model**</span> We recently developed a 5b version of our video generation model based on Wan2.2-TI2V-5b model. The 5b model achieves fast video generation as well as lower vram usage. To run the video generation with 5b model, you can now enable 5b model usage with a command line argument setting:
<span id="5B">**5B model**</span> We recently developed a 5B version of our video generation model based on Wan2.2-TI2V-5B model. The 5B model achieves fast video generation as well as lower vram usage. To run the video generation with 5B model, you can now enable 5B model usage with a command line argument setting:
```bash
VISIBLE_GPU_NUM=1
torchrun --nproc_per_node ${VISIBLE_GPU_NUM} code/panoramic_image_to_video.py \
--inout_dir="./output/example1" \
--resolution=720 \
--use_5b_model # enable this to generate video with light-weight 5b model.
--use_5b_model # enable this to generate video with light-weight 5B model.
```
- 🏡 **Step 3: Extract 3D Scene**
code/download_checkpoints.py
+1 -1
View File
@@ -17,7 +17,7 @@ def download_ckpt(local_dir, repo_id, filename):
print(f"File exists already: {local_path}")
os.makedirs("./checkpoints", exist_ok=True)
repo_id_list = ["Ruicheng/moge-vitl","Iceclear/StableSR","Iceclear/StableSR","Skywork/Matrix-3D","Skywork/Matrix-3D","Skywork/Matrix-3D","Skywork/Matrix-3D","Skywork/Matrix-3D"]
filename_list = ["model.pt","stablesr_turbo.ckpt","vqgan_cfw_00011.ckpt","checkpoints/text2panoimage_lora.safetensors","checkpoints/pano_lrm_480p.pt","checkpoints/pano_video_gen_480p.ckpt","checkpoints/pano_video_gen_720p.bin","checkpoints/lora_5b.safetensors"]
filename_list = ["model.pt","stablesr_turbo.ckpt","vqgan_cfw_00011.ckpt","checkpoints/text2panoimage_lora.safetensors","checkpoints/pano_lrm_480p.pt","checkpoints/pano_video_gen_480p.ckpt","checkpoints/pano_video_gen_720p.bin","checkpoints/pano_video_gen_720p_5b.safetensors"]
local_dir_list = ["./checkpoints/moge","./checkpoints/StableSR","./checkpoints/StableSR","./checkpoints/flux_lora","./checkpoints/pano_lrm","./checkpoints/Wan-AI/wan_lora","./checkpoints/Wan-AI/wan_lora","./checkpoints/Wan-AI/wan_lora"]
N = len(repo_id_list)
code/panoramic_image_to_video.py
+1 -1
View File
@@ -342,7 +342,7 @@ def main(args):
],
use_usp=True if dist.get_world_size() > 1 else False
)
lora_checkpoint = os.path.abspath("./checkpoints/Wan-AI/wan-lora/lora_5b.safetensors")
lora_checkpoint = os.path.abspath("./checkpoints/Wan-AI/wan-lora/pano_video_gen_720p_5b.safetensors")
model = add_lora_to_model(
getattr(pipe, "dit"),
"q,k,v,o,ffn.0,ffn.2".split(","),