Dvd

--- license: mit library_name: pytorch tags: - 3d-generation - voxel-generation - 3d-editing - diffusion - image-to-3d - text-to-3d - trellis pipeline_tag: image-to-3d --- # DVD: Discrete Voxel Diffusion for 3D Generation and Editing This repository hosts the pretrained DVD checkpoints for **Discrete Voxel Diffusion for 3D Generation and Editing**. DVD is a discrete diffusion pipeline that generates or edits a pure `64^3` voxel grid. The voxel output can be used directly, exported as a cubified voxel mesh, or passed as the sparse-structure anchor for [TRELLIS](https://github.com/microsoft/TRELLIS) stage 2 to produce final mesh and 3D Gaussian assets. - Paper: [arXiv:2605.07971](https://arxiv.org/abs/2605.07971) - Spaces: [DVD Image](https://huggingface.co/spaces/Zhengrui/dvd-image), [DVD Text](https://huggingface.co/spaces/Zhengrui/dvd-text) ## Checkpoints | Variant | Files | Use case | | --- | --- | --- | | Image generation | `dvd_img.json`, `dvd_img.safetensors` | image-conditioned voxel generation | | Image editing | `dvd_img_BSP_ft.json`, `dvd_img_BSP_ft.safetensors` | image-conditioned voxel editing | | Text generation | `dvd_text.json`, `dvd_text.safetensors` | text-conditioned voxel generation | | Text editing | `dvd_text_BSP_ft.json`, `dvd_text_BSP_ft.safetensors` | text-conditioned voxel editing | ## Minimal Usage ```python from dvd import DVDImageToVoxelPipeline, DVDTextToVoxelPipeline repo_id = "Zhengrui/dvd" image_dvd = DVDImageToVoxelPipeline.from_pretrained(repo_id, variant="base", device="cuda") image_edit_dvd = DVDImageToVoxelPipeline.from_pretrained(repo_id, variant="bsp", device="cuda") text_dvd = DVDTextToVoxelPipeline.from_pretrained(repo_id, variant="base", device="cuda") text_edit_dvd = DVDTextToVoxelPipeline.from_pretrained(repo_id, variant="bsp", device="cuda") ``` Example image-conditioned generation: ```python from PIL import Image from dvd import DVDImageToVoxelPipeline, export_cubified_voxels image = Image.open("assets/example_image/T.png") dvd = DVDImageToVoxelPipeline.from_pretrained("Zhengrui/dvd", variant="base", device="cuda") voxels = dvd.sample_voxels(image, seed=42, steps=256) export_cubified_voxels(voxels, "dvd_voxels.glb") ``` ## Voxel Convention DVD stores voxel outputs as: ```text samples: [B, D, H, W] coords: [N, 4] as [batch, x, y, z] ``` Saved `.npy` coordinate files usually omit the batc...

Source context: 0 downloads · 1 likes · Pipeline image-to-3d · Library pytorch · Repo Zhengrui/dvd