3d Face Reconstructor

--- language: en license: apache-2.0 library_name: pytorch tags: - computer-vision - 3d-face-reconstruction - face-mesh - depth-estimation - ml-intern --- # 3D Face Reconstructor A lightweight CNN model that reconstructs a 3D face mesh from a single 2D image. ## Model Overview | Property | Value | |----------|-------| | **Architecture** | ResNet-like CNN encoder + FC regressor | | **Input** | RGB face image (128×128) | | **Output** | 512 3D vertex coordinates (x, y, z) | | **Parameters** | ~11.3M | | **Training data** | Synthetic 3DMM-like parametric face meshes | ## Approach This model was trained on synthetically generated face meshes using a parametric 3D face model with: - **40 identity basis vectors** — capturing face shape variations across individuals - **30 expression basis vectors** — capturing facial expression variations - Random rotations (±0.35 radians) for pose robustness - Orthographic projection to 2D depth images - Gaussian-noise-augmented RGB rendering The CNN learns to invert the rendering process, recovering the full 3D mesh structure from a single 2D image. ## Training Details | Hyperparameter | Value | |----------------|-------| | Optimizer | Adam | | Learning Rate | 1e-3 with cosine warm restarts | | Batch Size | 32 | | Loss | MSE (vertex-wise) | | Epochs | 20 | | Best Val Loss | 0.0218 | ## Usage ```python import torch from PIL import Image import numpy as np # Load model ckpt = torch.load("model.pt", map_location="cpu") n_vertices = ckpt["mean_face"].shape[0] from model import Face3DNet model = Face3DNet(num_vertices=n_vertices) model.load_state_dict(ckpt["model_state_dict"]) model.eval() # Load and preprocess image img = Image.open("face.jpg").convert("RGB").resize((128, 128)) arr = np.array(img).astype(np.float32) / 255.0 tensor = torch.from_numpy(arr).permute(2, 0, 1).unsqueeze(0) # Predict 3D mesh with torch.no_grad(): vertices = model(tensor)[0] # [512, 3] ``` ## Live Demo Try the interactive demo: [https://huggingface.co/spaces/apapaxionga/3d-face-reconstructor-demo](https://huggingface.co/spaces/apapaxionga/3d-face-reconstructor-demo) ## Limitations - Trained on synthetic data — real-world generalization is not guaranteed - Coarse parametric mesh, not a high-resolution detailed scan - Best for roughly frontal or mildly rotated faces - No texture/color reconstruction ## Citation ```bibtex @software{3d_f...

Source context: 11 downloads · 2 likes · Pipeline depth-estimation · Library pytorch · Repo apapaxionga/3d-face-reconstructor

Source context: 6 downloads · 2 likes · Pipeline depth-estimation · Library pytorch · Repo apapaxionga/3d-face-reconstructor