Finegrain Box Segmenter

--- license: mit pipeline_tag: mask-generation library_name: refiners tags: - vision - image-segmentation - matting - remove background - background - background-removal - salient-object-detection - PyTorch - refiners datasets: - finegrain/finegrain-product-masks-lite --- # Release note for Finegrain Box Segmenter v0.1 ## Demo If you want to give the Finegrain Box Segmenter a try, the best way to is take a look at the [Finegrain Object Cutter Space](https://huggingface.co/spaces/finegrain/finegrain-object-cutter) we shipped on Hugging Face: it's a fun "prompt to cut out" experience that will enable you to create pixel quality and high resolution cutouts for any object in a photo, by just naming the object. <video src="https://huggingface.co/finegrain/finegrain-box-segmenter/resolve/main/object_cutter_potted_plant.mp4" autoplay loop></video> ## Motivation While building Finegrain, we needed a way to create pixel perfect and high resolution cutouts for objects in images. We looked at off-the-shelf solutions, but they simply didn't work for us: - On the one hand, traditional background removal models are great at producing HD cutouts, but unfortunately, different people will have different definitions for background and foreground in a given image - a way to prompt these models is missing. - On the other hand, new promptable approaches like SAM or SAM2 don't meet the quality bar for the use cases we are pursuing: they are generating internally a 256x256 low resolution mask - with built-in upscaling mechanisms that create artefacts and struggle with complex masks (a la Eiffel Tower). The Finegrain Box Segmenter avoids these pitfalls by training [MVANet](https://arxiv.org/abs/2404.07445) to be a box-promptable High Definition (1024x1024) object-cutout model, making no assumption on what is background and what is foreground: users are fully in control. ## License The Finegrain Box Segmenter is published under the MIT license. Have fun using it in your projects! If you want an optimized version (speed and accuracy wise), we offer an API - just [ping us](mailto:bonjour@lagon.tech)! ## Features The Finegrain Box Segmenter: - produces HD and pixel quality masks, - gives control to users via box prompting, - outputs alpha masks: you can use it as an end-to-end Matting Segmenter without any post-processing or trimap. ## Use cases You should think of th...