Nirman
camenduru

camenduru/dinov3-vitl16-pretrain-lvd1689m

These are Vision Transformer and ConvNeXt models trained following the method described in the DINOv3 paper. 12 models are provided: - 10 mo...

Model Documentation

Model Card for DINOv3



DINOv3 is a family of versatile vision foundation models that outperforms the specialized state of the art across a broad range of settings, without fine-tuning. DINOv3 produces high-quality dense features that achieve outstanding performance on various vision tasks, significantly surpassing previous self
  • and weakly-supervised foundation models.


    • Model Details



    These are Vision Transformer and ConvNeXt models trained following the method described in the DINOv3 paper. 12 models are provided:

  • 10 models pretrained on web data (LVD-1689M dataset)
  • 1 ViT-7B trained from scratch,
  • 5 ViT-S/S+/B/L/H+ models distilled from the ViT-7B,
  • 4 ConvNeXt-{T/S/B/L} models distilled from the ViT-7B,
  • 2 models pretrained on satellite data (SAT-493M dataset)
  • 1 ViT-7B trained from scratch
  • 1 ViT-L distilled from the ViT-7B


    • Each Transformer-based model takes an image as input and returns a class token, patch tokens (and register tokens). These models follow a ViT architecture, with a patch size of 16. For a 224x224 image, this results in 1 class token + 4 register tokens + 196 patch tokens = 201 tokens (for DINOv2 with registers this resulted in 1 + 4 + 256 = 261 tokens).

    The models can accept larger images provided the image shapes are multiples of the patch size (16). If this condition is not verified, the model will crop to the closest smaller multiple of the patch size.

    Model Description



  • Developed by: Meta AI
  • Model type: Vision Transformer, ConvNeXt
  • License: DINOv3 License


    • Model Sources



  • Repository: https://github.com/facebookresearch/dinov3
  • Paper: https://arxiv.org/abs/2508.10104


    • Uses



    The models are vision backbones providing multi-purpose features for downstream tasks.

    Direct Use



    The models can be used without fine-tuning, with downstream classifiers as simple as linear layers, to obtain competitive results:

  • on image classification, using k-NN classifiers on the class token
  • on image classification, with logistic regression classifiers applied on the class token
  • on image classification, with a linear layer applied on the class token and the average of the patch tokens
  • on image retrieval using nearest neighbors
  • on geometric and semantic 3D keypoint correspondances
  • on depth estimation, semantic segmentation, using linear layers
  • on unsupervised object discovery
  • on video segmentation tracking
  • on video classification, using a small 4-layer attentive probe


    • Downstream Use



    While fine-tuning the models can yield some gains, it is recommended to keep this option as a last resort: the frozen features are expected to provide good performance out-of-the-box.

    Bias, Risks, and Limitations



    Compared to DINOv2 and SEERv2, DINOv3 delivers somewhat consistent performance across income categories on geographical fairness and diversity, although with a notable performance drop in the low-income bucket compared to the highest-income bucket.

    DINOv3 also achieves relatively good scores across different regions, improving over its predecessor DINOv2. However, a relative difference is still observed between Europe and Africa.

    Recommendations



    Fine-tuning is expected to increase the biases in the features produced by the model as they will be tuned to the fine-tuning labels.

    How to Get Started with the Model



    The example below demonstrates how to obtain an image embedding with [Pipeline] or the [AutoModel] class.

    python
from transformers import pipeline
from transformers.image_utils import load_image

    url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/pipeline-cat-chonk.jpeg"
image = load_image(url)

    feature_extractor = pipeline(
    model="facebook/dinov3-vitl16-pretrain-lvd1689m",
    task="image-feature-extraction", 
)
features = feature_extractor(image)



    python
import torch
from transformers import AutoImageProcessor, AutoModel
from transformers.image_utils import load_image

    url = "http://images.cocodataset.org/val2017/000000039769.jpg"
image = load_image(url)

    pretrained_model_name = "facebook/dinov3-vitl16-pretrain-lvd1689m"
processor = AutoImageProcessor.from_pretrained(pretrained_model_name)
model = AutoModel.from_pretrained(
    pretrained_model_name, 
    device_map="auto", 
)

    inputs = processor(images=image, return_tensors="pt").to(model.device)
with torch.inference_mode():
    outputs = model(**inputs)

    pooled_output = outputs.pooler_output
print("Pooled output shape:", pooled_output.shape)


    Training Details



    Training Data



  • Web dataset (LVD-1689M): a curated dataset of 1,689 millions of images extracted from a large data
    • pool of 17 billions web images collected from public posts on Instagram

  • Satellite dataset (SAT-493M): a dataset of 493 millions of 512x512 images sampled randomly from Maxar RGB ortho-rectified imagery at 0.6 meter resolution


    • Training Procedure



    Training objective:

  • DINO self-distillation loss with multi-crop
  • iBOT masked-image modeling loss
  • KoLeo regularization on [CLS] tokens
  • Gram anchoring


  • Training regime: PyTorch FSDP2 (with bf16 and fp8 matrix multiplications)


    • Distillation:

  • Distillation follows the standard DINOv3 pretraining procedure, except the teacher is a frozen pretrained ViT-7B.


    • Evaluation



    Results

    The reader is referred to the associated paper for details on the evaluation protocols

    *Results for ViT backbones pretrained (or distilled) on web (LVD-1689M)*

    --> --> --> --> --> --> --> -->
    Global Tasks Dense Tasks
    Model Dataset IN-ReaL IN-R Obj.Net Ox.-H ADE20k NYU↓ DAVIS NAVI SPair
    DINOv3 ViT-S/16 LVD-1689M 87.0 60.4 50.9 49.5 47.0 0.403 72.7 56.3 50.4
    DINOv3 ViT-S+/16 LVD-1689M 88.0 68.8 54.6 50.0 48.8 0.399 75.5 57.1 55.2
    DINOv3 ViT-B/16 LVD-1689M 89.3 76.7 64.1 58.5 51.8 0.373 77.2 58.8 57.2
    DINOv3 ViT-L/16 LVD-1689M 90.2 88.1 74.8 63.1 54.9 0.352 79.9 62.3 61.3
    DINOv3 ViT-H+/16 LVD-1689M 90.3 90.0 78.6 64.5 54.8 0.352 79.3 63.3 56.3
    DINOv3 ViT-7B/16 LVD-1689M 90.4 91.1 91.1 72.8 55.9 0.309 79.7 64.4 58.7


    *Results for ConvNeXt backbones distilled on web (LVD-1689M)*

    Global Tasks Dense Tasks
    Model IN-ReaL IN-R Obj.Net ADE20k NYU↓
    @256px @512px @256px @512px @256px @512px
    DINOv3 ConvNeXt Tiny 86.6 87.7 73.7 74.1 52.6 58.7 42.7 0.448
    DINOv3 ConvNeXt Small 87.9 88.7 73.7 74.1 52.6 58.7 44.8 0.432
    DINOv3 ConvNeXt Base 88.5 89.2 77.2 78.2 56.2 61.3 46.3 0.420
    DINOv3 ConvNeXt Large 88.9 89.4 81.3 82.4 59.3 65.2 47.8 0.403


    *Results for ViT backbones pretrained (or distilled) on satellite (SAT-493M)*

    (GEO-Bench) Classification
    Model m-BEnet m-brick-kiln m-eurosat m-forestnet m-pv4ger m-so2sat mean
    DINOv3 ViT-L/16 73.0 96.5 94.1 60.6 96.0 57.4 79.6
    DINOv3 ViT-7B/16 74.0 97.2 94.8 62.3 96.1 62.1 81.1
    (GEO-Bench) Segmentation
    Model m-cashew m-chesapeake m-NeonTree m-nz-cattle m-pv4ger-seg m-SA-crop mean
    DINOv3 ViT-L/16 94.2 75.6 61.8 83.7 95.2 36.8 74.5
    DINOv3 ViT-7B/16 94.1 76.6 62.6 83.4 95.5 37.6 75.0


    Environmental Impact



  • Hardware Type: Nvidia H100
  • Hours used: 61,440 hours for ViT-7B model training
  • Cloud Provider: Private infrastructure
  • Compute Region: USA
  • Carbon Emitted: 18t CO2eq


    • Technical Specifications



    Model Architecture and Objective



    Vision Transformer models:

  • ViT-S (21M parameters): patch size 16, embedding dimension 384, 4 register tokens, 6 heads, MLP FFN, RoPE
  • ViT-S+ (29M parameters): patch size 16, embedding dimension 384, 4 register tokens, 6 heads, SwiGLU FFN, RoPE
  • ViT-B (86M parameters): patch size 16, embedding dimension 768, 4 register tokens, 12 heads, MLP FFN, RoPE
  • ViT-L (300M parameters): patch size 16, embedding dimension 1024, 4 register tokens, 16 heads, MLP FFN, RoPE
  • ViT-H+ (840M parameters): patch size 16, embedding dimension 1280, 4 register tokens, 20 heads, SwiGLU FFN, RoPE
  • ViT-7B (6716M parameters): patch size 16, embedding dimension 4096, 4 register tokens, 32 heads, SwiGLU FFN, RoPE


    • ConvNeXt models:

  • ConvNeXt Tiny (29M parameters)
  • ConvNeXt Small (50M parameters)
  • ConvNeXt Base (89M parameters)
  • ConvNeXt Large (198M parameters)


    • Compute Infrastructure



    #

    Hardware



    Nvidia H100 GPUs

    #

    Software



    PyTorch 2.7

    More Information



    See the blog post and the associated website.

    Citation



    BibTeX

    
@misc{simeoni2025dinov3,
  title={{DINOv3}},
  author={Sim{\'e}oni, Oriane and Vo, Huy V. and Seitzer, Maximilian and Baldassarre, Federico and Oquab, Maxime and Jose, Cijo and Khalidov, Vasil and Szafraniec, Marc and Yi, Seungeun and Ramamonjisoa, Micha{\"e}l and Massa, Francisco and Haziza, Daniel and Wehrstedt, Luca and Wang, Jianyuan and Darcet, Timoth{\'e}e and Moutakanni, Th{\'e}o and Sentana, Leonel and Roberts, Claire and Vedaldi, Andrea and Tolan, Jamie and Brandt, John and Couprie, Camille and Mairal, Julien and J{\'e}gou, Herv{\'e} and Labatut, Patrick and Bojanowski, Piotr},
  year={2025},
  eprint={2508.10104},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  url={https://arxiv.org/abs/2508.10104},
}

    Files & Weights

    FilenameSizeAction
    model.safetensors 1.13 GB