We often talk about RAG (Retrieval-Augmented Generation) as the real way to supercharge LLMs today, feeding the models in order to retrieve and incorporate new information.

This task is now possible also thanks to vector databases, that store all the data we want to use as embeddings (structured data vectors), so even if we have tons of PDFs, DOCs or XLS files we can convert all of them into basically numbers, store them in these databases and use them to feed things like a first level support chatbot, in which we can search our data before returning the answer to the user.

We can also use the embeddings by cosine similarity, so we can retrieve i.e. the manuals related to a specific product without calling an LLM at all.

These embeddings are produced by feeding the text to particular models, like the famous text-embedding-3-small of OpenAI, that are encoder only models: so they take words and create embeddings that we can store in our databases. There are a lot of similar models produced by third party that are cheap and lighter even to run on a CPU.

That’s cool but… what about images?

Well, in recent artificial intelligence development we can see an evolution of LLM models that, if we look at the usual suspects like Gemini, Anthropic and OpenAI, can be multimodal, so they can process images too by transforming them through a visual transformer and interacting directly with the LLM.
Vector databases now are mature to store image embeddings too, and so there are visual embeddings models, let’s introduce two of them: SigLIP 2 and DINOv2.
Both of them are suitable to “translate” images in embeddings, but their purposes are way different one from each other.

SigLIP 2

This is the SOTA of models that belongs to the CLIP family, created by OpenAI in ‘21.
CLIP is the acronym of Contrastative Language-Image Pretraining, that is the way we can use to understand the image feature in a zero-shot classification, so without a proper training: compared against the usual CNN networks that are trained to learn features with annotated data, these models uses a visual transformer (ViT) to connect text and images in latent space, so we can compare text description with an image via vector similarity, often cosine.

SigLIP 2 it’s the recent evolution of CLIP architecture, brought to the world by Google, that not only support multilanguage text embedding but also has a different loss function sigmoid loss function, different from the usual softmax used in previous iteration, that treats each image-text pair as an independent binary classification, making the matching more scalable and robust.

DINOv2

This is the second iteration of DINO, created by Facebook AI (now Meta) with DINO introduced in 2021 and DINOv2 in 2023.
The name comes from the contraction of Self-DIstillation with NO labels and works similar to a GAN, we saw in StyleGAN2 article here, where there are two ViT called teacher and a student, that this time collaborate together instead on fighting against each other.
They both can have two augmented view of the same image, the student should predict the output of the teacher on the view, with a cross entropy loss. This last function is effective and is used mainly on classification tasks.

The teacher isn’t trained directly, but it slowly absorbs the student’s knowledge over time, acting as a more stable version of it. This keeps the training signal consistent and prevents both networks from collapsing into the same output.

This demonstrates that self-supervised ViT, not only can be effective, but also produces a lot of rich features.

Let’s start the battle

These two embeddings models practically do the same thing: transform the images (and the text) in vectors, to be stored and searched to find similarities with cosine or dot product.
So there are no differences between one or another! Well, not so fast…

SigLIP 2 works by vector similarity connecting text and images in the same latent space, so it perform a semantic search: like with a “usual” LLM, this embeddings can be used to find images that match a text description, or comparing images by their overall meaning and context.
DINOv2 instead is a purely visual model: it has no notion of text, and it was trained in a self-supervised way to produce embeddings that are highly sensitive to visual structure, texture, shapes and spatial composition. This makes it great at finding images that look similar, regardless of what they actually represent semantically.

What does it means? Let’s make an example.

Assume that we have a dog photo in a garden chasing a butterfly, with plants, flowers and all the usual background elements that we can find in a backyard, and we want to search our database in order to find similar images.

SigLIP 2 will search this images and try to return some semantically similar images. That means that can return not only dog images in garden, but also garden images with flowers and dogs that are similar to the one in photo where it chasing the butterfly, but maybe running to catch a ball, or sleeping under a tree. It works well when the embedding database is full of similar images and it can works smoothly also using text search, because of the shared space between words and images. Ok what is the weak point? If we need images of a dog chasing a butterfly maybe won’t work: thats a geometric issue, not a semantic issue.

That is where DINOv2 can get best results: because its embeddings capture the visual structure of the scene, it will retrieve images where the overall composition matches (i.e. the pose of the dog, the spatial relationship between the elements, the texture of the grass and flowers) this makes it ideal when you need to find images that are visually similar to a reference, even without any text query.
Ok that’s perfect! Again, not so fast…
Because of his nature, DINOv2 can also retrieve cats chasing a butterfly: exactly because it doesn’t understand semantics, it might also return a cat chasing a butterfly, or even a child running after a ball as long as the visual structure of the scene is similar enough.

And the winner is…

So, in the end, who won the battle? Well, there isn’t an absolute winner: it depends on the result you want to achieve.

If your aim is to use a database and perform a natural language query, looking for a specific product that has images and attributes, your choice should be SigLIP 2.

If you want to retrieve images by similarity and you want to rely on images only, DINOv2 is better.

And what if I want to use images to retrieve attributes instead? Well… use both! If you use a vector database, no one is stopping you from creating a multi-format embedding that handles both SigLIP 2 and DINOv2 embeddings. Imagine if you want to label the attributes of a car, such as the type (hatchback, van, coupe), number of doors (3 or 5), presence of alloy wheels, etc.

The first query can be done with DINOv2, so we can isolate the geometry of the car and the shapes of the doors, wheels, etc.

On those results, we can perform a SigLIP 2 search, retrieving parameters by similarity.

Cover image: Midjourney
Model graph: Claude Sonnet 4.6
Text: Me 🙂 (grammar corrections made with Gemini 3.1 Pro)