How much time did you spend parsing PDFs in your last RAG project? How many Python libraries did you juggle to scrape tables from annual reports or separate headers from footers? And how often was the chatbot's answer ultimately wrong because it simply couldn't “see” a diagram?
Let's talk about the elephant in the room – or rather, the elephant in the PDF that our AI system mistakes for a line break.
Retrieval-Augmented Generation (RAG) has arrived in the enterprise environment. It is the standard for feeding LLMs with company knowledge. And how does this data get into our system? The common technique for this could date back to the 1990s: text extraction.
We take highly visual documents – PowerPoint presentations, technical drawings, financial reports with complex charts – and put them through the meat grinder we call OCR (Optical Character Recognition). The result is often a jumble of data. We lose layout, context and visual semantics. We make our RAG system blind to anything that isn't an ASCII character.
With the release of new retrieval models, most recently Qwen3-VL-Embedding and the matching reranker, this is currently changing fundamentally. We are moving away from “text-based RAG” towards true “multimodal RAG” (or visual retrieval).
Today, we'll take a look at why this isn't just a technical gimmick, but rather something that changes the way we build knowledge management architectures. And because we like “more engineering and less magic” here, I've built a demo app that you can try out right away (a current Mac or Nvidia GPU is recommended).
The problem with the 'Blind RAG'
Let's take a look at the status quo. A typical ingestion process today looks like this:
- A PDF file arrives (e.g. instructions for an industrial machine).
- We use tools such as “PyPDF2”, “Unstructured” or “Azure Document Intelligence”.
- We try to extract the text.
- We despair at two-column layouts (does the tool now read line by line across both columns or column by column?).
- Images? Diagrams? Warning notices in red boxes? These are often ignored or converted into a caption by another model.
- Tables? Often require special treatment, and even commercial tools regularly struggle to recognise column and row headers or cause rows to slip out of place.
- This (error-prone) text is converted into vectors.
The result: a huge loss of information.
Company knowledge is rarely contained in plain text. It is contained in the relationship between a table row and the column heading. It is contained in the screenshot that shows exactly where to click in the menu. It is contained in the colour of a warning symbol.
If a RAG system only reads text, it has a hard performance ceiling. Companies can use the best LLM in the world for generating answers – but if the context they submit consists only of chopped-up text fragments and misplaced lines, the answer will be rubbish. Garbage In, Garbage out. It looks like a hallucination, but it's the fault of the ETL pipeline.
The paradigm shift: Visual Retrieval
The idea behind Multimodal RAG is simple but radical: we stop forcing documents into text.
Instead, we treat a page in a document (or a screenshot) for what it is: an image.
The new paradigm looks like this:
- Ingestion: We render the PDF page as an image.
- Embedding: A vision-language model (VLM) projects this image directly into a vector space.
- Retrieval: When the user asks, ‘Where can I see the sales trend?’, the system does not search for the word ‘sales trend’, but for an image that semantically matches this question (e.g. a bar chart).
That sounds logical, but until recently it was hardly possible in practice. The models were proprietary, too expensive, not multilingual, too slow, or unable to map text and images well enough into the same mathematical space. Approaches such as ColPali were promising, but often architecturally complex (keyword: multi-vector retrieval), requiring special database setups.
This is where Qwen3 comes in.
Why Qwen3-VL is a game changer (and heralds the future of RAG)
With the release of Qwen3-VL-Embedding and Qwen3-VL-Reranker (Apache 2.0 licence!), the Qwen team (Alibaba Cloud) has delivered two building blocks that we needed to make the whole thing production-ready.
1. The Embedding Model
Normally, we have text embedding models (such as “text-embedding-3-large”) and image models (such as “CLIP”). They speak different languages. Qwen3-VL embedding maps both into the same vector space.
We can ask a pure text question (‘What was the latency?’) and the vector ends up in the mathematical space right next to the image of the latency chart. The system “understands” what is in the image without us ever having to run OCR.
2. The Reranker
That's the real star. Vector search (ANN) is fuzzy. We often get 50 results back, 40 of which are irrelevant. A reranker reorders these results by carefully checking the query and document in pairs.
The Qwen3-VL reranker is a cross-encoder that understands images and text. It looks at our query and the image it finds and returns a score indicating how relevant the image really is. This massively increases precision.
Both models are small enough (2B to 8B parameters) to be hosted locally or cost-effectively in the cloud. Data protection? Check!
Implications: more than just 'better search'
What does this mean for businesses?
Firstly, the ETL pipeline is shrinking.
We no longer need complex parsing rules for every new document format. PDF? Screenshot? PowerPoint? Simply render it as an image, embed it, and you're done. This drastically reduces the amount of custom code that developers have to maintain. It's a shift from “data engineering heavy” to “model heavy”.
Secondly: We tap into “dark data”.
Many companies have terabytes of scanned contracts or technical drawings that were previously unusable for RAG because OCR failed due to poor scan quality or handwriting. A VLM is much more robust against noise than classic OCR engines.
Thirdly: new query types.
Users can now upload a photo of a defective component and ask, ‘What is this and how do I repair it?’ Since images and text are stored in the same vector space, image-to-document retrieval works “out of the box”.
Of course, there are also trade-offs:
- Storage & Compute: Storing and processing images requires more space and GPU power than pure text.
- Latency: Ingestion takes longer (inference on a VLM is more expensive than a “split()” string operation).
- Hallucinations: VLMs are powerful, but can still be prone to errors with very fine visual details (e.g. “does it say 1.25 or 1.26?”) if the resolution is not correct.
Hands-on: the showcase
Enough theory. I wanted to know if it really works as well as claimed, so I built a small demo.
The goal: to index a longer PDF and answer questions that cannot be answered without visual understanding.
The architecture:
- Ingest: A PDF is converted page by page into images (75 DPI). Text remains text.
- Embed: Each image and text chunk is converted into a vector using “Qwen3-VL-Embedding” and stored in a vector database (here: in-memory, disk-backed).
- Query: The user asks a text question and/or uploads an image.
- Retrieve: We search for the top k most similar images and text chunks.
- Rerank: The “Qwen3-VL-Reranker” sorts the hits.
- Synthesize: A generative VLM (e.g. “Qwen3-VL-2B-Instruc” or “GPT-5-mini”) receives the question and the most relevant images and writes the answer.
Example:
We use OpenAI's ‘The State of Enterprise AI’ report as an example, a highly visual 24-page PDF document with numerous graphics, boxes and some multi-column text. In addition, there is my CamundaCon 2023 slide deck with 38 slides. The PDFs can be uploaded under ‘Ingest Files’ and are indexed immediately. We can also upload an insignificant text document to test textual retrieval.
I ask: ‘How is the growth of business customers of ChatGPT in Germany?’
A text RAG would possibly fail here, as only the global average is mentioned in the text; the figures for the individual countries are only shown in a bar chart. The multimodal system finds the page because it recognises the structure of a chart and the context of the figures as “paying business customers”.
Or: ‘Which page has the pink, violet, and blue pixel art that looks a bit like a sunset on the horizon?’
Here, I am describing an artistic graphic purely visually. The system delivers exactly the right page. This example shows the relevance of the reranker: embeddings have to squeeze all the information on a page into a limited vector in advance, without knowing which information will actually be needed later in queries. Due to terms such as ‘sunset’ and ‘horizon’ in the query, the slide with the photo of a sunset even achieves a slightly higher embedding score than the slide being searched for. However, when the reranker compares each preselected image and the query side by side, it is easy to see that only one slide is truly relevant to the query, and it moves from the bottom of the ranking to the top – by a wide margin.
Last example: this time, I upload a photo of myself and search for the introductory slide in my CamundaCon deck. In the photo, I am posing completely differently and wearing glasses. In addition to the photo, I write, ‘Find the slide with a portrait of this person on it.’
Here, I had to adjust the default hyperparameters, which are set to be very resource-efficient, in order to be successful. This is because embedding alone fails again here; most slides from the deck are rated similarly (not a single page from the OpenAI report appears in the top 25, presumably because its A4 format does not fit the ‘slide’). If the reranker sees the correct slide thanks to a sufficiently large top-k (here: top-25), it is very easy for it to identify it as the correct one, as in the previous example.
The combination of embedding and reranker therefore serves its purpose! The embedding similarity search makes a rough selection, and the reranker evaluates the details and finds the needle in the haystack.
You can try it out for yourself here: Demo repo on GitHub
The repo uses “uv” for dependency management. If you have “uv” installed, simply run “uv run qwen3vl_multimodal_rag_server.py” for a pure search without response generation. The latter can be activated in various ways; see the readme file for details.
Conclusion: When is it worth switching?
Multimodal RAG is no longer a pipe dream. The tools are here, they are open source, and they are powerful.
Should companies now throw away all their text RAGs? No.
If they mainly need to search clean Markdown files, wikis, or plain text contracts, they should stick with text RAG. It is faster and cheaper.
But if their use case includes:
- technical documentation,
- financial reports with numerous charts,
- presentations (PPTX),
- or scanned old files
then visual retrieval is the way to go. It saves months of development time for fragile OCR parsers and provides users with a search experience that feels “natural”.
We are moving away from systems that read to systems that see. And that's a good thing.
Try out the demo, throw in your own PDFs and tell me what you think.
