From Classical RAG to Question-Embedding Indexing

In many user-facing applications, such as documentation search or user manuals, the goal is not only to retrieve relevant text but also to provide answers that directly match the user’s intent.

The classical RAG (Retrieval-Augmented Generation) approach typically combines text indexing via embeddings with a semantic search step. The embeddings are generated from text passages (answers), which are then stored in an index. When a query is submitted, the system searches for the most semantically similar passages and uses them to generate an answer.

However, this raises an important issue: we are indexing the answers, not the questions. As a result, there can be a mismatch between how a user phrases their query and how the answer was originally represented in the index. This often leads to relevance problems.

ICT-QEI: Inverse Cloze Task for Question-Embedding Indexing

A promising alternative is ICT-QEI, which shifts the focus from answer-based indexing to question-based retrieval.

• Inverse Cloze Task (ICT): Instead of indexing passages, the text is treated as a source from which questions are generated. A model learns to associate each generated question with the part of the text that provides its answer.

• Question-based Retrieval / Q-Embedding Indexing: The index is built from the generated questions. When a new user query is submitted, the system searches among semantically similar questions and retrieves the corresponding answer passage.

This approach improves alignment with user intent, since both the query and the index are expressed in the same form: questions.

Key Challenges

Two main difficulties arise:

• Chunking strategy: Traditional embedding approaches split text into arbitrary chunks. To generate meaningful questions, however, the splitting must be logical and context-aware, not just size-based.

• LLM generation cost: Producing high-quality, diverse questions requires significant LLM inference, which can be expensive at scale.

When to Use This Approach

Question-based indexing is particularly valuable for structured, high-value documents, such as contracts or compliance manuals, where the cost of LLM-based preprocessing is justified by the precision and reliability of the retrieval process.