When generating responses, LLMs (Large Language Models) follow specific processes to retrieve and incorporate information. The method can vary depending on whether the model is using only its training data or is enabled to perform external searches.
How LLMs Retrieve Information
1. Input Processing
- The user's prompt is converted into a vector representation (embedded in vector space).
2. Query Expansion (Synthetic Fan-Out)
- The model automatically generates multiple paraphrased versions of the original query to improve retrieval accuracy.
3. Search of External Sources
- Searches are performed across curated vector databases or trusted APIs, depending on the model's capabilities and configuration.
- Only approved and designated sources are used for retrieval.
4. Relevance Scoring
- Retrieved documents are scored based on:
- Semantic similarity to the prompt
- Authority of the source
- Format and usefulness of the content
5. Passage Chunking
- Instead of retrieving entire documents or full web pages, the system extracts the most relevant 100–300 word chunks (passages) from the sources.
6. Context Injection
- The selected chunks are injected into the LLM prompt as external context, allowing the model to generate a more informed and accurate response.
Key Points to Remember
- LLMs do not retrieve or use full documents; only relevant passages are used.
- When enabled, external search augments the model's responses with up-to-date or authoritative information from curated sources.
- The retrieval process is designed to maximize relevance and accuracy while minimizing unnecessary or irrelevant information.
- Users can influence retrieval quality by writing clear, focused prompts.
Example Scenario
Question: What is the capital of France?
LLM Process:
- The prompt is embedded and paraphrased.
- The system searches trusted databases for relevant passages.
- A 100–300 word chunk mentioning "Paris" as the capital is retrieved and injected as context.
- The LLM generates the response "Paris," supported by the external context.
Frequently Asked Questions
Do LLMs search the web in real time?
Some LLMs can perform real-time searches across curated databases or trusted APIs, but they do not arbitrarily browse the open internet. Only pre-approved sources are used.
Can LLMs access private or confidential data?
No. LLMs cannot access private or proprietary information unless it is explicitly provided as input or included in the connected external sources.
What kind of information do LLMs retrieve?
LLMs retrieve concise, relevant text chunks (not entire documents) from selected sources to support accurate and context-aware responses.