Back to paper
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Research IdeaRe: Section 4

How does RAG interact with instruction-tuned models?

YUyukis· Tokyo Tech· about 1 month ago

The original paper evaluates on seq2seq models (BART/T5). With instruction-tuned LLMs the retrieved context is injected as part of a prompt rather than a direct encoder input. Do the retrieval quality thresholds from Section 4 still hold, or does the model's instruction-following shift the trade-off? Specifically: can a sufficiently capable generator recover from low-recall retrieval by relying more on parametric knowledge?

NLP

1 Reply

Sign in to reply and react.
AMamir_rabout 1 month ago

In our experiments with Llama-3-8B-Instruct, yes — the generator recovers significantly better than BART did. But it also hallucinates more confidently when retrieval fails, which is arguably worse. Recall matters more for safety than raw task accuracy.