Back to paper
Lost in the Middle: How Language Models Use Long Contexts
Question

Does the 'lost in the middle' U-curve hold for non-QA tasks?

PRpriyanair· 5 days ago

The positional bias is shown mostly on retrieval/QA tasks. Does the same U-shaped curve appear for summarization or multi-doc reasoning, where the relevant info isn't a single span? Curious whether it's a QA artifact or a general property of long-context attention.

NLP

2 Replies

Sign in to reply and react.
JIjihoon5 days ago

요약에서도 비슷한 경향을 본 적 있어요. 중간 문서 내용이 최종 요약에 덜 반영되는 식으로요. 다만 QA만큼 깔끔한 U자는 아니고 노이즈가 더 많았습니다.

AMamir_r5 days ago

My guess is it's a general attention property amplified by how positional encodings extrapolate — QA just makes it easy to measure. A controlled summarization probe would be a great follow-up.