1
Don't trust large context windows
I recently watched a video that put a name on something I'd been feeling. The author splits an LLM's context window into two zones. There's the smart zone , where the model is sharp, and the dumb zone , where attention drops off and the model starts forgetting what you told it five minutes ago. The cutoff sits…