LLM knowledge is a lot more "patchy" than you'd expect. I still don't have great intuition for it. They learn any thing in the specific "direction" of the context window of that occurrence and may not generalize when asked in other directions. It's a weird partial generalization. The "reversal curse" (cool name) is imo a special case of this.