We validate this flag-based eager evaluation hypothesis with a series of carefully designed causal mediation analysis. If we swap this flag with another item in the list, in the question-before context the LM consistently picks the item carrying the flag. However, the question-after prompt doesn’t show any sensitivity to this intervention. Check our paper for full details!

📊 We measure this with a *causality* score: if the predicate is abstractly encoded in the query states of these "filter heads", then transferring it should change the output. For example: in the figure the answer should change to "Peach" (or the changed format accordingly). We test this across a range of different semantic types, presentation formats, languages, and even different tasks that require a different "reduce" step after filtering.

🤔 But do these heads play a *causal* role in the operation? To test them, we transport their query states from one context to another. We find that will trigger the execution of the same filtering operation, even if the new context has a new list of items and format!

🔍 In Llama-70B and Gemma-27B, we found special attention heads that consistently focus their attention on the filtered items. This behavior seems consistent across a range of different formats and semantic types.

How can a language model find the veggies in a menu? New pre-print where we investigate the internal mechanisms of LLMs when filtering on a list of options. Spoiler: turns out LLMs use strategies surprisingly similar to functional programming (think "filter" from python)! 🧵

But, even without attention, we can do a similar experiment: we can block out information flow by mean-ablating the subject/non-subject tokens to *all* the future tokens in different layers to understand how Mamba uses Conv + SSM in which layers relay what information.

How about path-dependent attention knockout experiments? @megamor2 used these to understand factual information flow in transformers. But in Mamba, which has no attention, these methods cannot be applied "out-of-the-box" (Section 6 of our paper)

Then we ask: can we apply ROME to update/insert a fact in Mamba? ROME can be applied directly as well! When applied to different projection matrices in MambaBlock across all the layers, we see that ROME can successfully update/insert a fact in a range of layers.

Activation patching can also reveal interesting insights about the role of different components inside of a MambaBlock. Notice that the "s" states are active at the late site, similar to attention in transformers 🤯. Whereas the "o" and "g" states are active at the early site.

Can we combine these insights and apply activation patching methods to RNNs, even if they do not have attention heads or MLPs? It turns out that activation patching, also known as causal tracing, can be directly applied in Mamba just like it can be done in transformers!

How does Mamba store knowledge? Is it very different from transformers? New pre-print with @diatkinson and @davidbau, where we investigate the mechanisms of factual recall within Mamba.