Karim Touma (@karim_op) header
Karim Touma (@karim_op) profile picture

think mythically, act pragmatically, lead systemically. - I have discovered a truly marvelous proof of p=np, which this Bio is too narrow to contain.

San Pedro Garza García
Joined February 2009
Photo by Karim Touma (@karim_op)Photo by Karim Touma (@karim_op)Photo by Karim Touma (@karim_op)Photo by Karim Touma (@karim_op)Photo by Karim Touma (@karim_op)Photo by Karim Touma (@karim_op)Photo by Karim Touma (@karim_op)Photo by Karim Touma (@karim_op)Photo by Karim Touma (@karim_op)Photo by Karim Touma (@karim_op)
Antes de darle a un agente permiso para emitir reembolsos, define tres cosas: monto máximo, evidencia exigida y cuándo debe escalar a una persona. La autonomía se diseña decisión por decisión. ¿Qué permiso revisarías primero en tu operación?
70
Hoy estaba armando una minuta desde un audio, me di cuenta que Gemini aún sigue auto-percibiendose como Bard 🤡
1
2
1,070
Karim Touma retweeted
There's a duckduckgo_search package. You don't even need an API key to use it
48
239
20
3,998
466,389
Multi-Head RAG (MRAG), aims to improve retrieval accuracy for complex queries requiring that require fetching multiple documents with substantially different contents. Real-world use cases to demonstrate MRAG improves up to 20% in relevance over standard RAG baselines. Paper - Multi-Head RAG: Solving Multi-Aspect Problems with LLMs 📌 For multiple documents with substantially different contents the embeddings of these documents may be distant in the embedding space, making them hard to retrieve. 📌 So the key idea behind MRAG is to leverage the activations from the multi-head attention layer of the transformer decoder, instead of just the feed-forward layer, to generate embeddings. Different attention heads can capture different aspects of the data, so using these activations results in embeddings that better represent the various facets of documents and queries. The Transformer architecture can be seen as a pipeline with many (e.g., 96 for GPT-3 ) blocks, where a single block consists of an attention module and a feed-forward module. Each individual attention module is multi-headed: it consists of multiple parts called heads that learn different sets of weight matrices. It is conjectured that these different heads could capture different aspects of the processed data. They use this as a driving design feature that facilitates capturing the potential multi-aspectuality of the data without increasing space requirements compared to standard RAG 📌 MRAG uses the outputs from each attention head for the last token as a set of "single-aspect" embeddings. The full set of these embeddings forms the "multi-aspect" embedding. Importantly, this requires no additional space compared to standard RAG that just uses the final decoder layer output. 📌 The MRAG pipeline involves 1) Preparing a data store with multi-aspect embeddings for each document/chunk, 2) Generating a multi-aspect embedding for the input query, 3) Using a special multi-aspect retrieval strategy to find the most relevant documents. This voting-based retrieval strategy considers the importance of each attention head. 📌 MRAG is versatile and can be integrated with existing RAG pipelines, data stores, and benchmarking frameworks. It achieves retrieval improvements without requiring extra LLM queries, model instances, storage, or inference passes.
10
85
2
425
51,735
Karim Touma retweeted
I think AI agentic machine translation has huge potential for improving over traditional neural machine translation, and am releasing as open-source a demonstration I'd been playing with as a fun weekend project. Using an agentic workflow, this demonstration (i) Prompts an LLM to translate from one language to another, (ii) Reflects on the translation to come up with constructive suggestions, (iii) Uses the suggestions to refine the translation. In our limited testing, this is sometimes competitive with, and sometimes worse than, leading commercial providers. But it gives a highly steerable translation system where by simply changing the prompt, you can specify the tone (formal/informal), regional variation (do you want Spanish as spoken in Spain or as spoken in Latin America?), and ensure consistent translation of terms (by providing a glossary). This is not mature software. But I hope the open-source community can make agentic translation work much better. Given how a simple reflection workflow already gives decent results, I think there's significant headroom to make agentic translation much better. Releasing an early software prototype like this is something new I decided to try to see if it is helpful to the developer community. I'd love any feedback on this. Thanks to Joaquin Dominguez, @nedteneva, @JohnSanterre for help with this. github.com/andrewyng/transla…
63
339
52
1,700
549,399
Karim Touma retweeted
🛝Building RAG with Powerpoint presentations Will use unstructured and HuggingFace to do RAG over your powerpoint presentations! Another fantastic notebook example from @mariaKhalusova colab.research.google.com/dr…
7
99
2
501
46,258
you guys didn't believe me last time so here's an uncut video of me building and deploying a fullstack AI app in less than 10 mins using @v0, @cloudflare ai and workers, @honojs and nextjs there's no better time to build than now. and there's no reason to not keep shipping.
i shipped something today with another friend at a hackathon made in 15 minutes you know, making your online persona is kinda hard, so why not make it instantly pfp.dhr.wtf
49
52
12
1,080
206,645
Karim Touma retweeted
If you want an open-source version of the ChatGPT CSV/chart capabilities, then look no further - @MarcusSchiesser has created a comprehensive full-stack template that you can use to jumpstart your own data analyst. This agent template includes the following: 1. An agent that can reason over CSV files with intermediate steps 2. Chart visualization 📊 All behind a full chat interface with native chat history, streaming, and sources. Check it out on create-llama: github.com/run-llama/create-…
Build a data analyst using @llama_index in no time. Just start a project in create-llama and add the code interpreter tool (using the sandbox from @e2b_dev) Upload a CSV file and get some insights. 🖥️ npx create-llama 📃 github.com/run-llama/create-…
5
33
238
55,918
Karim Touma retweeted
Hidden gem google repos you've never heard of. The biggest alpha ever .... Google/seqio Google/MaxText Google/praxis Google/paxml Google/saxml Google-research/big_vision
4
6
124
13,880
Karim Touma retweeted
it's 1:28AM and I just finished this abomination. fully illustrated toy calculation of 1 transformer layer. why would I make this? idk ask my thesis advisor, "not everyone knows how a transformer works, you have to give an example"
116
298
48
4,146
405,514
Karim Touma retweeted
Introducing Gemini UI to Code 🧑‍💻 A small but powerful app that uses an agentic framework to convert an image to code. Perfect as a starter template or for learning new tricks. Thanks to its improved Vision capabilities, Gemini 1.5 Pro is perfect for this task. 🪄
25
72
12
506
196,764
Generate Structured LLM Outputs with Instructor!🚀 Ever felt overwhelmed by handling structured outputs from large language models (LLMs)❓ Say hello to Instructor, a Python library that makes it a breeze to work with structured LLM outputs. Built on the robust framework of Pydantic, Instructor offers a clean, transparent, and easy-to-use API that ensures your LLM interactions are nothing short of spectacular. Here are some key features: Structured to Perfection: Define the structure of your LLM outputs with Pydantic models for seamless integration. Retry Made Easy: Configure the number of retries effortlessly, ensuring your requests always get through. Validation for Accuracy: Ensure all LLM responses meet your exact specifications with built-in validation. Stream Like a Pro: Handle streaming responses with ease, thanks to comprehensive support for Lists and Partial objects. Flex Your Backend: Compatible with various LLM providers beyond OpenAI, Instructor is as versatile as it gets. Multi-language Support: Whether you're coding in Python, TypeScript, Ruby, Go, or Elixir, we've got you covered. I have shared link to their GitHub in the next tweet! Find me → @akshay_pachaar ✔️ For more content like this! Check out Instructor in action with this simple example:
7
81
3
447
57,794
Karim Touma retweeted
📩Local RAG with email Emails are one of the most natural things to do RAG over, but you may want to do it in a local way. Luckily, @mariaKhalusova has you covered with a great resource on how to do this with Unstructured, Ollama and LangChain colab.research.google.com/dr…
7
67
360
37,447
Karim Touma retweeted
Question: if you're building agents, how are you building memory? - Are you doing the basic thing of a rolling window over your conversation history? - Are you using different types of indexing like vector search or knowledge graphs? - Are you doing abstractive summarization? Drop your thoughts, would love to know which patterns developers are adopting
🎥 Webinar Recording: Memory for Autonomous Agents 🧠 The recording of our recent webinar on memary - a fully open-source reference implementation for long-term memory in autonomous agents - is now available. In this session, the authors of memary, @JulianSaks, Kevin Li, Seyeong Han provide a deep dive into the project and engaged in a thought-provoking discussion and Q&A session around memory, including a multi-agent future with co-learning. In case you haven’t seen it, memary introduces a novel architecture that combines sequential memory with knowledge graphs. This helps provide personalization in the context window but also maintains a larger KG you can query. If you missed the live webinar or want to revisit it, you can now watch the recording here: youtu.be/o0DPxvgML5c Check out the source repo too: github.com/kingjulio8238/mem…
17
43
1
288
66,764
OK. Each neuron has a billion tubulins switching at 10 megahertz for 10^16 ops/sec/neuron for 10^27 ops/sec/brain. But computation doesn’t give you consciousness. Here we showed quantum effects in microtubules at warm temps inhibited by anesthesia. pubs.acs.org/doi/10.1021/acs…
This quoted post is unavailable.
4
15
1
84
6,373
An excellent video on Kolmogorov Arnold Network. #KAN
2
72
2
442
31,497
New market maps covering the latest AI startups. 1. AI agents from Insight Partners
51
186
13
1,039
327,296
Karim Touma retweeted
I've spent the past ~3 weeks going through the entire history of deep learning and reimplementing all the core breakthroughs. It has completely changed my beliefs about deep learning progress and where we're headed. Progress tracker in thread (all resources at the end) 👇
52
356
43
2,808
637,130
Some useful prompting strategies here:
26
358
28
3,716
1,023,480
Introducing `Personalized Agentic RAG` - where the LLM remembers key details about the user and automatically chooses the tool for RAG. We'll build: 👏 ChatGPT like memory ⚙️ Function calling 🧙 Multi-Agent orchestration code: git.new/personalization
7
107
5
559
45,087