bohemian, private eye, hacker, designer, teacher, data scientist.

Chicago
Joined October 2008
Ulan bunca yüzyıl 'ennui' diye hava at, varoluşsal felsefeler yükle, ihtişamlı bi ıstırap muamelesi çek... Bildiğin dopamin eksikliği ya. D vitaminin eksikse çık güneş gör denir, dopaminin eksise sosyal medyaya ara ver denir, bu kadar basit. (Sartre'a söylemeyin dayak yemeyelim.)
62
Certainty is caused by a severe lack of imagination.
67
Rest in weird beauty, you lovely poet.
1
77
Dick Laurent is dead. He was my sunshine, he was my lost highway. He was kind, masterful, and insane. He probably parked his spaceship in a dog’s dream, smiling down on us Laura Palmers. Goodbye innocently perverse prince. Goodbye moon. Goodbye cowboy. Today’s number is 7.
97
Can the UFC sign Dakota Ditcheva already please? She's going to add a ton of excitement to flyweight.
2
229
If true, I’m 100% rooting for Caitlin Clark’s decision to leave an ungrateful WNBA for Europe. After years of obscurity and financial trouble, a generational talent/savior like Bird or Jordan descended from the heavens, and I can’t imagine how WNBA could have fumbled it worse.
2
10
1,315
Paul Auster, the contractor that built the lean to expansion of my heart, is dead. Starting from the moment I read his New York trilogy, I felt admiration, love, and a warm bond with this wonderful man. I bequeath that extension to him now. He can live there forever, rent-free.
1
160
Martin Scorsese talking to his parents about sauce is canon Italianamerican (1974) Scorsese
250
2,275
527
14,862
# On the "hallucination problem" I always struggle a bit with I'm asked about the "hallucination problem" in LLMs. Because, in some sense, hallucination is all LLMs do. They are dream machines. We direct their dreams with prompts. The prompts start the dream, and based on the LLM's hazy recollection of its training documents, most of the time the result goes someplace useful. It's only when the dreams go into deemed factually incorrect territory that we label it a "hallucination". It looks like a bug, but it's just the LLM doing what it always does. At the other end of the extreme consider a search engine. It takes the prompt and just returns one of the most similar "training documents" it has in its database, verbatim. You could say that this search engine has a "creativity problem" - it will never respond with something new. An LLM is 100% dreaming and has the hallucination problem. A search engine is 0% dreaming and has the creativity problem. All that said, I realize that what people *actually* mean is they don't want an LLM Assistant (a product like ChatGPT etc.) to hallucinate. An LLM Assistant is a lot more complex system than just the LLM itself, even if one is at the heart of it. There are many ways to mitigate hallcuinations in these systems - using Retrieval Augmented Generation (RAG) to more strongly anchor the dreams in real data through in-context learning is maybe the most common one. Disagreements between multiple samples, reflection, verification chains. Decoding uncertainty from activations. Tool use. All an active and very interesting areas of research. TLDR I know I'm being super pedantic but the LLM has no "hallucination problem". Hallucination is not a bug, it is LLM's greatest feature. The LLM Assistant has a hallucination problem, and we should fix it. Okay I feel much better now :)
662
2,334
700
14,687
2,361,389
I officially HATE Google Search now. Long time ago, Google realized that users wanted to find what they were looking for within the first 1-3 results, and developed an algorithm to do that. Today, the entire first half of the results page are ads. I guess Bing it is o_0
2
164
Irmak Sirer retweeted
ChatGPT can now see, hear, and speak. Rolling out over next two weeks, Plus users will be able to have voice conversations with ChatGPT (iOS & Android) and to include images in conversations (all platforms). openai.com/blog/chatgpt-can-…
1,647
8,434
4,504
37,550
10,672,462
Irmak Sirer retweeted
Yann LeCun is obviously a legend but I found this tweet to be quite misinformed. The whole point of "emergent abilities" such as few-shot prompting and chain-of-thought prompting, is that we clearly *did not* explicitly train or fine-tune them into the model. These abilities require scaling up to a lot of parameters, which is the reason we call them "emergent"---they are quantitive behavior shifts that come from qualitative changes. At least this is how researchers use the term. All we did was train a language model with a lot of parameters on a big dataset. These abilities were not intentionally added into the model during training---rather, people found them via evaluations after the model was trained. A great pointer to this is Jacob Steinhardt's post: bounded-regret.ghost.io/emer… What's also significant is that we often aren't able to predict these emergent phenomena ahead of time, and this has safety implications since risks can also be emergent. Also, I don't know why finetuning is mentioned, because these abilities are clearly in the base model, which is should be apparent from papers before RLHF became popular... (e.g., PaLM-1, GPT-3 paper) But even if you insist to take a finetuning as an example, ChatGPT RLHF was originally done in English, and it extended to many other languages. This multilingual ability was not "finetuned" into the model, which is a pretty good example of zero-shot generalization. Please correct me if I'm wrong, but the technical use of the term definitely still holds. Mustafa's tweet is also a bit misinformed, in my circles people don't use "emergence" to mean autonomy or agency. Again, what AI researchers usually mean is that it is a qualitative change that arises from quantitative changes.
99
141
35
1,345
363,485
Irmak Sirer retweeted
National Lab (LBNL) results support LK-99 as a room-temperature ambient-pressure superconductor. Simulations published 1 hour ago on arxiv support LK-99 as the holy grail of modern material science and applied physics. (arxiv.org/abs/2307.16892) Here's the plain-english explanation: - The simulations modeled what the original Korean authors proposed was happening to their material - where copper atoms were percolating into a crystal structure and replacing lead atoms, causing the crystal to strain slightly and contract by 0.5%. This unique structure was proposed to allow this amazing property. - @sineatrix from Lawrence Berkeley National Lab simulated this using heavy-duty compute power from the Department of Energy, and looked to see what would happen to the 'electronic structure' of this material, meaning, what are the available conduction pathways in the material. - It turns out that there are conduction pathways for electrons that are in just the right conditions and places that would enable them to 'superconduct'. More specifically, they were close to the 'Fermi Surface' which is like the sea-level of electrical energy, as in '0 ft above sea-level.' It's believed currently that the more conduction pathways close to the Fermi surface, the higher the temperature you can superconduct at (An analogy might be how its easier for planes to fly close to the surface of the ocean due to the 'ground effect' that gives them more lift.) This plot in particular shows the 'bands', or electron pathways, crossing above and below the Fermi surface. - Lastly, these interesting conduction pathways only form when the copper atom percolates into the less likely location in the crystal lattice, or the 'higher energy' binding site. This means the material would be difficult to synthesize since only a small fraction of crystal gets its copper in just the right location. This is insanely bullish for humanity.
478
5,418
2,800
18,544
13,301,866
Irmak Sirer retweeted
As a former materials physicist, I am genuinely unsure of what to make of this superconductivity paper. 🧵 👇
53
250
45
3,240
1,344,553
You can tell the authors of the new room temperature superconductor paper believe in their results because they published two papers about it concurrently: one with six authors, and one with only three. The Nobel prize can be shared by at most three people.
221
1,982
528
18,303
2,999,872
Today might have seen the biggest physics discovery of my lifetime. I don't think people fully grasp the implications of an ambient temperature / pressure superconductor. Here's how it could totally change our lives.
2,166
18,670
5,150
113,786
30,784,703
Christopher Nolan's 12 Angry Men
59
345
39
2,976
165,343
I really like the cleverly simple (if you’re familiar with the specific math) big picture descriptions of what’s going on and how softmax is used in different contexts. For a technical blog post, I think this is excellent.
I hit a bug in the Attention formula that’s been overlooked for 8+ years. All Transformer models (GPT, LLaMA, etc) are affected. Researchers isolated the bug last month – but they missed a simple solution… Why LLM designers should stop using Softmax 👇 evanmiller.org/attention-is-…
1
284
This is a strangler fig tree, Inside this tree is a hollow space where a different tree stood The strangler fig's seeds have made their way into the canopy of a host tree and germinated. As the fig's roots grow, they cascade down the trunk, Once they are in the ground it competes for nutrients and water with the host. Then It gradually tightens its grip around the host tree This process hinders the flow of water and nutrients, causing the host tree to weaken and eventually die. Over time, the strangler fig takes over completely, once the process is complete, the original tree decomposes, a hollow centre is all that remains to mark its existence
934
14,374
1,637
131,394
14,664,949