Ankur Teredesai retweeted
Replying to @karpathy
The tldr I use: "LLMs always hallucinate. Sometimes their hallicinations align with your reality". Whether or not the prompt makes them hallucinate in a way that aligns with reality depends very much on the prompter's ability to check, and thus.. xitter.cf/rao2z/status/171871473… )
Why we should view LLMs as powerful Cognitive Orthotics rather than alternatives for human intelligence #SundayHarangue
LLMs are amazing giant external non-veridical memories that can serve as powerful cognitive orthotics for us, if rightly used (c.f. xitter.cf/rao2z/status/170346612…).
The trick, IMHO, is to exploit them without deluding ourselves in the process.
The delusion comes chiefly from our incessant need to confuse them for human intelligence, merrily applying anthropomorphic concepts such as thinking , thoughts, reasoning and self-critiquing to LLMs. (c.f. xitter.cf/rao2z/status/170832978…;
xitter.cf/rao2z/status/169710160…;
xitter.cf/rao2z/status/164890979…)
This anthropomorphization is quite futile--and, as shown in the case of some of the current Ersatz Natural Science AI literature--even counterproductive and misleading.
Sure we didn't quite foresee how impressive the approximate omniscience of these n-gram models on steroids would be, but that doesn't have to make us assume they do everything humans do.
Unless human-level #AI is your singular goal, you don't necessarily need to think auto-regressive LLMs suck (as @ylecun puts it colorfully).
LLMs can be very effective complementary cognitive orthotics without subsuming human intelligence.
LLMs do some things way way way better than humans do (the litmus test--from my perspective--being converting anything to iambic pentameter in seconds 😋xitter.cf/rao2z/status/171863933…), and do other things (planning, reasoning, self critiquing, mental modeling) much worse (c.f. ;
xitter.cf/rao2z/status/170161286…;
xitter.cf/rao2z/status/171580081…; xitter.cf/rao2z/status/164346320…)
If we can manage to tone down the "LLMs are Zero-shot <XXX>" studies rife with confirmation biases that conflate approximate retrieval for other capabilities associated with human intelligence (c.f. xitter.cf/rao2z/status/155308270…; xitter.cf/rao2z/status/170785801…), we can:
1. Focus on the right way of leveraging the strengths of LLMs. This can certainly be done in LLM-modulo architectures, with either humans or other specialized sound reasoners in the loop).
2. To the extent human-level intelligence is still your holy grail, keep open research avenues that don't just involve scaling up autoregressive architectures. (It is in this sense that I sympathize with @ylecun's AR-LLMs suck comment--they do, currently, suck the oxygen out of the research room 😅)
tldr; you can get much farther with LLMs & #AI if you think of them as cognitive orthotics and can stop anthropomorphizing them.. (c.f. xitter.cf/rao2z/status/167866072…;
xitter.cf/deliprao/status/168681…)
Solving Tensor Puzzles will set you free of the matrix. Great way to learn. Thanks @srush_nlp
Do you (really) know PyTorch 🔥?
Try out my Tensor Puzzles 🧩. 16 mini-puzzles for those ready to take off the stackoverflow training wheels.
github.com/srush/Tensor-Puzz…
Ankur Teredesai retweeted
🎖 This really is such a bookmark-worthy post:
📊 “The Evolution of a ggplot (Ep. 1)” by @CedScherer
buff.ly/3hP4A3B #rstats #dataviz
[GIF: shows each stage of evolution of the plot]
Ankur Teredesai retweeted
Glad to announce that we have 3 papers and 1 tutorial accepted at @2021Ichi Details coming soon. Collaborative work with @ankurt @md_carly @vikasnitr and others #ICHI2021
Ankur Teredesai retweeted
First time I heard #LinearAlgebra in a movie today: ‘the eigenvalue is all wrong, just program a Fourier transform’ from ‘No way out’ (Kevin Costner et al). Didn’t make much sense of course, but still commendable. Any other such gems I am missing out? #Mathinmovies
Ankur Teredesai retweeted
So proud to see the day that a woman has finally broken through the impenetrable glass ceiling of American high political office @KamalaHarris
I would absolutely use this primer for ML coursework using python.
Probably one of the best free resources I've found online!
An introduction to linear algebra for machine learning with Python 🐍.
Bookmark this link in your browser and use it as a reference whenever you need it.
pabloinsente.github.io/intro… via @CodeBug88
Ankur Teredesai retweeted
Always. Even when it's hard. Even when it's unpopular. Even when it's painful. Just do the right thing. #MLKDay2021
"The time is ALWAYS right to do what is right."
Dr #MartinLutherKingJr
Ankur Teredesai retweeted
tfw the two hardest problems in computer science turn out to be "not building a hostile workplace" and "not actively participating in white supremacy/genocide"
Ankur Teredesai retweeted
Learning one of the most important ML and data science concepts, Bayes Theorem in the context of chilling with your BAE 😀
Probability of chilling given Netflix = (Probability of Netflixing given chilling x Probability of chilling) / Probability of Netflixing
A Machine learning algorithm walks into a bar and the bartender asks, “what will you have?”
The machine learning algorithm says: “whatever is the majority class having.” Because it is biased and has been for years.
Ankur Teredesai retweeted
"The number one skill required for learning any complex system is patience." - Kelsey Hightower
Ankur Teredesai retweeted
“There are decades where nothing happens; and there are weeks where decades happen.”
The first three Wednesday’s of 2021:
1/6: insurrection
1/13: impeachment
1/20: inauguration
Every sentence is a gem of advice. To be or not to be.
Every thought something to think twice.
Academic or otherwise.
😉🥂
Ankur Teredesai retweeted
15 years! So hard to believe... feels like a lifetime ago!
Happy 15th Birthday jQuery! On January 14, 2006, the software engineer John Resig announced the creation of the jQuery JavaScript library at BarCamp NYC.
webdesignmuseum.org/web-desi…
#jQuery #InternetHistory
Ankur Teredesai retweeted
New paper testing whether "natural language understanding" benchmarks actually test language understanding. Answer: No.
Do SotA natural language understanding models care about word order?
Nope 🙃, 75% to 90% of the time, for BERT-based models, on many GLUE tasks (where they outperformed humans).
"marijuana cause cancer" == "cancer cause marijuana" Ouch...
arxiv.org/abs/2012.15180 1/4
Ankur Teredesai retweeted
Science often requires having the mental fortitude to persist with a long term research agenda, even when the rest of the field moves on.
A fascinating case study, worth the read! wired.co.uk/article/mrna-cor…
How Ugandan Nasa scientist Catherine Nakalembe uses satellites to boost farming.. bbc.co.uk/news/world-africa-…



































