Building 🧠 for 🤖

Tokyo-to, Japan
Joined June 2018
Marko Simic retweeted
Meet @svegas18, @AustinBaggio , @christinetyip from autoresearch@home: > Wednesday, they release the most comprehensive autoresearch repo & leaderboard > Same day, we plan and launch an autoresearch hack > Thursday, they decide to fly y from Canada & NYC to present their work
1
7
1
30
3,156
How is that that one of the most powerful ideas in the entire history of technology is now under coordinated attack? If I told you that one little software concept powers the website you're reading this on, the router in your house that connects you to the internet, the phone in your pocket and your TV streaming service, what would you think? What if I said that the NASDAQ and New York stock exchanges where your retirement portfolio trades  is powered by the same ideas?  Or that those same concepts power 100% of the top 500 supercomputers on the planet doing everything from hunting for cures for cancer, to making rockets safer, to mapping the human brain? Not only that, it also powers all the major clouds, from Amazon Web Services, to Google Cloud and Alibaba and more. You'd probably think that was a good thing, right?  Not just good, but absolutely critical to the functioning of modern life at every level.  And you're be right. That critical infrastructure is "open source" and it powers the world. It's everywhere, an invisible bedrock beneath our entire digital ecosystem, underpinning all the applications we take for granted every day. It seems impossible that something that important would be under attack. But that's exactly what's happening right now.  Or rather, it's happening again.  It's not the first time. In the early days of open source, a powerful group of proprietary software makers went to war against open source, looking to kill it off. SCO Unix, a proprietary Unix goliath, tried to sue Linux into oblivion under the guise of copyright violations. Microsoft's former CEO, Steve Ballmer, called open source cancer and communism and launched a massive anti-Linux marketing barrage.  They failed and now open source is the most successful software in history, the foundation of 90% of the planet's software, found in 95% of all enterprises on the planet. Even Microsoft now runs on open source with the majority of the Azure cloud powered by Linux and other open software like Kubernetes and thousands upon thousands of packages like Docker, Prometheus, machine learning frameworks like Pytorch, ONNX and Deepspeed, managed databases like Postgres and MySQL and more. Imagine if they'd been successful in their early attacks on open source?  They would have smashed their own future revenue through short-sightedness and total lack of vision. And yet open source critics are like an ant infestation in your house.  No matter how successful open source gets or how essential it is to critical infrastructure they just keep coming back over and over and over. Today's critics of open source AI try to cite security concerns and tell us only a small group of companies can protect us from our enemies so we've got to close everything down again and lock it all up behind closed doors. But if open source is such a security risk, why does "the US Army [have] the single largest installed base for RedHat Linux" and why do many systems in the US Navy nuclear submarine fleet run on Linux, "including [many of] their sonar systems"?  Why is it allowed to power our stock markets and clouds and our seven top supercomputers that run our most top secret workloads? But they aren't stopping with criticism there.  They're pushing to strangle open source AI in its crib, with California bills like SB 1047 set to choke out American AI research and development while tangling up open weights AI in suffocating red tape, despite a massive groundswell of opposition from centrist and left Democrats, Republicans, hundreds of members of academia and more, the bill passed and now goes to the governor.  Eight members of Congress called on the governor to veto the bill. Speaker Pelosi took the unprecedented step of openly opposing a bill in a state Assembly that is primarily Democrat. And yet still the bill presses forward. Why? Do too many people just not understand what open source means to the modern world? Do they just not see how critical it is to everything from our power grid to our national security systems and to our economies? And the answer is simple: They don't. That's because open source is invisible. It runs in the background.  It quietly does its job without anyone realizing that it's there.  It just works. It's often not the interface to software, it's the engine of software, so it's under the hood but not often the hood itself.  It runs our severs and routers and websites and machine learning systems.  It's hidden just beneath the surface. The average person has no idea what powers their Instagram and Facebook and TikTok and Wikipedia and their email servers and their WhatsApp and Signal.  They don't know it's making their phone work or that the trades of their retirement portfolio depend on it. And that's a problem. Because if people don't even know it exists, how can we defend it? 1/
40
285
61
1,171
207,718
It's amazing to me that the year is 2024 and some people still equate task-specific skill and intelligence. There is *no* specific task that cannot be solved *without* intelligence -- all you need a sufficiently complete description of the task (removing all test-time novelty and uncertainty), and you can achieve arbitrary levels of skills while entirely by-passing the problem of intelligence. In the limit, even a simple hashtable can be superhuman at anything.
To those of you who claiming that smarter-than-human AI is decades away, what *specific tasks* are you willing to bet me that AI won't be able to do within 5 years?
43
173
34
1,186
211,040
Marko Simic retweeted
503
1,367
237
9,942
946,211
Marko Simic retweeted
AI at Meta
@AIatMeta18 Apr 2024
More technical details on the new Meta Llama 3 models announced today. 🦙🧵
30
253
27
2,576
367,883
can’t believe they asked what videos OpenAI scraped for training Sora and it’s literally this meme
35
286
14
4,310
370,216
New video series: Physics Informed Machine Learning! Physics may be embedded into AI/ML in 5 stages: 1 choose what to Model 2 curate training Data 3 design an Architecture 4 craft a Loss Function, and 5 implement Optimization Algorithm to train the model youtube.com/watch?v=JoFW2uSd…
23
194
13
1,245
78,600
Bye bye video evidence! 👋 Anything in Any Scene can add objects into videos and maintain the same level of photorealism as the original footage. The thing can even handle occlusions and lighting as well as generate shadows for the inserted objects 🤯 anythinginanyscene.github.io…
22
160
38
760
89,543
Marko Simic retweeted
Nous Research
@NousResearch15 Jan 2024
Introducing our new flagship LLM, Nous-Hermes 2 on Mixtral 8x7B. Our first model that was trained with RLHF, and the first model to beat Mixtral Instruct in the bulk of popular benchmarks! We are releasing the SFT only and SFT+DPO model, as well as a qlora adapter for the DPO today. Mixtral Nous-Hermes 2 DPO: huggingface.co/NousResearch/… Mixtral Nous-Hermes 2 SFT: huggingface.co/NousResearch/… Mixtral Nous-Hermes 2 DPO Adapter: huggingface.co/NousResearch/… (1/2)
35
113
22
682
490,470
While intelligence *leverages* compression in important ways in representation learning, intelligence and compression are by nature opposite in key aspects. Because intelligence is all about *generalization to future data (out of distribution)* while compression is all about *efficiently fitting the distribution of past data*. If you're optimal at the latter, you're terrible at the former. If you were an optimal compression algorithm, the behavior policy you would develop during the first 10 years of your life (maximizing your extrinsic rewards such as candy intake, while forgetting all information that appears useless as per past rewards) would be entirely inadequate to handle the next 10.
Based on our latest study: arxiv.org/abs/2311.13110, compression seems to be all there is in current AI systems, including GPT-4. The remaining question is: Can compression alone lead to general intelligence or even consciousness? My bet is a clear NO.
55
159
25
1,001
376,323
Marko Simic retweeted
Elon Musk
@elonmusk18 Nov 2023
12,577
27,025
2,020
450,754
38,724,808
Marko Simic retweeted
Moral of the story: don't let EA's on your board
25
27
3
714
56,383
Silly AI regulation hype One cannot regulate AI research, just like one cannot regulate math. One can regulate applications of AI in finance, cars, healthcare. Such fields already have continually adapting regulatory frameworks in place. Don’t stifle the open-source movement! LAION's open letter: laion.ai/blog/transparent-ai… @hardmaru's tweet on the 2023 Forbes interview: xitter.cf/hardmaru/status/…
Interview with Jürgen Schmidhuber, ‘Father Of Modern AI’, who says his life’s work won’t lead to dystopia. He goes quite in depth into his views about the future of AI and AGI. A refreshing view of things, compared to other ‘influential leaders’ in AI. reddit.com/r/MachineLearning…
47
193
32
1,130
365,403
Marko Simic retweeted
Fight inflation by more money printing! Genius move, 4D chess stuff! 👏👏👏 – at Minato-ku, Tokyo
Japan to approve $110bn stimulus package to fight inflation s.nikkei.com/3Mq7oTu
1
1
132
Marko Simic retweeted
LM Studio 0.2.8 now available! 🎃👻 ... 🥁 ... NEW: Local Server Config Presets! 💾 Presets for Mistral, Zephyr, OpenHermes & more ⚙️ No need to manually add prompt prefix/suffix in your client code (e.g. ChatML) ✅ Expect model output quality boost with AutoGen, Rivet
20
38
7
219
105,981
Marko Simic retweeted
Great educational content at #nostrasia 😆
1
1
2
328
Marko Simic retweeted
The U.S. political and fiscal situation is hopelessly broken, but we have one unparalleled asset as a country: cutting-edge innovation in AI driven by a completely free and unregulated market for software development. That just ended. whitehouse.gov/briefing-room…
241
289
43
1,799
275,825
In 270 days, the Department of Commerce will determine whether they will allow open-weights or not. if you support open model weights and want something actionable to do, then figure out how to lobby your opinion to them.
33
150
58
668
468,824