@hypernicon

AI researcher, technologist, armchair philosopher, composer, and family man

Texas
Joined March 2023
"The Apple of Loving Grace" -GPT (made with AI)
1
44
Imagine thinking that Geoff Hinton invented LLMs...
it's what the people INSIDE of tech are saying about LLMs that should concern you like, you know, geoffrey hinton, the guy who basically invented them
1
24
I feel like the whole immigration debate is missing a key aspect of the problem: So long as employers and universities can recruit from abroad, then have little incentive to fix the absolute disaster in the US education system K-12. So, if we can cut skilled immigration and make it harder for international students to study in US universities, then corporations and universities will suddenly have a reason to fix US education.
2
4
460
And the first fix should to abolish education schools, teachers' unions, and teacher certification. Teaching children is not a separate discipline from other subjects. Elementary school teachers should have degrees in History, Literature, Psychology, Math, or Science. Nearly every "advance" coming out of the Education departments has failed and is in fact damaging to our children: whole-word reading, socio-emotional learning, multi-cultural history rather than western civ, multiple-ways-to-do-it math at the elementary level, etc.
3
291
This analysis is spot on. The real H1b problem is almost entirely in her category 3 (non-tech companies huring te h workers), which is also likely the vast majority of H1bs in the entire country. I work mainly in startups, and the H1bs I interact with there are outstanding. But most H1bs are working for companies like Walmart or Disney, and they are doing jobs that could easily be done by Americans from any part of the country, jobs that are frankly not that difficult and only require an IQ of 120 or so.
It's that time of the year where everyone goes nuts on H1b. Let's just talk about tech jobs as a whole and where abuse exists today. Buckle up. There are 3 categories of tech jobs. 1. Tech jobs in cutting-edge large tech companies (Goog/OAI/Anthropic etc) 2. Tech jobs in cutting-edge startups (YC & other early stage companies). 3. Tech jobs in non-technical large companies (Banks/ Healthcare etc)
2
239
Individuals differ greatly from population averages, but stereotypes have many avatars.
3
95
To run a large country, you need a large bureaucracy. But how do you run the bureaucracy? With another bureaucracy? This is the problem of governance in the modern world. 1/3
1
3
231
Representative republicanism has failed; the representatives cannot meaningfully constrain the bureaucrats. Democracy, representative or direct, is a recipe for disaster. 2/3
1
1
152
The answer is what I call "the rule of algorithm", a new system of governance where representatives vote on a set of utilities or outcomes. An computer system then generates and administers policies towards those outcomes, constrained by a set of values (e.g. Bill of Rights) 3/3
55
Does anyone else think need a better notation for linear tensor operations? Reading papers without the code it's sometimes hard to see what the operation actually was. Does this bother anyone else? I suggest we should do something based on einsum notation, e.g. $\left[\underset{a}{ijk}\underset{b}{kn}\right]_{in}$ for $\sum_{j,k} a_{ijk} b_{kn}$.
4
217
Yep, though it's more like a century and a half (to the 1870's) than just over a century.
"The ejection of Maduro from Caracas is best understood as an invitation to turn the political clock back just over a century. The more I contemplate the contemporary scene, the more I think we have collectively accepted the invitation." 1/5
2
221
So the essence of this story is that if you pay people on the internet to take surveys, somebody somewhere on the planet will figure out how to grift some money for fake results. That, and that university administrators at Harvard are horribly credulous investigators. That, and that it really doesn't take particularly good ideas to get tenure at Harvard. "People who act inauthentically feel the need to purify themselves afterwards" ... you call that _research_?
For those interested in the @francescagino @HarvardHBS case, please read this transcript. It explains in great detail the sources/causes of the data anomalies in her papers that were the basis for Harvard taking away her tenure. Those that have been convinced that Francesca is guilty of academic fraud generally focus all of their attention on the @DataColada analysis. While Data Colada deserves credit for identifying the data anomalies, it did not provide any proof or evidence as to the source of the anomalies, whether they came from Gino, her research associates or otherwise. This transcript will open your eyes to the truth. Please give it a careful read. Thank you. theginocase.info/wp-content/…
1
176
The cold start problem is one of the main issues with RL historically. You can reinforce action trajectories until you see them. The other big problem is credit assignment; not all actions share equally in a reward, and RL rewards all of them anyways.
It's wild to see RL people get mad at LLMs and larping RL purity, like RL is everything you need for AGI and LLMs are just some nonsense. For years, one of the biggest (top 3 at most) problems with current RL algos was the cold start problem. If you start from scratch, you're super limited in what you can actually achieve, and how quickly. A general-purpose desktop agent could *never* be trained with pure RL, and even warm start with imitation would be super finicky. In come LLMs. Monstrosities with tons of general knowledge. The perfect vessels for agent initialization, and finally some path towards practical generalist agents. But I guess training in the same games and simulations for a decade is easier.
2
165
Next time someone tells you something will "definitely" happen, keep in mind they only mean it's got about a 78% chance. Conversely, if someone tells you it's highly unlikely they'll do something, there's still a 17% chance they'll do it. People are weird.
There is no word in English that means someone is 30% sure something will happen
1
225
If this was your grandmother, I'm very sorry for your loss. For every else ... here is the modernized version of "Grandma got run over by a reindeer" ...
Your chances of being attacked by a shopping cart are very low, but never zero. 🛒
1
302
What a weird take. In reality, wealth is goods and services. In a healthy, functioning market system, money primarily enables the exchange of goods and services. You make money by creating goods or services. So if someone makes a lot of money in a capitalist system, it should mean that person created a lot of goods or services. So this whole idea of a fixed pool of value is just kind of silly. You can make more value by providing more services. Nothing stops other people from ALSO generating goods and services and making money EXCEPT for access to base resources, and that isn't as big of a factor as people like to make it out to be in most cases. Mostly, value is created in services more so than in goods, so wealth is really about the value of labor. Marx argued that capital would conspire to replace labor with machines, but that just misplaces the argument, because in a society like ours you can find people to help you purchase most machines if you're not a total social misfit. The right conversation is "who's blocking your ability to create goods and services", but that has only a tangential connection to "capitalism", and it's most certainly not fixed by "socialism", which puts even more constraints on people's ability to create wealth.
The worst part about capitalism is that when someone makes a lot of money, you can’t be happy for them because it means there’s less money for everybody else.
1
3
270
Nice work that trades training for search in latent space in order to generate images.
Preprint of today: Beyer et al., "Highly Compressed Tokenizer Can Generate Without Training" -- github.com/lukaslaobeyer/tok… The latent space of tokenizers already provides a good enough abstraction to work with -- you don't have to use a diffusion model on top to inpaint, etc!
1
193
The brain-as-computer analogy definitely has not been proven, but it hasn't been disproven either. The argument that just because we haven't done something we can't do in the future it is a fallacy. And the argument that we can't create something we don't understand is not a strong claim either.
If the brain is a “computer”, then why haven’t we been able to fully figure out how it functions? We can’t figure out why we get Alzheimer’s, seizures, brain cancer, etc in many cases. The hubris of thinking you can recreate something you can’t understand
2
153
I never knew that arXiv was moderated until just now...
I'm going to explain how profound levels of dissent in physics are driven out of the community. Q: "Why avoid the arXiv? That isn't peer reviewed or even moderated! Anyone can put anything on it!" A: "Unmoderated?? The old P. Ginsparg Los Alamos National Labs server? Who knew!"
102
I recently read through the Three-Body Problem trilogy by Cixin Liu, and I can understand why Obama liked it so much. The tone is pessimistic. The humans are hopeless losers who have trouble doing anything pessimistic, who ultimately suffer an inevitable doom. The humans are so devoted to compassion and protecting every human life that they get pretty much everyone killed by shutting off development, and somehow that's supposed to be viewed as a positive human trait. So, in other words, Cixin Liu and Barack Obama share the same negative outlook on humanity and our future. I can't recommend the book, as it's disjointed and self-contradictory throughout. Major plotlines develop around claims about reality that are somehow ignored shortly thereafter. Like, if the first thing any civilization does when it encounters another civilization is to destroy it without establishing contact, then (a) how is the whole first book about ongoing contact between an alien civilization and earth? and (b) how are there "trade routes" and concerns about interspecies etiquette in the third book? And that's just one example. The second book in the series was pretty good, but overall, not worth the time.
2
75