Aerospace engineer turned entrepreneur. I tweet about building and growing https://xitter.cf/t.co/xpqMR2vvdh ( @curiosity_ai )
Munich, Bavaria
Joined October 2017
- Tweets7.5K
- Following4K
- Followers577
- Likes51.9K
Rafael Oliveira retweeted
RAG vs. Long-Context LLMs
I have yet to see a convincing paper or technical blog showing that long-context LLMs can or will replace RAG.
So far I've seen specific long-context applications where long-context LLMs thrive and current retrieval benchmarks are not convincing.
This new paper reports that longer-context LLMs suffer from diminished focus on relevant information, which is one of the primary issues that a RAG system addresses (i.e., uses more relevant information).
They propose an order-preserving RAG mechanism that improves performance on long-context question answering.
It's not perfect and in fact as retrieved chunks increases the quality of responses go up and then declines. But there is a sweet spot where it can achieve better quality with a lot fewer tokens than long-context LLMs.
I am getting better results for RAG plus the added benefit of efficiency as well. Things could change rapidly but still not counting out RAG for now.
Rafael Oliveira retweeted
had the opposite of the intended effect in the end, teams run by non coding scrum masters and SAFe scaled agile consultants mandating all teams have the same practices.
Rafael Oliveira retweeted
After I published my guide to making slides I got a lot of great feedback & especially loved that others had already used it for their talks 🤩
In part 2, I'm showcasing examples from @t_redactyl and @vboykis and dive deeper into aesthetics & process.
ines.io/blog/beautiful-slide…
Rafael Oliveira retweeted
New open source OCR model just dropped! This one by Microsoft features the best text recognition I've seen in any open model and performs admirably on handwriting.
It also handles a diverse range of vision tasks. You can play with it here: huggingface.co/spaces/gokayg…
ALT A picture of a terminal window showing extracted text next to an image of handwriting with colorful bounding boxes around it. The extracted text pretty closely matches the messily handwritten document.
This quoted post is unavailable.
Rafael Oliveira retweeted
Branding.
In the long run, the only unfair advantage that companies have over their competitors is their brands. In the long run, any advantage in technology disappears, and every company is competing to hire from the same pool of engineers to build things.
If you have a good brand, you can earn high profit margins, even if what you produce is the same as your competitors. People would rather pay for something they can trust rather than go with something free and unreliable.
That's why Apple and nVidia are the most valuable companies in the world.
Take nVidia for example: other companies make hardware that's just as good, including Intel and AMD. If you are going to invest $10 million in building a data center to train models, you don't nVidia.
But because of branding, you choose to pay a premium for nVidia anyway, making the company the most valuable in the world.
AI companies hope to achieve the same thing: make such a solid brand that people will pay them a premium eventually.
Rafael Oliveira retweeted
To qualify as Science a piece of research must be correct and reproducible.
To be correct and reproducible, it must be described in sufficient details in a publication.
To be 'published' (to receive a seal of approval) the publication must be checked for correctness by reviewers.
To be reproduced, the publication must be widely available to the community and sufficiently interesting.
If you do research and don't publish, it's not Science.
Without peer review and reproducibility, chances are your methodology was flawed and you fooled yourself into thinking you did something great.
No one will ever hear about your work.
No one will pick it up and build on top of it.
No one will build new technology and products with it.
Your work will have been in vain.
You'll die bitter and forgotten.
If you never published your research but somehow developed it into a product, you might die rich.
But you'll still be a bit bitter and largely forgotten.
Rafael Oliveira retweeted
Yeah, maybe? I first heard about it from @deepfates quoting this tweet from @allgarbled xitter.cf/allgarbled/status/1786…
Rafael Oliveira retweeted
Slop is the new name for unwanted AI-generated content simonwillison.net/2024/May/8…
Rafael Oliveira retweeted
Introducing ChatQA-1.5, a family of models that surpasses GPT-4-0613 and Command-R-Plus on RAG and conversational QA.
ChatQA-1.5 has two variants:
Llama3-ChatQA-1.5-8B, huggingface.co/nvidia/Llama3…
Llama3-ChatQA-1.5-70B, huggingface.co/nvidia/Llama3…
We also open source our instruction tuning data, offer the ChatRAG Bench for evaluation, and provide a fine-tuned multi-turn QA retriever for the community.
For more details, visit the following links:
huggingface.co/collections/n…
Rafael Oliveira retweeted
In the last 5 years I've seen a good number of solo founders go from $0 to $10k/month.
Some of them are friends, and I know how much they struggled in the beginning.
None of them had huge launches.
They had been stuck below $1k/month for a while.
Their initial metrics were not great.
But contrary to the popular narrative to fail fast and keep churning out new products until you hit big, they stuck to it.
One small marketing win at a time for weeks, months, and years until they got there.
Rafael Oliveira retweeted
The reason I don't order stuff on Alexa is exactly the same as why I wouldn't order it on the Rabbit R1.
Shopping is a terrible use case for products with no screen imo.
Rafael Oliveira retweeted
I had a crazy conversation with a friend who wants to start building startups too, his name is @csonotes and before this he was always a freelancer
The problem he had was he thought every idea he had was not big enough or did not have "product founder fit"
With that he meant he wanted to build a product that fit close to himself as a personality and he gave me as an example of someone with great "product founder fit"
I asked him do you really think I always had a burning passion for interior design and that's why I did Interior AI? Or I have always wanted to enter the model photography industry with Photo AI? Or I'm very very passionate about jobs which is why I have Remote OK?
Nomad List might be the only exception where I acutally do have "product founder fit"
I told him I think it's a highly overrated concept because I had no idea what businesses I would be running when I started and still don't know what I will be running in the next few years, because it's impossible to plan what business you will build. The market changes so fast and most things you do fail. So it's more like surfing waves and the market CHOOSING YOU to do something, not you choosing what to do. You try smell and follow the customer demand, and only then you can have a viable business...for a while, until it ends and the demand goes elsewhere
Elon Musk started by digitizing the Yellow Pages phone book as a startup, now he's building rockets to Mars, sure he has great "product founder fit" now but that's a bit of a big thing to ask for when you're just starting out as a entrepreneur doing your first thing.
Regarding "nothing being big enough for him" (lol btw), you almost always start small, but small things can grow big (lol): Facebook started as an online year book and now it's a bleeding edge VR tech consortium of social media apps. Only if you have millions/billions to burn like again Elon did later in his career, you can start building things more fundamentally from principles where you want humanity to go. Even then it's crazy hard to get that demand, see how Bezos' space startup is doing compared (and he also has the billions to burn!), it's just hard (lollllll).
So yeah IMHO:
1) ignore this "omg it's not big enough" idea, just ship a small thing,
2) if customer demand, or no customer demand, your ship is now in the ocean and you'll start sailing and see where you end up, things start happening cause you took action and made something (and hopefully launched it, talked about it),
3) "product founder fit" is mostly a myth, nobody does exactly what they want to do, and you can't just CONTROL the market like a god to make exactly the business YOU want, you can at maximum steer the ship a little bit, but the waves/wind (aka the market) are 100000x more powerful than you and they'll bring you somewhere else you don't expect to end up!
⛵️ Enjoy sailing the stormy oceans of entrepreneurship!
