@diffbot

Never write another web scraper. Diffbot structures information from the web, so you don't have to.

The Future
Joined September 2009
Diffbot πŸ€– retweeted
The web isn't a database. @diffbot makes it one. 10B+ entities and 1T facts extracted from 60B+ pages, rebuilt every 4-5 days. DuckDuckGo, Snapchat, and Dow Jones run on it. Massive powers the proxy infra behind their continuous crawl.
1
1
5
290
Diffbot πŸ€– retweeted
Ever wondered what your white name should have been? Introducing: whatismywhitename.com Upload a picture of you, and let the puppy guess your name! Let's test out nominative determinism 🫑 (Immigrants who named themselves will correlate more highly. Give us feedback plz) Our thanks to: - @modal for their generous credits toward training this meme model - @diffbot for the clean, diverse dataset! - @leannch86920 for the training research! - Everyone NOT named David (biggest & noisiest dataset ever)
7
8
62
7,882
State of E-commerce Data Providers - Q4 2025 E-commerce runs on constant measurement: prices, promos, availability, seller changes, and "what the shelf actually looks like" across retailers and marketplaces. The challenge is stable collection at scale, retries when sites break, anti-bot evasion, clean geo signals, and then turning messy HTML into usable structured data. In preparation for the holiday season, we mapped the landscape of e-commerce data providers: Competitive intel + digital shelf: @dataweavein, @Price2Spy, @bigdataNODE, @Profitero, @WiserInc Marketplace intelligence + data: @junglescout, @H10Software, @datahawkco, @SellerSprite_EN Trade, Supply Chain, Imports / Exports: @Trademo1, @ImportYeti, @datamyne Scraper APIs & Extraction Platforms: @zytedata, @diffbot, @Stratalis, (AutoScraping handle?), @serpapi Managed Data Extraction & Services: @groupBWT, @Data_Ox, @epctex, @MrScraper_ Retail Media & Ad Platforms: @Pacvue, @PerpetuaLabs, @Teikametrics Network & runtime infra for e-com scraping: @playwrightweb, Puppeteer, @browserless
1
464
YouTube, TikTok, Mastodon, & Threads are mostly there but need optimizing. Diffbot goes incredibly far with articles & that’s also moving along well. Reddit & Bluesky are readily available but I haven’t spent the time. X is finished by the endpoint gets rate limited 😞
I am in love with scraping using Shortcuts. I have about 8 sets of shortcuts for everyday social media sites that I'm developing in tandem. I'll be releasing them as I finish them – thread starts here.
1
2
8
2,999
Not Diffbot!
BREAKING: The Internet Massive outage being reported across platforms including Spotify, Google Cloud, AWS, Cloudflare, Claude, YouTube, Gmail, and many, many, more
1
4
981
A datacenter story...
1
4
837
Diffbot πŸ€– retweeted
San Diego developers, join us and our technical partners @neo4j, @Intuit, Eyepop.ai, @Replit , and @diffbot at our HackNight next week!
We're excited to join @neo4j , @Intuit, EyePop.ai, @Replit and @diffbot as technical partners for the upcoming Startup San Diego - FirstWave Innovator HackNight happening Wednesday, February 19th at the Intuit San Diego Campus. This is going to be an epic night where 100+ developers will come together to create innovative solutions for five select startups. πŸš€Β Join as a developer for free or get tickets: lu.ma/4brcg3lz
1
4
7
2,009
Diffbot πŸ€– retweeted
One of the most fun parts of #aitoolshacknight last week was seeing @davidpomerenke win best demo using @Get_Writer Framework! In just a couple hours, he used @diffbot to scrape AI safety papers & built a web app in Writer Framework to visualize the data. Way to go David! πŸŽ‰
1
4
10
932
Diffbot πŸ€– retweeted
Hack Night @github is on September 10th and we've got some incredible speakers and awesome technology companies (@weaviate_io @neo4j @diffbot @goteleport and @Get_Writer!) bringing some of the hottest LLM and RAG tech to the industry. Don't miss out! lu.ma/ozt7jtq5
Join me on Sept 10th at SF GitHub HQ. I'll be presenting a lighting talk on how I use @goteleport to access my local LLM home lab powered by my banana sized GPU. lu.ma/ozt7jtq5
5
14
1,731
Diffbot πŸ€– retweeted
Join me on Sept 10th at SF GitHub HQ. I'll be presenting a lighting talk on how I use @goteleport to access my local LLM home lab powered by my banana sized GPU. lu.ma/ozt7jtq5
2
1
6
2,356