
Tweeting about all things web-scraping. We're also an API that handles headless browsers and rotates proxies for you. @TinySeedFund 2020
Paris
Joined August 2019
- Tweets253
- Following4
- Followers5.8K
- Likes726
Mmm. Nothing like a fresh CAPTCHA first thing in the morning. Rate limits with lunch. Getting your IP banned right before a deadline...
Yeah. Nobody says that.
Empowering thousands of developers to build without the hassle of managing scraping infrastructure.
Now, we’re proud to build as part of Source.ai – a data ecosystem for a world built on intelligence.
Still us. Just a wider horizon. 🔍
Learn more: source.ai/news/scrapingbee-w…
Today, a browser can render anything, and that's why it feels like the safe default. But let's be precise about what we are actually launching.
A browser is not a library you call. We're talking about an entire operating system that you boot.
Because today, a browser brings a lot of stuff, like a process scheduler, its own memory allocator, audio-video codecs, a lot of IPs, and of course, its network stack.
Chromium today is around 35 million lines of code, which is in the same order of magnitude as the Linux kernel.
If you want to take a deep dive into web hydration, watch the full recording of Etienne Ellie's keynote at OxyCon
youtube.com/watch?v=cGqH3aM5…
The most expensive line of code in a scraping task is the one that launches the browser.
Not because it's slow, but because of what it costs you on every single page. And a scraping task is not one page, it's often hundred thousands a day.
What you spend loading one, you spend again every time, until it stops being a technical details, and it becomes how many machines you need to rent.
If you want to take a deep dive into web hydration, watch the full recording of Etienne Ellie's keynote at OxyCon youtube.com/watch?v=cGqH3aM5…
Every scrape is a bet between cost and success. Bid too low and you hit a block; bid too high and you overpay for simple HTML.
At Extract Summit in Austin, our developer Kostas Jakeliūnas delivered a quick lightning talk on how Auto-Mode automates this balance:
• Climbs from 1-credit fetches to stealth proxies only when needed
• Inspects DOM content instead of relying on fake 200 OKs
• Charges only for the tier that delivers data (0 credits on failure)
• Keeps budgets safe with max_cost
What actually happens under the hood during Web Hydration? Our lead engineer, Etienne Ellie broke this down live at OxyCon, and the demo repo is now public
github.com/ScrapingBee/scrap…
If web scraping is core to your work, keep this on your radar. Tomorrow at OxyCon® 2026, our Lead Engineer, Etienne Ellie, presents Web Hydration: Loading Dynamic Content Without a Browser.
Register for free: oxylabs.io/oxycon/etienne-el…
#oxycon
Need to find data analysts in France, software engineers in Portugal working for US firms, or Fortune 500 CTOs?
Skip setting up complex search filters. With our Agentic Employee Search, describe who you need in a plain English prompt and get structured profiles back in seconds.
Docs linked below 👇
scrapingbee.com/documentatio…
From Brazil to Australia, our classic proxies now support 42 countries.
Testing localized content in Tokyo? Tracking prices across European markets? Scraping geo-specific data from Canada? Just pick your country from the dropdown.
The full list of supported countries is right this way scrapingbee.com/documentatio…
Fresh from the hive: Auto-Mode is live.
The problem? Every scraping target behaves differently. One blocks standard proxies. Another needs JS rendering. A third requires premium infrastructure.
Auto-Mode solves this. Add mode=auto and it tests configs automatically, starting simple and escalating only when needed. Stops at first success.
Set max_cost to cap how far it goes. Check sb-auto-cost to see what worked.
Manual mode stays for full control. Auto-Mode is for when you want extraction to just work.
The ScrapingBee CLI now has Interactive Mode (REPL).
Run scrapingbee without arguments and you get a persistent terminal session. Set parameters once with :set and they stick across every command. Type :view to check results without opening files. Watch crawl progress in real time.
You stop arrow-up-editing the same 200-character command five times to test one flag change.
Live in v1.5.0+. Upgrade: uv tool upgrade scrapingbee
Docs: scrapingbee.com/documentatio…
If you're tracking Amazon prices, you probably need more than just the Buy Box winner. Our new Pricing endpoint gives you the full seller stack for any ASIN: who's winning the Buy Box, what everyone else is charging, FBA vs merchant-fulfilled, shipping, Prime eligibility, ratings
Geolocation targeting for Google APIs is live.
Now, you can run searches from exact coordinates instead of country-level targeting. Set latitude, longitude, and radius in meters.
Docs 👉 scrapingbee.com/documentatio…
Your ScrapingBee account just got a direct line to Google Gemini.
You can now send a prompt to /api/v1/gemini, get back structured answers in plain text, markdown, and with citations. Same API key, same infrastructure, now connected to Gemini.
Docs: scrapingbee.com/documentatio…
"Why can't I just pull all the ads Google shows for a keyword?"
You can now.
Our Google API supports search_type=ads and returns every ad position + full copy, sitelinks, extensions
Same endpoint, same cost as a standard search
ScrapingBee just upgraded: 2x to 5x more concurrent requests across all plans.
We improved our infrastructure to deliver faster job completion and more headroom for production-scale workflows. The upgrade is already live.
Hey devs and tech product folks! This is the second and last day of @devworld_conf in Amsterdam!
There's a lot happening on each stage today, but if you're here – make sure to stop by our Booth T007 and chat with our team!
















