Tweeting about all things web-scraping. We're also an API that handles headless browsers and rotates proxies for you. @TinySeedFund 2020

Paris
Joined August 2019
Mmm. Nothing like a fresh CAPTCHA first thing in the morning. Rate limits with lunch. Getting your IP banned right before a deadline... Yeah. Nobody says that.
4
302
Empowering thousands of developers to build without the hassle of managing scraping infrastructure. Now, we’re proud to build as part of Source.ai – a data ecosystem for a world built on intelligence. Still us. Just a wider horizon. 🔍 Learn more: source.ai/news/scrapingbee-w…
1
5
1,316
Today, a browser can render anything, and that's why it feels like the safe default. But let's be precise about what we are actually launching. A browser is not a library you call. We're talking about an entire operating system that you boot. Because today, a browser brings a lot of stuff, like a process scheduler, its own memory allocator, audio-video codecs, a lot of IPs, and of course, its network stack. Chromium today is around 35 million lines of code, which is in the same order of magnitude as the Linux kernel.
2
1
318
The most expensive line of code in a scraping task is the one that launches the browser. Not because it's slow, but because of what it costs you on every single page. And a scraping task is not one page, it's often hundred thousands a day. What you spend loading one, you spend again every time, until it stops being a technical details, and it becomes how many machines you need to rent.
2
2
247
Every scrape is a bet between cost and success. Bid too low and you hit a block; bid too high and you overpay for simple HTML. At Extract Summit in Austin, our developer Kostas Jakeliūnas delivered a quick lightning talk on how Auto-Mode automates this balance: • Climbs from 1-credit fetches to stealth proxies only when needed • Inspects DOM content instead of relying on fake 200 OKs • Charges only for the tier that delivers data (0 credits on failure) • Keeps budgets safe with max_cost
1
182
If web scraping is core to your work, keep this on your radar. Tomorrow at OxyCon® 2026, our Lead Engineer, Etienne Ellie, presents Web Hydration: Loading Dynamic Content Without a Browser. Register for free: oxylabs.io/oxycon/etienne-el… #oxycon
2
537
Need to find data analysts in France, software engineers in Portugal working for US firms, or Fortune 500 CTOs? Skip setting up complex search filters. With our Agentic Employee Search, describe who you need in a plain English prompt and get structured profiles back in seconds. Docs linked below 👇 scrapingbee.com/documentatio…
8
11,972
From Brazil to Australia, our classic proxies now support 42 countries. Testing localized content in Tokyo? Tracking prices across European markets? Scraping geo-specific data from Canada? Just pick your country from the dropdown. The full list of supported countries is right this way scrapingbee.com/documentatio…
4
3,243
Fresh from the hive: Auto-Mode is live. The problem? Every scraping target behaves differently. One blocks standard proxies. Another needs JS rendering. A third requires premium infrastructure. Auto-Mode solves this. Add mode=auto and it tests configs automatically, starting simple and escalating only when needed. Stops at first success. Set max_cost to cap how far it goes. Check sb-auto-cost to see what worked. Manual mode stays for full control. Auto-Mode is for when you want extraction to just work.
1
1
1,892
The ScrapingBee CLI now has Interactive Mode (REPL). Run scrapingbee without arguments and you get a persistent terminal session. Set parameters once with :set and they stick across every command. Type :view to check results without opening files. Watch crawl progress in real time. You stop arrow-up-editing the same 200-character command five times to test one flag change. Live in v1.5.0+. Upgrade: uv tool upgrade scrapingbee Docs: scrapingbee.com/documentatio…
5
1,044
If you're tracking Amazon prices, you probably need more than just the Buy Box winner. Our new Pricing endpoint gives you the full seller stack for any ASIN: who's winning the Buy Box, what everyone else is charging, FBA vs merchant-fulfilled, shipping, Prime eligibility, ratings
1
707
Geolocation targeting for Google APIs is live. Now, you can run searches from exact coordinates instead of country-level targeting. Set latitude, longitude, and radius in meters. Docs 👉 scrapingbee.com/documentatio…
1
525
Your ScrapingBee account just got a direct line to Google Gemini. You can now send a prompt to /api/v1/gemini, get back structured answers in plain text, markdown, and with citations. Same API key, same infrastructure, now connected to Gemini. Docs: scrapingbee.com/documentatio…
1
521
"Why can't I just pull all the ads Google shows for a keyword?" You can now. Our Google API supports search_type=ads and returns every ad position + full copy, sitelinks, extensions Same endpoint, same cost as a standard search
381
ScrapingBee just upgraded: 2x to 5x more concurrent requests across all plans. We improved our infrastructure to deliver faster job completion and more headroom for production-scale workflows. The upgrade is already live.
3
1,084
Hey devs and tech product folks! This is the second and last day of @devworld_conf in Amsterdam! There's a lot happening on each stage today, but if you're here – make sure to stop by our Booth T007 and chat with our team!
1
2
769