So what makes Scrapling MCP better than Playwright MCP and other known free MCPs that automate browsers or scrape the web? Scrapling MCP in v0.5 can achieve the same browser automation as others, but: - It provides Chainable browser actions. Unlike any other MCP, the agent makes one call to the MCP to take an action in the browser, then thinks, then makes another call for the following action, and so on. In Scrapling, the agent can send a sequence of actions to run in the browser without repeating calls, saving tokens and letting the agent plan ahead. For example, a website that shows a game to test whether you are human, based on your quick reactions, can be solved this way. The actions cover all waiting conditions, so a sequence can include clicking a button, waiting 5 seconds, ensuring the network is stable, typing in an input field, pressing Enter, waiting for a specific selector to appear, waiting for all JavaScript to execute, and then finishing. Imagine Scrapling doing all of that for the Agent instead of the agent sending the calls one by one! - The MCP can see and interact with closed shadow-DOM elements like CAPTCHAs. I don't think this needs explaining, but it means you can rest assured the agent can interact with anything the website throws at you. Previously, you had to give your agent access to computer-use add-ons so it could see the screen and interact with elements like this through coordinates; now it can do it directly through the browser. - There is a lot of focus on saving tokens! Here, we block ads; let you select elements instead of the whole page for all operations; let you save browser state and repeat it later; and protect against prompt injection better than most tools. For example, we delete all hidden elements and zero-width elements, so we save more tokens while protecting you more! - Stuff we already had from before: * We provide a Cloudflare solver that doesn't require AI. * You can do requests through Scrapling without browsers that fake TLS fingerprints, so you only use browsers when you have to (and with a lot of control). * Control over what reaches the AI. Extract selected elements as Markdown, HTML, or text. Search page snapshots or read captured API responses. This can reduce unnecessary output. Other than that, you will find the things that you already see in other tools, like moving and clicking the mouse, using the keyboard and typing slowly, interacting with dropboxes and similar elements, doing drag and drop with the mouse, executing JavaScript on the page, interacting with dialog popups, seeing the requests that are happening in the background and the parts that the Agent is interested in each request, and full control over the type of content returned to the agent to be HTML/Markdown/Text content and so on. Now tell me, have you tried this update yet, and what do you think? 🔥

Check out the video on YouTube: youtube.com/watch?v=NHwWEiTO… Full release notes: github.com/D4Vinci/Scrapling… Full migration guide on moving to v0.5 translated to 10 languges: scrapling.readthedocs.io/en/…

Last night, someone did 12 PRs within 10 minutes to Scrapling. First, I blocked him, closed all PRs without reading them, and labeled them all as "AI-Slop". This is not the first time this has happened, and you can't convince me that PRs spanned this quickly were reviewed. From what I have seen so far, they don't. Now don't get me wrong, I use Fable and Astra a lot to help me manage Scrapling stuff and the development process as a solo maintainer, even with all the help I get from our awesome contributors. But the question is: how long will we have to deal with those low-quality AI-slop bombs that waste every project maintainer's time? I see this in nearly every open-source project, and most of it comes from accounts that look like that dude's: hundreds of PRs a month across many known projects and languages, and they call themselves "AI-forward Engineer" or something similar! Is that the future now, and do we just have to accept it? Low-quality PRs everywhere, as long as you ship fast? Will @github ever provide an option to block this? I guess we will never know!

Today, Scrapling surpassed the Python language repo in terms of number of stars on GitHub haha Now it's ranked 144 on GitHub among all repos by number of stars 🎉

Today, Scrapling has surpassed Scrapy on Github, which made Scrapling the 2nd top Python repo in Web Scraping after Crawl4ai in terms of Github stars 🎉 That comes after staying at the top of GitHub's trends for 4 consecutive days and making another million downloads in exactly 34 days! Thanks to everyone who helped us come so far ❤️‍🔥

Thanks @OpenAI for this gift! 🎉 We are going to hugely benefit from it in @Scrapling_dev

🚨 Scrapling v0.4.8 is out with a new insane update🕷️ New spider templates: CrawlSpider, SitemapSpider, and a standalone LinkExtractor. Stop rewriting "follow links matching X" boilerplate in every parse(). Smarter adaptive default + a request fingerprint fix too. Link below 👇

🚨 Scrapling v0.4.6 is out! What's new: - Built-in ad blocking for browser fetchers (~3,500 tracker domains blocked at the network level) - DNS-over-HTTPS support to prevent DNS leaks when using proxies - New page_setup callback for registering event listeners before page navigation - Several bug fixes from the community Also, now the MCP server and the CLI commands with the `--ai-targeted` option would block ads by default to even save more tokens and time than what we already do :D Let me know what you think. Full release notes below 👇

It's only available in the latest version (v0.4.5) and above. Also, I don't know any other library that has this feature built in 👀

🕷️ Scrapling v0.4.5 is out! New Spider Development Mode -- cache responses on the first run, replay them on every subsequent run. Iterate on your `parse()` logic without re-hitting target servers a single time. Just set `development_mode = True` ✨ Here's the full release notes: github.com/D4Vinci/Scrapling… Let me know what you think!

🚨 Scrapling v0.4.4 is now here This update fixes important bugs in proxy rotation and sessions, and, most importantly, adds a new feature to make the spiders comply with the website's robots.txt file. Scrapy lets you toggle robots.txt in a settings file and manually set your own download delay. Scrapling keeps it simple -- one flag on the spider class and it handles the rest. It pre-fetches robots.txt for all your start domains concurrently, respects Disallow, Crawl-delay, and Request-rate automatically, and adjusts delays per domain without touching your concurrency settings. Here's the full release notes: github.com/D4Vinci/Scrapling… Let me know what you think!

Now Claude will tweet those tips and tricks tweets with generated images like below :)

I'm not a very social person, so I needed something to keep Scrapling's account active. So I used Claude's Cowork, Opus, Scrapling Agent Skill, and Scheduled tasks to get tweets like this. Now you will get daily, a bit deep tips and tricks on some stuff you probably didn't know you could do with Scrapling 🔥 Now I need to see if I can improve this. Probably through photos, what do you think?

🚨 Scrapling v0.4.3 is here This update introduces a ton of changes, including 3 new MCP tools to persist browser sessions across tools, a new option to capture background requests during page fetch to easily scrape SPAs, a new sanitizer to protect against common prompt-injection attacks, and more. Here's the full release notes: github.com/D4Vinci/Scrapling… Let me know what you think! We are so back 🔥 Expect the next update soon 🚀

🚨 Scrapling v0.4.2 is here A new maintenance update with important changes 👉Bug fixes - The function get_all_text() now captures tail text nodes. This will make the MCP server and commands see text that was missed before. - Referer now returns a bare Google url instead of a Google search URL. The previous logic was incorrect and may have produced a fingerprinting signal. - Fixed an issue with extra flags concatenation in all browsers. - Fixed a type hints issue with Python versions below 3.12 that caused it to crash. 👉Other changes - Updated the Agent Skill on the repo and on Clawhub. - Updated all browsers and Playwright versions to the latest. - Added a French translation to the main README file. Full release notes here: github.com/D4Vinci/Scrapling… Let me know what you think

Congratulations, everyone, we have published an Agent skill for Scrapling 🎉 This skill should be readable by @openclaw, Claude Code, and other agentic tools. It encapsulates almost all of the documentation website's content in Markdown, so the agent doesn't have to guess anything. It can be used to answer almost 90% of any questions you would have about scrapling. We tested it on OpenClaw and Claude Code. If you encounter any issues, please open a ticket or use our Discord server. It's uploaded to the GitHub repo and to ClawHub right now. Check the links below 👇 ▶️Also, thanks to everyone who tried to make skills for scrapling and uploaded them to ClawHub, but almost all of them have some inaccuracies or issues in some parts. Now the official skill is here, please use it instead. Let me know your thoughts 🔥

Thanks, @AnthropicAI, for this Gift, and @lydiahallie for helping with the bug I faced. Scrapling made +16.5k GitHub stars in the last two weeks, and this will help me go above and beyond! Great things are coming ahead!

OpenClaw users can now calm down a bit🧘‍♂️ I'm testing the OpenClaw skill for Scrapling and will officially publish it soon. Also, expect it to be added to Clawhub 🦀 After that, expect the Claude Code Plugin/Skill soon.

🚨 Scrapling v0.4.1 is here I have never imagined this version of Scrapling to do so well like that, and all this feedback from the community 🙏 Yesterday, Scrapling was #1 on GitHub's trending list across all programming languages, and this update is my way of saying thanks! Here's what to expect with this update: - Cloudflare solving is now much more efficient and nearly twice as fast. - The stealth mode of the browser is now better and faster than before. - Improved the MCP schema so it's now accepted by strict tools like Open Code and VS Code Copilot without issues. - Improved the MCP server tokens consumption by a large margin. - Scrapling's MCP server is now registered on the MCP registry. - Added a new code snippet to show how to install the browsers deps through code instead of using the commandline to allow easier automation. and more. Check out the full details here: github.com/D4Vinci/Scrapling… So, what do you think about this update?