Turn websites into clean markdown, JSON, and LLM-ready context.
Joined March 2026
- Tweets91
- Following1
- Followers77
- Likes102
Pinned Tweet
webclaw v0.7.0
Browser
• Local JavaScript rendering with --render auto
• JS rendering in crawls, batches and the MCP server
• Page actions: click, scroll, type before extracting
• Gets past Cloudflare challenges (--solve, 15/16 pages)
• Every local render presents as regular Chrome
• Persistent browser profile between runs
• Block images, fonts or media while rendering
• Capture a page's network responses
• Screenshots, PDF, MHTML and WARC output
Extraction
• --query keeps only what answers a question (92.7% recall in 781 tokens vs 7,022 for the full page)
• CSS selectors and XPath 1.0
• Find elements by text, or every element like an example
• Data tables as JSON
• Chunks for retrieval
• Markdown with numbered link citations
• Hidden elements dropped from content
Crawling
• Worker-pool crawler that sees links, redirects and Crawl-delay
• Respects robots.txt by default
• Crash-safe crawls with streamed JSONL and checkpoints
• Query-guided crawling that stops once the answer looks found
• Site URLs from the Common Crawl index with --map
• Opt-in disk cache
MCP and packaging
• MCP over streamable HTTP with a bearer token, 14 tools
• Python package bundling the CLI
• One-command benchmarks against open-source extractors
Licence is now AGPL-3.0-only.
webclaw.io
two ways to run webclaw:
hosted: 500 free credits/mo, no card
self-host: agpl rust core, forever, your box
cli + rest + 12 mcp tools
same extraction engine
start on free, graduate when the agent is real
webclaw.io
introducing webclaw free
your agents can scrape the web without a card
→ 500 credits / month
→ 1 credit = 1 page (render + antibot + extract)
→ markdown / json / llm-clean text
→ mcp + CLI
no trial. nothing expires.
webclaw.io
webclaw v0.6.22
• Optional hosted model provider
• No local inference probes on managed deployments
• Encoded characters preserved in docs and sitemaps
• Correct MCP pre-init errors
• PDF header boundary fix
• Limits for pathological PDFs
webclaw.io
Repo at:
github.com/0xMassi/webclaw
SerpApi is sponsoring webclaw.
It is the largest single contribution the project has had, and it goes
straight into keeping the core open and the free tier free.
If you pull structured SERP data, they are worth a look: serpapi.com
Thanks for backing independent open source.
New in webclaw:
the /lead endpoint.
One company URL in, an outreach-ready lead out: founders and leadership with their LinkedIn and X, plus summary, tech stack, pricing, and the emails published on the site. Verified only, no guessed contacts.
7,819 startups to try it on, free:
webclaw.io/data/startups
Webclaw retweeted
webclaw crossed 1,800 stars on GitHub this week.
It started because scraping APIs charge you to fight Cloudflare and lose.
Now 1,800 of you think a Rust engine you can read, fork, and self-host is the better deal. Thank you.
Back to shipping.
Webclaw retweeted
webclaw.io mentioned!
If you want to ship faster and better you have only one path to follow and join here: shipper.club
The best builders community in the game
Thanks @orcdev
webclaw for OSS is live.
Build an open-source loader, MCP server, action, or CLI on webclaw and we cover the usage: 100,000 credits a month, free, renewing while the project stays alive.
The trade: a "Powered by webclaw" badge in your README and website if you have one.
No card. A human reads every application within a few days.
webclaw.io/oss
You keep an X tab open just to watch one account post.
That's a cron job with anxiety.
X Monitors runs it for you. Point one at a profile, a keyword, a list, or a tweet's replies.
Filter by likes, language, or keyword and skip the retweets.
Every match hits your Discord, Slack, or webhook about a minute after it posts.
Build one no-code in the dashboard, or call it from Python, TS, or Go.
Live now → webclaw.io
Next step: 2000 ✨
+1,500 stars ⭐
I built webclaw to turn any website into clean markdown an LLM can read. It's now a CLI, an MCP server, a REST API, and SDKs for JS, Python, and Go.
Everyone who starred, filed an issue, or sent a PR got it here. Thank you. Plenty left to build.
github.com/0xMassi/webclaw
Start to earn today.
webclaw.io/affiliate
Refer one person to webclaw. Get paid every month they stay.
20% of every payment, no cap, no expiry. Free to join, paid in PayPal, Wise, or bank transfer.
Start earning: webclaw.io/affiliate
You can use webclaw also to find your possible leads.
I have the entire Product Hunt catalog, from day one to today, enriched with emails and social profiles, and sorted by category. I scraped the whole site with webclaw.io. This list could be useful for anyone building tools to generate B2B leads.
If you want to support us 👇
webclaw.io/sponsor
webclaw.io just landed its 6th sponsor: MangoProxy.
Residential, ISP, datacenter, and mobile proxies across 200+ locations. That's the access layer a scraper leans on when sites start blocking you.
Use code 0XMASSI for 8% off their ISP static proxies:
mangoproxy.com/?utm_source=g…
Claim you api key here -> webclaw.io/pricing
One Reddit search gave me 10 people describing a problem my product solves.
Then ~15 lines ranked them by buying intent, so I reply to the ready ones first. One search call, one extract call.
webclaw.io + the JS SDK. Code below.
