Web Scraping & Crawling Tools

Open source web scraping and crawling tools you can self-host. Compare Scrapy, Puppeteer, and Playwright alternatives by throughput and license. Updated Sept 2026.

firecrawl

52,2514,515

Firecrawl is an API that turns any URL into clean, LLM‑ready data. It crawls sites (no sitemap required), scrapes dynamic pages, and outputs markdown, HTML, structured JSON, screenshots, links, and metadata.

8/26/20254515 contributors