devshero/shadowcrawl**Pure Rust MCP Server** ShadowCrawl is a high-performance, Zero-Docker MCP server written in Rust. It serves as a 100% private, sovereign alternative to Firecrawl, Jina Reader, and Tavily. Unlike other scrapers, ShadowCrawl v2.3.0 runs as a single standalone binary with native Chromium control (CDP) and a Human-In-The-Loop (HITL) fallback system, ensuring you can bypass 99.9% of bot protections (Cloudflare, DataDome) without complex infrastructure. Why AI Agents love ShadowCrawl: - Zero-Docker & Zero-Config: No Redis, No Qdrant, No SearXNG. Single binary setup. - God-Tier Anti-Bot Bypass: Native Chromiumoxide (CDP) with JS-level stealth injection and HITL (Human-In-The-Loop) fallback for solving CAPTCHAs. - Internal Meta-Search: Parallel search across Google, Bing, DuckDuckGo, and Brave without external API keys. - Smart Ad-Blocking: Built-in high-speed aho-corasick engine to strip ads and trackers before extraction. - Semantic Memory: Embedded LanceDB for 100% local, private research history recall. - AI-Optimized Markdown: Delivers ultra-clean content stripped of "Buddhist Era" dates and web noise. Tools list: search_web — federated search (No API Key needed) search_structured — search + top result scraping scrape_url — single URL extraction scrape_batch — multi-URL parallel scraping crawl_website — bounded recursive crawling extract_structured — schema-driven extraction research_history — semantic recall from prior runs proxy_manager — proxy list/status/switch/test/grab operations non_robot_search — [NEW] The "Nuclear Option" for Boss-level anti-bots (LinkedIn/Cloudflare) with HITL.