Web Crawler MCP Server by jitsmaster

MCP Servers Chevron right icon

Web Crawler

Provider

jitsmaster

Classification

community

Est. Downloads

This is our estimate of how many downloads occurred of this server across the MCP ecosystem (not specific to any single platform). We use a mix of publicly available data, social signals, and more to feed an algorithm that drives this estimation.

141

Released On

Jan 10, 2025

Popularity Ranking

Our estimate as to where this MCP server implementation ranks on the global leaderboard of usage.

#5,808 (#4,082 this week)

Configurable web crawler that extracts structured content from websites while respecting robots.txt rules and offering customizable settings for depth, delay, and concurrency.

GitHub Repo (1 stars)

Related Servers

FireCrawl

Mendable

Integration with FireCrawl to provide advanced web scraping capabilities for extracting structured data from complex...

Classification

official

Est Downloads (All Time)

725k

Release Date

Dec 6, 2024

Website Downloader

pskill9

Integrates wget-based website downloading capabilities to enable offline archiving and analysis of web content while...

Classification

community

Est Downloads (All Time)

21.3k

Release Date

Dec 24, 2024

Markdown Web Crawl

JMH

Python-based web crawler extracts website content into markdown files, enabling efficient content aggregation and...

Classification

community

Est Downloads (All Time)

424

Release Date

Jan 7, 2025

Web Content Extractor

Brian W. Smith

Extracts and processes web content using TypeScript, Cheerio, and Turndown for tasks like scraping, summarization,...

Classification

community

Est Downloads (All Time)

1.7k

Release Date

Jan 8, 2025

UseScraper

Tane van Wifferen

Integrates with UseScraper API to extract web content in various formats with advanced proxy options and custom data...

Classification

community

Est Downloads (All Time)

565

Release Date

Jan 11, 2025

Deep Web Research

qpd-v

Enables deep web searches and content extraction through automated Google searches, webpage visits, and screenshot...

Classification

community

Est Downloads (All Time)

11.7k

Release Date

Jan 13, 2025

Website Downloader (Windows)

angrysky56

Windows-compatible website downloader for efficient web content retrieval and storage, leveraging asynchronous...

Classification

community

Est Downloads (All Time)

706

Release Date

Jan 17, 2025

Browser Use

Saik0s

Integrates with web browsers to enable automated web scraping, form filling, and interaction with web applications.

Classification

community

Est Downloads (All Time)

122k

Release Date

Jan 18, 2025

Web Search

divslingerx

Integrates with Google Search using Puppeteer to enable web searches and extract structured results for real-time...

Classification

community

Est Downloads (All Time)

—

Release Date

Jan 20, 2025

Firecrawl

Msparihar

Integrates with the Firecrawl API to enable web scraping and intelligent content searching for extracting structured...

Classification

community

Est Downloads (All Time)

282

Release Date

Feb 8, 2025

Simple Website Fetcher

Chris Boden

Fetches website content using Python and MCP, enabling web scraping and content analysis tasks

Classification

community

Est Downloads (All Time)

424

Release Date

Feb 12, 2025

Simple Tool (Website Fetcher)

Kirill Markin

Integrates with web servers to fetch and retrieve webpage content for analysis and processing tasks.

Classification

community

Est Downloads (All Time)

282

Release Date

Feb 18, 2025

Serper Search and Scrape

Marco Pesani

Integrates with the Serper API to enable web searches and webpage content extraction, supporting research, content...

Classification

community

Est Downloads (All Time)

34.2k

Release Date

Feb 21, 2025

Firecrawl

Cody De Arkland

Integrates with the Firecrawl API to enable website crawling, content translation, and structured data extraction for...

Classification

community

Est Downloads (All Time)

141

Release Date

Feb 26, 2025

WebSearch Tools

m4s1t4

Integrates with web search APIs and crawling tools to enable internet-wide information retrieval, content extraction,...

Classification

community

Est Downloads (All Time)

141

Release Date

Mar 4, 2025

Crawlab

Crawlab Team

Integrates with Crawlab to enable AI-driven web scraping, task automation, and data extraction workflows through...

Classification

official

Est Downloads (All Time)

848

Release Date

Mar 4, 2025

Scrapezy

Integrates with the Scrapezy API to extract structured data from websites based on user-specified prompts, enabling...

Classification

official

Est Downloads (All Time)

3.5k

Release Date

Mar 5, 2025

Web Scraper & DuckDuckGo Search

Surya Madhav

Provides web scraping and DuckDuckGo search capabilities for retrieving and processing web content as markdown.

Classification

community

Est Downloads (All Time)

141

Release Date

Mar 7, 2025

Hyperbrowser

Enables web browsing capabilities through tools for content extraction, link following, and browser automation with...

Classification

official

Est Downloads (All Time)

30.3k

Release Date

Mar 17, 2025

Web Crawler

orange-fruit01

Enables web crawling capabilities for extracting website content in markdown format, containerized with Docker for...

Classification

community

Est Downloads (All Time)

—

Release Date

Mar 17, 2025

Browser Tools

Chi

Enables web browsing capabilities through tools for content extraction, link following, and browser automation with...

Classification

official

Est Downloads (All Time)

—

Release Date

Mar 17, 2025

Oxylabs Web Scraping

Oxylabs

Integrates with Oxylabs web scraping services to extract, clean, and structure web content for real-time data...

Classification

official

Est Downloads (All Time)

58.2k

Release Date

Mar 18, 2025

DeepWebResearch

Pedro

Enables deep web research through parallel multi-query execution, intelligent content extraction, and screenshot...

Classification

community

Est Downloads (All Time)

141

Release Date

Mar 18, 2025

Web Research

PhialsBasement

Enables AI web browsing through Google search, webpage extraction, and screenshot capture tools with session...

Classification

community

Est Downloads (All Time)

989

Release Date

Mar 18, 2025

Crawl4AI (Web Scraping & Crawling)

Ritvij Kumar Sharma

Integrates web scraping and crawling capabilities with tools for single-page content extraction, metadata retrieval,...

Classification

community

Est Downloads (All Time)

3.7k

Release Date

Mar 6, 2025

FireCrawl

Mendable

Enables web scraping, crawling, and data extraction through the FireCrawl API with configurable parameters for...

Classification

official

Est Downloads (All Time)

—

Release Date

Mar 18, 2025

Web Fetch

Zach Caceres

Provides web content retrieval capabilities through specialized tools for fetching HTML, JSON, text, and Markdown...

Classification

community

Est Downloads (All Time)

282

Release Date

Dec 25, 2024

Web Search

Freddie

Enables web search capabilities by scraping Google search results without API keys, returning structured data with...

Classification

community

Est Downloads (All Time)

565

Release Date

Mar 18, 2025

Browser Automation

Big Sweet Potato Studio

Enables web browsing capabilities through Puppeteer and Chrome, allowing navigation, content extraction, and...

Classification

community

Est Downloads (All Time)

4.7k

Release Date

Mar 19, 2025

Scrapling Fetch

cyberchitta

Enables AI to access text content from websites protected by bot detection mechanisms through three protection levels...

Classification

community

Est Downloads (All Time)

7.7k

Release Date

Mar 20, 2025

Web Content Retriever

Andreas Hornqvist

Lightweight web content retrieval server that enables fetching and accessing website information during conversations...

Classification

community

Est Downloads (All Time)

—

Release Date

Feb 24, 2025

Scrappey

pim97

Bridges AI models with Scrappey's web automation platform, enabling browser session management, HTTP requests, and...

Classification

community

Est Downloads (All Time)

—

Release Date

Mar 21, 2025

Web Content Explorer

wolfyy970

Fetches and explores web content with recursive link-following capabilities, using a dual-strategy approach with...

Classification

community

Est Downloads (All Time)

989

Release Date

Mar 22, 2025

AgentQL

Tiny Fish

Extracts structured data from web pages based on natural language descriptions, converting website content into JSON...

Classification

official

Est Downloads (All Time)

8.1k

Release Date

Mar 15, 2025

Web Content Pick

kilicmu

Extracts structured content from web pages using customizable selectors for crawling, parsing, and analyzing HTML...

Classification

community

Est Downloads (All Time)

5.6k

Release Date

Mar 5, 2025

WebSearch (Google)

Manh La

Provides real-time web search capabilities via a dedicated crawler service with configurable result limits, language...

Classification

community

Est Downloads (All Time)

7.3k

Release Date

Mar 25, 2025

Docs Scraper

felores

Scrapes web-based documentation through multiple crawling strategies like sitemap traversal and menu navigation,...

Classification

community

Est Downloads (All Time)

282

Release Date

Mar 25, 2025

Browser Automation

Jan Spoerer

Selenium-powered browser automation server that enables web interactions like navigation, clicking, form filling, and...

Classification

community

Est Downloads (All Time)

848

Release Date

Mar 17, 2025

Mattermost Fetch

Kakehashi Inc

Web scraping server that fetches and processes web content with robust features like robots.txt compliance, LRU...

Classification

community

Est Downloads (All Time)

424

Release Date

Mar 26, 2025

One Search

zac ma

Provides a unified search and web scraping platform that integrates multiple search providers like SearxNG and...

Classification

community

Est Downloads (All Time)

9.2k

Release Date

Mar 18, 2025

Documentation Crawler

Jason Ruan

Documentation crawler service that fetches and serves technical documentation from specified sources, enabling access...

Classification

community

Est Downloads (All Time)

—

Release Date

Mar 28, 2025

Prysm Web Scraper

Pink Pixel

Provides web scraping capabilities with three specialized tools (scrapeFocused, scrapeBalanced, scrapeDeep) for...

Classification

community

Est Downloads (All Time)

3.8k

Release Date

Apr 4, 2025

Web Scraper

un4ckn0wl3z

Extracts clean text content from web pages using BeautifulSoup and HTTPX with proper error handling for failed...

Classification

community

Est Downloads (All Time)

141

Release Date

Apr 8, 2025

ScrAPI

DevEnterprise Software

Enables web scraping from sites with bot detection, captchas, or geolocation restrictions through residential proxies...

Classification

official

Est Downloads (All Time)

4.7k

Release Date

Apr 8, 2025

Web Scraper

Vishwajeet Dabholkar

Connects Claude Desktop to a local web scraper for retrieving and processing website content in markdown format with...

Classification

community

Est Downloads (All Time)

424

Release Date

Apr 9, 2025

Brave Deep Research

suthio

Combines Brave Search with web scraping to provide deep research capabilities by extracting full content from pages...

Classification

community

Est Downloads (All Time)

1.7k

Release Date

Mar 10, 2025

AI Cursor Scraping Assistant

The Web Scraping Club

Streamlines web scraper development for e-commerce sites by analyzing website structure, detecting anti-bot...

Classification

community

Est Downloads (All Time)

6.1k

Release Date

Apr 13, 2025

Browser Scraping & Search

Raghu6798

Enables web scraping, internet searching, and local file interactions through Playwright, Firecrawl, and Tavily...

Classification

community

Est Downloads (All Time)

282

Release Date

Apr 20, 2025

Crawl4ai Web Crawler

Ken Ye

Provides a bridge to the Crawl4ai web crawling service, enabling extraction of website content with configurable...

Classification

community

Est Downloads (All Time)

565

Release Date

Apr 21, 2025

WebforAI Text Extractor

Yuta Kobayashi

Extracts and processes web page content into clean, readable markdown format for research and information gathering

Classification

community

Est Downloads (All Time)

282

Release Date

May 3, 2025

Website Scraper and Analyzer

Elizabeth Siegle

Enables website analysis and content extraction using Cloudflare Workers, providing tools to scrape, summarize, and...

Classification

community

Est Downloads (All Time)

848

Release Date

May 1, 2025

Crawl4AI RAG

Cole Medin

Integrates web crawling with RAG functionality to enable website content retrieval, storage in vector databases, and...

Classification

community

Est Downloads (All Time)

273k

Release Date

May 4, 2025

Crawl4AI

Bjorn Melin

Provides web scraping and crawling capabilities for extracting content from single pages, conducting multi-page...

Classification

community

Est Downloads (All Time)

—

Release Date

May 5, 2025

Puppeteer Vision Web Scraper

Denis Jannot

Automates web scraping by intelligently handling cookie banners, CAPTCHAs, and paywalls to extract clean markdown...

Classification

community

Est Downloads (All Time)

814

Release Date

May 15, 2025

Web Curl

Rayss

Enables web content retrieval and API interaction with customizable resource blocking, authentication options, and...

Classification

community

Est Downloads (All Time)

1.3k

Release Date

May 24, 2025

Better Fetch

flutterninja9

Fetches and processes web content with nested URL crawling capabilities. Transform any documentation site or web...

Classification

community

Est Downloads (All Time)

4.1k

Release Date

May 28, 2025

Web Scraper

navin4078

Provides web scraping capabilities through specialized tools for extracting text, links, images, tables, headlines,...

Classification

community

Est Downloads (All Time)

706

Release Date

Jun 7, 2025

Read Website Fast

Just Every

Extracts web content and converts it to clean Markdown format using Mozilla Readability for intelligent article...

Classification

community

Est Downloads (All Time)

8.1k

Release Date

Jun 12, 2025

Web Crawler Data Bridge

Ben Caulfield

Advanced search and retrieval for web crawler data. Supports WARC, wget, Katana, SiteOne, and InterroBot crawlers.

Classification

community

Est Downloads (All Time)

18.8k

Release Date

Jun 19, 2025

Pulse Fetch

PulseMCP

Reliable, configurable tool to fetch or extract clean data from publicly available webpages while bypassing anti-bot...

Classification

official

Est Downloads (All Time)

2.4k

Release Date

Jul 1, 2025

Browser Automation

Diamond-IT-615

Enables browser automation through a WebSocket-connected browser extension, providing tools for extracting page...

Classification

community

Est Downloads (All Time)

141

Release Date

Jul 6, 2025

Crawl4AI

Cole Medin

Provides advanced web crawling and RAG capabilities through Crawl4AI with semantic search, knowledge graph...

Classification

community

Est Downloads (All Time)

424

Release Date

Jul 15, 2025

Web Scraper

Laurence Morgan

Provides web scraping capabilities using Chrome's headless browser with automatic HTTP fallback, handling...

Classification

community

Est Downloads (All Time)

282

Release Date

Jul 15, 2025

Crawl4AI

omgwtfwow

Integrates with Crawl4AI to provide web crawling, content extraction, screenshot capture, PDF generation, and batch...

Classification

community

Est Downloads (All Time)

1.3k

Release Date

Jul 31, 2025

Crawl4AI RAG

Jason Guo

Combines web crawling with vector search to enable content extraction, semantic indexing, and retrieval-augmented...

Classification

community

Est Downloads (All Time)

Release Date

Jul 31, 2025

Crawl4AI

Nexus Digital Automations

Integrates Crawl4AI's async web crawler to extract webpage content in HTML or markdown formats, capture screenshots,...

Classification

community

Est Downloads (All Time)

141

Release Date

Aug 1, 2025

Tavily

James Kanyiri

Integrates with Tavily's web intelligence platform to provide real-time web search, URL content extraction,...

Classification

community

Est Downloads (All Time)

141

Release Date

Aug 10, 2025

Spider Web Scraper

Bosegluon2

Provides web scraping capabilities using Puppeteer and Cheerio to search Bing, extract webpage content, and convert...

Classification

community

Est Downloads (All Time)

141

Release Date

Aug 14, 2025

Firecrawl

Supercorp

Integrates with Firecrawl API to provide web scraping and search capabilities, enabling extraction of markdown...

Classification

community

Est Downloads (All Time)

Release Date

Aug 21, 2025

Scrapling

D4Vinci

Provides advanced web scraping capabilities with browser fingerprint impersonation, dynamic content fetching via...

Classification

official

Est Downloads (All Time)

—

Release Date

Oct 13, 2024

AnyCrawl

AnyCrawl Team

Integrates with the AnyCrawl API to provide web scraping and crawling capabilities with configurable depth limits,...

Classification

official

Est Downloads (All Time)

644

Release Date

Sep 15, 2025

WebScout

pyscout

Automates reverse engineering of chat interfaces through browser automation and network traffic analysis, capturing...

Classification

community

Est Downloads (All Time)

Release Date

Oct 3, 2025

Markdown Web Extractor

Vishwajeet Dabholkar

Extracts clean markdown content from web pages using headless Chrome browser, filtering out navigation and...

Classification

community

Est Downloads (All Time)

141

Release Date

Oct 6, 2025

Scraper

cotdp

Extracts content from websites through four core tools for HTML scraping, markdown conversion, text extraction, and...

Classification

community

Est Downloads (All Time)

565

Release Date

Oct 31, 2025

WebClone

Ruslan Magana

Website cloning engine with intelligent crawling, asset downloading, PDF generation, authentication support, and...

Classification

community

Est Downloads (All Time)

141

Release Date

Nov 25, 2025

WebSearch

Undici77

Parallel web search across multiple search engines, social media platforms, and web archives with concurrent...

Classification

community

Est Downloads (All Time)

565

Release Date

Nov 16, 2025

WaterCrawl

Web crawling and scraping with search capabilities

Classification

community

Est Downloads (All Time)

989

Release Date

Apr 21, 2025

Crew Risk

DeeppathAI

Evaluates websites for crawling compliance and safety by analyzing robots.txt files, detecting anti-crawling...

Classification

community

Est Downloads (All Time)

424

Release Date

Nov 3, 2025