wet-mcp
Active·★ 19·Apache-2.0·Updated 2026-10-21
WET - Web Extended Toolkit MCP Server provides AI agents with comprehensive capabilities for web search, content extraction, and library documentation retrieval. It features a 5-strategy scraping chain, integrated metasearch, and zero-configuration local models for embeddings and reranking, operating without requiring API keys.
#Web Search#Content Extraction#AI Agent Tooling#Web Scraping#Documentation Search#Knowledge Retrieval#LLM Integration
01
Features
01Web Search: Embedded SearXNG metasearch (Google, Bing, DuckDuckGo, Brave) with query expansion, TTL cache, and standardized citation format. Optional cloud search backends.
02Content Extract: 5-strategy escalation chain for URL content extraction, providing smart chunks structured output (clean text, markdown, JSON-LD, code blocks, metadata), batch processing, deep crawling, and local file conversion.
03Library Docs: Auto-discover and index documentation with FTS5 hybrid search, HyDE-enhanced retrieval, and version-specific docs for AI agents.
04Zero Config: Built-in local Qwen3 embedding + reranking, no API keys needed. Optional cloud providers can be selected per task for higher-quality results.
02
Compatibility
Python
Runtime
Verified via docs
Docker
Deployment
Verified via docs
Cloudflare Workers
Cloud Deployment
Verified via docs
MCP
Protocol
Verified via docs
03
Quick start
1
$ docker run -d --name wet-mcp-http -p 8084:8080 -v wet-data:/data -e MCP_TRANSPORT=http -e PUBLIC_URL=https://wet.example.com n24q02m/wet-mcp:latest
04
Use cases
↳AI Agent Research: Enabling AI agents to perform comprehensive web research and academic literature reviews to gather information for complex tasks.
↳Structured Data Extraction: Automatically extracting structured and unstructured data from web pages, including clean text, markdown, JSON-LD, and code blocks, for further processing by agents.
↳Developer Documentation Access: Providing AI agents with version-aware and topic-specific access to software library documentation for code generation, debugging, or learning.
↳Offline Content Processing: Converting local files (PDF, DOCX, HTML, etc.) to Markdown for AI agent consumption, enabling analysis of local knowledge bases.
05
Alternatives
olostep-mcp-server★ 24
MCP server for Olostep — the web scraping, crawling, and search infrastructure used by top AI companies. Gives any MCP-compatible AI agent the ability to scrape, crawl, batch-extract, and search the web in real time.
extract-llms-docs★ 35
Extract documentation for AI agents from any site with llms.txt support. Features MCP server, REST API, batch processing, and multiple export formats.
wet★ 19
Open-source MCP server for AI agents: web search, content extraction, and library docs -- 5-strategy scraping, runs without API keys.
search-mcp★ 26
Free web search and page fetching for agents. Hosted MCP, no API key required.
gigaxity-deep-research★ 62
Open-source deep research MCP. Qwen3-30B-A3B-Thinking via OpenRouter, cited web synthesis for Claude Code, Codex, Cursor, Hermes and any MCP-compatible agent.
searxng-mcp★ 25
Private web search MCP: SearXNG metasearch → ML reranking → 4-tier fetch cascade (Firecrawl / Crawl4AI / raw / Wayback) with per-domain learning, Ollama query expansion and synthesis. No cloud APIs.
Comments
Log in to leave a comment
No comments yet. Be the first!