extract-llms-docs
Active·★ 35·NOASSERTION·Updated 2026-10-20
llm.energy is a web application and MCP server designed to fetch, parse, and organize documentation from websites following llms.txt and install.md standards. It transforms this raw documentation into structured, agent-ready formats for AI assistants, LLMs, and developer tooling.
#Documentation Extraction#AI Agent Tooling#LLM Integration#install.md Generation#Web Scraping#API#TypeScript
01
Features
01Smartly detects llms.txt and install.md standards on websites.
02Transforms documentation into agent-ready formats, including an AGENT-GUIDE.md.
03Supports exporting documentation in Markdown, JSON, YAML, or ZIP formats.
04AI-powered generation of install.md instructions from GitHub repos or documentation URLs.
05Provides an MCP server for integration with AI clients like Claude Desktop and Cursor.
02
Compatibility
TypeScript
Language
Verified via docs
Node.js
Runtime
Verified via docs
Next.js
Web Framework
Verified via docs
MCP Clients
Integration
Verified via docs
03
Quick start
1
$ git clone https://github.com/nirholas/extract-llms-docs.git
2
$ cd extract-llms-docs
3
$ pnpm install
04
Use cases
↳Feed documentation directly to AI coding assistants for enhanced context.
↳Build AI agents with up-to-date documentation for improved context-awareness.
↳Establish documentation pipelines specifically for Retrieval-Augmented Generation (RAG) systems.
↳Automatically aggregate documentation from various sources.
↳Generate llms.txt and install.md instructions for new or existing projects.
05
Alternatives
wet-mcp★ 19
Open-source MCP server for AI agents: web search, content extraction, and library docs -- 5-strategy scraping, runs without API keys.
searxng-mcp★ 25
Private web search MCP: SearXNG metasearch → ML reranking → 4-tier fetch cascade (Firecrawl / Crawl4AI / raw / Wayback) with per-domain learning, Ollama query expansion and synthesis. No cloud APIs.
arxiv-reader-mcp★ 25
Want to search arXiv papers, fetch metadata, and extract full-text PDFs without leaving your editor? This MCP server connects any MCP-compatible client (Claude Code, etc.) directly to arXiv.
crw★ 1.2k
Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scrape, /crawl, /search). 2.3x faster than Tavily, 1.5x faster than Firecrawl in 1K-URL benchmarks. 6 MB RAM, single binary. Self-host or use managed cloud.
Comments
Log in to leave a comment
No comments yet. Be the first!