Cradle
★ 2.6k·MIT·Updated 2024-11-07
Cradle is a framework that enables foundation models to control computers using human-like interfaces, taking screenshots as input and generating keyboard/mouse actions as output. It supports controlling various games and software, allowing AI agents to perform complex tasks across different applications.
#AI Agents#General Computer Control#LLM#Computer Vision#Human-Computer Interaction#Image Generation
01
Features
01Enables foundation models for general computer control.
02Utilizes a unified human-like interface: screenshots as input, keyboard and mouse as output.
03Supports diverse applications including popular games (e.g., RDR2, Stardew Valley) and productivity software (e.g., Chrome, Outlook).
04Provides a modular framework designed for easy adaptation and migration to new game and software environments.
05Integrates with various Large Language Model (LLM) APIs, including OpenAI, Azure OpenAI, and Claude.
02
Compatibility
OpenAI
API Supported
Verified via docs
Azure OpenAI
API Supported
Verified via docs
Anthropic Claude
API Supported
Verified via docs
AWS Restful API for Claude
API Supported
Verified via docs
VS Code
IDE Supported
Verified via docs
PyCharm
IDE Supported
Verified via docs
03
Quick start
1
$ pip install -r requirements.txt
04
Use cases
↳Automating gameplay in complex video games.
↳Performing in-game management and progression tasks.
↳Controlling and interacting with various desktop applications for productivity or creative tasks.
05
Alternatives
groundingLMM★ 969
[CVPR 2024 🔥] Grounding Large Multimodal Model (GLaMM), the first-of-its-kind model capable of generating natural language responses that are seamlessly integrated with object segmentation masks.
ai-doc-gen★ 768
AI-powered multi-agent system that automatically analyzes codebases and generates comprehensive documentation. Features GitLab integration, concurrent processing, and multiple LLM support for better code understanding and developer onboarding.
kreuzberg★ 9.4k
A polyglot document intelligence framework with a Rust core. Extract text, metadata, and structured information from PDFs, Office documents, images, and 50+ formats. Available for Rust, Python, Ruby, Java, Go, PHP, Elixir, C#, TypeScript (Node/Bun/Wasm/Deno)- or use via CLI, REST API, or MCP server.
video-search-and-summarization★ 1.9k
NVIDIA AI Blueprint for video search and summarization (VSS) is a GPU-accelerated reference architecture for building video analytics agents with real-time verified alerts, visual Q&A, and automated reporting. The VSS Blueprint uses vision language models (VLMs) such as NVIDIA Cosmos, LLMs such as NVIDIA Nemotron, RAG, and NVIDIA NIMs.
Comments
Log in to leave a comment
No comments yet. Be the first!