Tag “mcpserver #crawling #agents #mcpservers” — 17

Exa Search

smithery.ai

Fast, intelligent web search and web crawling. Get fresh information about libraries, APIs, and SDKs.

timers studio

smithery.ai

gemini-extension.json n'est pas un mécanisme d'auto-discovery. Les Gemini Extensions sont publiées via Google AI Studio / Extensions Gallery, pas via crawl du .well-known/. Ton fichier est invisible pour Gemini. 3. ChatGPT ai-plugin.json correspond à la spec OpenAI Plugins, dépréciée en 2024. Remplacée par les GPTs (publiés via GPT Store) et les Actions (configurées dans un GPT custom). Ton fichier est mort techniquement. 4. llms.txt Proposition de standar

Crawlinx

smithery.ai

Crawlinx is a hosted Model Context Protocol server that lets your AI agent audit a website's technical SEO and act on the results. It crawls a site (50 pages by default, up to 200 via max_pages), runs 156 technical-SEO checks, and returns an LLM-ready Markdown report with a prioritized fix plan — small enough to keep in context on a real site. ☁️ Remote server over Streamable HTTP — no local install, no API key, no signup 🔎 156 checks — titles, meta, canonicals, orphan pages

Bright Data

smithery.ai

One (free) MCP for the Web. Easily search, crawl, navigate, and extract websites without getting blocked. Ideal for discovering and retrieving structured insights from any public source - effortlessly and ethically. New features are live - check out what’s new: https://brightdata.com/ai/mcp-server/launch-week

Site Scorer

smithery.ai

Analyze websites to determine how effectively they are structured for automated crawling and data extraction. Receive actionable insights and ready-to-use prompts to fix structural issues and improve content visibility. Track recent reports to monitor improvements in site accessibility and performance.

Nimble MCP Server

smithery.ai

The Nimble MCP Server gives AI agents the ability to search, extract, map, crawl, and structure data from any website in real time. It exposes Nimble’s full web data platform as MCP tools that any compatible AI client can use.

Oxylabs MCP

smithery.ai

Official Oxylabs MCP server - scrape any URL with JavaScript rendering and geo-targeting, parse Google and Amazon results, plus AI Studio tools (AI scraper, crawler, browser agent, search). Runs on Oxylabs infrastructure.

AgentIndex

smithery.ai

Discovery platform for AI agents. Find any AI agent by capability — search 20,000+ indexed agents across GitHub, npm, MCP, and HuggingFace.

ShearQuery — Barber & Beauty Industry Data

shearquery.com

Barber & cosmetology exam pass rates, booth rent with open chairs, and Texas licensee counts

GroundRoute: Web Search for AI Agents across 6 Engines ($10 free credit)

smithery.ai

GroundRoute gives your AI agent web search across six engines (Serper, Brave, Exa, Tavily, Firecrawl, Perplexity) through a single search tool. That one tool covers web, news, and academic search, synthesized answers with citations, and full-page fetch, all selectable with a mode parameter, so you never wire up or pay for six separate APIs. It routes each query to the cheapest engine that meets a quality bar, caches repeats, and fails over automatically. Pricing is gain-sha

crawlforge-mcp

smithery.ai

CrawlForge MCP is a production-ready MCP server that gives AI agents the power to scrape websites, extract structured data, run deep research, bypass anti-bot detection, and process documents. It packages 27 specialized web scraping tools into a single MCP server with credit-based pricing and a free tier.

Crawlie

smithery.ai

**Technical SEO + AI-search audits for agents.** Crawl any website and get a structured report your agent can actually work with — broken links, redirect chains, missing titles/canonicals, duplicate content, structured data, and GEO (generative engine optimization) readiness: is your site answerable by ChatGPT, Claude, and Perplexity? ### What your agent can do - **Audit** a single URL, a list, or crawl an entire site - **Slice saved reports** without re-crawling — top fixes

Gkmex Crane Inventory

smithery.ai

Public, zero-auth, read-only Gkmex used-crane inventory. Search current mobile and crawler cranes or retrieve one listing by ID.

Crawlora

smithery.ai

Crawlora is a hosted MCP server exposing 319 structured web-data tools across search, maps, commerce, social, finance, travel, app stores, and reviews — each returning clean JSON instead of HTML to parse. Streamable HTTP transport with API-key auth (x-api-key / Bearer). No proxies, browsers, or parsers to maintain. Pay only on successful responses — 2,000 free credits per month, no card. Docs and playground: https://crawlora.net/docs

swarm-web-crawller

smithery.ai

A powerful web crawling MCP server built for autonomous AI agent swarms. It provides high-speed, parallel web data extraction, allowing distributed multi-agent systems to seamlessly harvest real-time web intelligence. Scale your swarm’s automated research capabilities with optimized, structured web data delivery built directly for LLM tool use.

Deep Research Server

smithery.ai

Perform comprehensive web research by combining AI-powered search and deep content crawling to gather extensive, up-to-date information on any topic. Aggregate and structure research data into detailed JSON outputs optimized for generating high-quality markdown documentation with LLMs. Customize documentation prompts and output paths to fit your workflow and save research results securely.

veed-crawl

smithery.ai

Veedcrawl lets agents search and understand public social videos across YouTube, TikTok, Instagram, X/Twitter, and Facebook. Use this MCP server to: - Search social videos by topic, keyword, creator, or trend - Fetch video metadata, stats, descriptions, tags, and thumbnails - Get transcripts from public videos - Extract structured insights from videos using a custom prompt and JSON schema - Inspect public TikTok and Instagram creator profiles Good for trend re