Enables AI to cite your content in conversations.
Recommended action: Allow access and monitor which pages get cited.
Category
AI Assistant
Primary use case
AI-powered conversations
Trust level
Generally safe
Trust Levels
- Trusted
- Generally safe
- Review recommended
- Caution advised
Trust levels are an indication based on category, operator, and robots.txt compliance. Always review bot activity for your specific situation.
Learn how we assess trustrobots.txt
Respected
kagi-fetcher Traffic (Last 90 Days)
Not enough network data yet.
Track this bot on your siteWhat is kagi-fetcher?
kagi-fetcher fetches web content for Kagi AI's suite of tools including Assistant, Research, and other knowledge discovery features to answer user queries.
What kagi-fetcher means for your site
kagi-fetcher fetches your pages in real-time when someone asks Kagi a question your content can answer. Your content may be quoted, summarized, or linked in the AI's response. This is a new form of visibility: your pages help answer real questions, but the traffic often stays within the AI platform. Pages with clear structure, direct answers, and strong branding are more likely to be cited and clicked.
What should you do?
- Allow kagi-fetcher access to your content
- Ensure important pages have clear, structured answers near the top
- Add branded CTAs so users recognize your site when cited
- Monitor which pages get cited most in BotSights
- Compare citation volume with click-through to identify optimization opportunities
See kagi-fetcher on your own site
BotSights tracks every kagi-fetcher visit in real time, including which pages it crawls, how often, and from where.
How to identify kagi-fetcher
kagi-fetcher identifies itself with the user-agent pattern "kagi-fetcher". It respects robots.txt. Each visit corresponds to a real user query, not systematic crawling.
kagi-fetcherHow to block kagi-fetcher
Three robots.txt options below. Pick the one that matches your goal. Each snippet lists every known kagi-fetcher user-agent pattern so the rules apply regardless of which one the bot announces.
Edit robots.txt with care
A single misplaced line can de-index your entire site. Common mistake: pasting User-agent: * followed by Disallow: / blocks every bot, not just kagi-fetcher, including Googlebot. Always paste the snippet between existing rules (not over them), keep the User-agent line scoped to kagi-fetcher's patterns, and verify with Google's robots.txt tester before deploying. If you are not sure, ask a developer first.
Option 1: Block all access
Tells kagi-fetcher not to crawl any URL on your site. Use this when you want the bot completely off your content.
User-agent: kagi-fetcher
Disallow: /Option 2: Block specific paths only
Keep public content crawlable but exclude sensitive or non-public sections. Add one Disallow: line per path. Replace the example paths with your own.
User-agent: kagi-fetcher
Disallow: /admin/
Disallow: /private/
Disallow: /checkout/Option 3: Slow down with a crawl delay
Crawl-delay is a voluntary directive that asks the bot to wait the given number of seconds between requests. Useful when kagi-fetcher is hammering your origin and slowing the site down for real visitors, but you do not want to block it outright. The value is in seconds, so 10 means at most one request every ten seconds. Not all bots honour this directive (Googlebot ignores it; Bingbot, Yandex, and many AI crawlers do respect it).
User-agent: kagi-fetcher
Crawl-delay: 10Frequently Asked Questions
What is the User-Agent for kagi-fetcher?
kagi-fetcher identifies itself with the User-Agent string "kagi-fetcher". You can use this in robots.txt rules or server-side filters.
Should I block kagi-fetcher?
Blocking kagi-fetcher prevents this AI from citing your content in real-time user answers. Unless you have a specific reason to restrict access, allowing it increases your AI visibility and potential referral traffic.
Is kagi-fetcher the same as a training crawler?
No. Training crawlers (like GPTBot or ClaudeBot) download content to improve AI models. kagi-fetcher fetches pages in real-time to answer specific user questions and cites sources back. Blocking a training crawler does not block kagi-fetcher.
Does kagi-fetcher send traffic to my site?
Sometimes. When kagi-fetcher cites your content, the AI response often includes a link. Whether users click depends on how prominently your content is presented and how your title and meta description are written. BotSights tracks both citations and click-through rates.
Does kagi-fetcher respect llms.txt?
llms.txt is an emerging standard for guiding AI assistants. Kagi have not yet committed to honoring it formally. kagi-fetcher does respect robots.txt, which is the established control mechanism today.
How can I verify that a request is really kagi-fetcher?
Check the request's User-Agent against "kagi-fetcher", then verify the source IP. Check the operator's docs for IP ranges or reverse-DNS verification. BotSights does this automatically and shows a verified badge per visit.
Can I see exactly which pages kagi-fetcher cites?
Yes. BotSights tracks every visit from kagi-fetcher per page, plus the click-through rate from real users who follow citations back to your site. This is page-level visibility you cannot get from server logs alone.
See which pages AI assistants access and cite
Track AI assistant visits, monitor page-level citation activity, and find which content drives the most AI visibility.
- AI citation tracking per page
- See which AI platforms use your content
- Find high-opportunity pages for AI visibility
Free plan available. No credit card required. Setup in 2 minutes.