# https://www.robotstxt.org/robotstxt.html # CareerGenius: search engines and AI assistants are welcome on the public pages. # Signed-in app pages (editor, Job Hub, account) send "X-Robots-Tag: noindex" (see _headers), so they stay crawlable # for that header but are not indexed. # A summary for AI assistants and agents: https://careergenius.org/llms.txt (full version: /llms-full.txt) User-agent: Googlebot User-agent: Bingbot User-agent: Applebot User-agent: GPTBot User-agent: OAI-SearchBot User-agent: ChatGPT-User User-agent: ClaudeBot User-agent: Claude-SearchBot User-agent: Claude-User User-agent: PerplexityBot User-agent: Perplexity-User Allow: / User-agent: * Disallow: Sitemap: https://careergenius.org/sitemap.xml