# robots.txt — generated by tvo-portal-sdk # All crawlers — including AI/LLM crawlers — may index public content. # Transactional funnels and private paths are blocked for everyone. User-agent: * Allow: / Disallow: /apply Disallow: /dubai-express-visa/add-applicant Disallow: /dubai-express-visa/edit Disallow: /add-applicant Disallow: /edit-application Disallow: /checkout Disallow: /payment-successful Disallow: /payment-failed Disallow: /payment-pending Disallow: /actuator/ Disallow: /error # AI / LLM crawlers — explicitly welcomed (SEO-CLAUDE.md §22). # A bot with its own group ignores the * group, so rules repeat. User-agent: GPTBot User-agent: OAI-SearchBot User-agent: ChatGPT-User User-agent: ClaudeBot User-agent: Claude-User User-agent: Claude-SearchBot User-agent: PerplexityBot User-agent: Perplexity-User User-agent: Google-Extended User-agent: Applebot-Extended User-agent: Meta-ExternalAgent User-agent: Meta-ExternalFetcher User-agent: Amazonbot User-agent: Bytespider User-agent: CCBot User-agent: cohere-ai User-agent: MistralAI-User User-agent: DuckAssistBot Allow: / Disallow: /apply Disallow: /dubai-express-visa/add-applicant Disallow: /dubai-express-visa/edit Disallow: /add-applicant Disallow: /edit-application Disallow: /checkout Disallow: /payment-successful Disallow: /payment-failed Disallow: /payment-pending Disallow: /actuator/ Disallow: /error Sitemap: https://dubaivisitorvisa.com/sitemap.xml