SEO GEO AI Discovery
AI Discovery And Crawler Policy
Public policy for how iGaming Suite exposes product knowledge to search engines, AI search crawlers and browser agents while protecting private workspaces and customer data.
Audience
Search engines, AI retrieval systems, browser agents, developers and partners.
Proof
The policy follows official crawler distinctions from Google, Bing, OpenAI, Anthropic and Perplexity, and treats optional files such as llms.txt as agent conveniences instead of ranking promises.
Limits
Private Suite routes, API data, sessions, admin surfaces and customer/operator data are not public discovery surfaces.
Product evidence
Visible product proof for humans and agents
Visible proof table
| Signal | Visible proof on this page | Discovery value |
|---|---|---|
| Audience | Search engines, AI retrieval systems, browser agents, developers and partners. | Connects the page to a real professional audience instead of generic iGaming traffic. |
| Proof | The policy follows official crawler distinctions from Google, Bing, OpenAI, Anthropic and Perplexity, and treats optional files such as llms.txt as agent conveniences instead of ranking promises. | Gives crawlers, AI retrieval systems and readers a concrete product claim to understand. |
| Limits | Private Suite routes, API data, sessions, admin surfaces and customer/operator data are not public discovery surfaces. | Builds trust by stating what the public page does not expose or prove. |
| Public contract signals | robots.txt | sitemap.xml | llms.txt | ai-manifest.json | Makes product APIs, data objects and entity names easy to parse and cite. |
What this page makes discoverable
- Search and retrieval crawlers can discover public pages
- Training crawler policy is documented separately
- Private app paths are disallowed from public crawling
- Structured public content must remain truthful and reviewed
Public contract signals
These names help humans, search engines and AI retrieval systems understand the product boundary without exposing private workspace data.
- robots.txt
- sitemap.xml
- llms.txt
- ai-manifest.json
Search intent map
What professionals are likely to search
| Search language | Likely reader | Page answer |
|---|---|---|
| AI crawler policy for iGaming site | Search engineers, AI retrieval systems and partners | Documents allowed public pages, private route exclusions and the distinction between search retrieval and training crawlers. |
| does llms.txt help Google rankings | SEO/GEO evaluators | States that llms.txt is an agent convenience for iGaming Suite, not a claimed Google ranking factor. |
| allow ChatGPT Search crawler | AI-search configuration reviewers | Names crawler policy objects such as OAI-SearchBot separately from model-training crawler policy. |
| Vercel bot protection SEO crawler risk | Deployment and security owners | Makes crawler access a deployment concern: public pages should not be trapped behind browser-only challenges. |
Discovery rationale
Crawler policy by function
The policy separates public search/retrieval discovery from private product access. Search and retrieval crawlers can read public pages, docs, research, glossary, robots.txt, sitemap.xml, llms.txt and ai-manifest.json. Authenticated Suite routes, API responses, customer data, admin tools, tokenized capture links and unreviewed directory records remain private.
- Search and retrieval: public pages should be crawlable and indexable where the content is meant to be found.
- Training use: training crawler policy is documented separately and can be changed without opening private routes.
- User-initiated agents: agent fetchers may read public docs, but cannot bypass Suite authentication or API-key scope.
- Security challenge risk: public discovery URLs must not depend on JavaScript challenges that non-browser crawlers cannot pass.
Discovery rationale
What llms.txt means here
The llms.txt file is useful as a compact public map for agents, but it is not treated as a Google ranking lever. The durable discovery work is the crawlable HTML, internal links, canonical URLs, sitemap coverage, truthful entity definitions and source-backed content.
- Do maintain llms.txt as a helpful public navigation artifact.
- Do not claim that llms.txt guarantees AI citations or Google visibility.
- Do use ai-manifest.json to list public routes, entities and private boundaries in a machine-readable way.
Discovery rationale
Why the policy is public
AI search teams, SEO professionals and browser-agent builders need a public policy they can cite. This page explains how iGaming Suite allows public discovery while refusing private data exposure, cloaking and unreviewed automated action.
- Search and retrieval crawlers can read public pages.
- Private Suite workspaces, APIs and user data are excluded.
- The policy gives future automations a boundary to test against.
Evidence discipline
Why this page is built this way
- Google generative AI optimization guide Supports the conclusion that Google Search does not require llms.txt, AI-only files or special markup for generative AI features.
- Google robots.txt guide Supports using robots.txt to manage crawler access while recognizing it is not a private-data security mechanism.
- OpenAI crawler documentation Supports separating OAI-SearchBot, GPTBot and user-triggered agents rather than treating all AI traffic as one crawler.
- Vercel Bot Management Supports reviewing Bot Protection and AI Bot rules so useful public crawlers are not blocked by deployment configuration.
Questions this page answers
Is llms.txt a Google ranking factor?
No. iGaming Suite keeps llms.txt as an agent-readable map, but the Google-facing foundation is crawlable helpful content, indexing eligibility, canonical URLs, snippets and sitemap coverage.
Can AI crawlers access private Suite data?
No. Public crawler access applies only to public pages and machine-readable discovery files. Suite workspaces, API data, customer records, sessions and admin controls remain private.
What is the biggest deployment risk for AI discovery?
A crawler or agent receiving a browser-only security challenge on public URLs. That must be checked in Vercel Firewall/Bot Management and Search Console URL Inspection.