PingTalks PingTalks
AI 20 min read

AI Search Optimization: Get Cited by AI Overviews and ChatGPT

How Google AI Overviews, AI Mode, ChatGPT, Perplexity, Gemini and Copilot find and cite pages, from their own docs, plus the bots, tracking and 90-day plan to get cited.

Marketer typing a question into an AI search bar on a laptop, illustrating AI search optimization

Key takeawaysThe story in 20 seconds

01Google says AI Overviews and AI Mode have no additional requirements: a page must be indexed and eligible for a snippet to be shown as a supporting link.

02OpenAI’s OAI-SearchBot controls inclusion in ChatGPT search, while GPTBot controls training use, and OpenAI keeps the two settings independent.

03Google-Extended limits Gemini training and grounding, and Google says it does not affect Google Search inclusion or ranking.

04llms.txt is a 2024 proposal from Jeremy Howard, not an official standard, and Google says AI Overviews need no AI text files.

AI search optimization is the work of getting your pages picked, quoted and linked as sources inside AI answers: Google AI Overviews, AI Mode, ChatGPT search, Perplexity, Gemini and Microsoft Copilot. The short version: let the right bots in, get indexed, and publish the clearest, most specific answer on your topic.

This pillar shows how each platform finds its sources, using each platform’s own documentation, then covers what to put in robots.txt, what llms.txt really is, how to track AI referrals in GA4, and a 30/60/90-day plan built for Indian marketers and founders.

What is AI search optimization?

AI search optimization, also called generative engine optimization (GEO) or answer engine optimization (AEO), is the practice of making your content easy for AI search systems to find, trust and cite, so your brand appears as a linked source when Google, ChatGPT, Perplexity, Gemini or Copilot answers a question.

Classic SEO wins you a blue link. AI SEO wins you a citation inside a generated answer, and often a brand mention even when nobody clicks. The two overlap heavily. Google’s Search Central documentation says plainly that there are no additional requirements to appear in AI Overviews or AI Mode.

So treat AI search as a layer on top of good SEO, not a replacement. If your fundamentals are shaky, fix them first with our complete SEO guide for beginners and marketers. Then use this page to win the citation.

TermWhat people meanWhere you will hear it
AI SEOSEO work aimed at AI answers as well as classic rankingsAgencies, in-house SEO teams
Generative engine optimization (GEO)Getting cited by tools that generate answers, such as ChatGPT and PerplexityResearch papers, AI marketing vendors
Answer engine optimization (AEO)Structuring content so a system can lift a direct answerVoice search and featured snippet discussions
AI search optimizationThe umbrella for all of the aboveThis guide

Why AI search matters more in India

Google search AI features are moving fastest in India. Google’s India blog said on 8 October 2025 that AI Mode, which had already launched across India in English and Hindi, was adding seven more Indian languages: Bengali, Kannada, Malayalam, Marathi, Tamil, Telugu and Urdu.

The same post says India has Google’s largest user base for voice and visual search, and that Search Live, the voice and camera mode inside AI Mode, came to India first outside the US. That is a direct hint for AI digital marketing in India: regional-language and spoken-style questions are now answered by AI, not ten blue links.

7Indian languages added to Google AI Mode on 8 October 2025, per Google’s India blog
~3xhow much longer AI Mode questions are than traditional searches, globally, per Google
24 hrsroughly how long OpenAI says robots.txt changes take to affect ChatGPT search
10,000URLs you can send in one IndexNow POST, per the IndexNow documentation

How do Google AI Overviews and AI Mode choose sources?

Google AI Overviews and AI Mode pick supporting links from pages that are already indexed and eligible to show a snippet in Google Search. Both can use a query fan-out technique, running several related searches across subtopics, then link to pages that support the generated answer. Google lists no extra files, markup or requirements.

Google’s “AI features and your website” page, last updated 10 December 2025, is the official source here. Read it once. It is short, and it kills most of the myths you will see on LinkedIn.

What AI Overviews do

AI Overviews summarise a complicated topic or question and link to sites for further reading. Google says they appear only when its systems judge them additive to classic results, which is why many queries never trigger one.

What AI Mode does

AI Mode is built for queries that need exploration, reasoning or comparison. It returns a fuller AI-generated response with links to supporting websites. Google notes that AI Overviews and AI Mode may use different models and techniques, so the same query can surface different links in each.

Why query fan-out changes your content plan

With fan-out, Google issues multiple related searches across subtopics and data sources while it builds the answer. Google says this lets the response show a wider and more diverse set of links than a classic search.

Our read: you no longer need to rank first for the head term to be cited. You need to own the sub-questions. A page that answers “GST on digital marketing services” precisely can be pulled into an answer about “how to price an agency retainer in India” because the fan-out went looking for that sub-answer.

What Google says you can control

Google lists the same best practices it gives for all of Search. Here is the list, in plain terms:

  • Allow crawling in robots.txt and in your CDN or hosting setup.
  • Make content findable through internal links.
  • Give users a good page experience.
  • Keep important content available as text.
  • Support text with high-quality images and videos where useful.
  • Make sure structured data matches the visible text on the page.
  • Keep Merchant Center and Business Profile information current.

To limit what appears, Google points to nosnippet, data-nosnippet, max-snippet and noindex. Because AI is part of Search, Googlebot’s robots.txt rules are your crawl control for AI Overviews and AI Mode too. If a control is not working, Google suggests checking the HTML Googlebot received with the URL Inspection tool, then allowing time for a recrawl, which it says can take from several days to several months.

Watch out: a page-wide nosnippet makes the page ineligible for a snippet, and Google requires snippet eligibility for a supporting link. If you only want to hide one block, such as pricing or a coupon code, wrap that block in data-nosnippet instead.

AI search optimization on a phone showing Google AI Mode results
Google AI Mode now answers questions in English, Hindi and seven more Indian languages.

How do ChatGPT, Perplexity, Gemini and Copilot find your pages?

ChatGPT search, Perplexity, Gemini and Microsoft Copilot each depend on crawlers and search indexes: OpenAI’s OAI-SearchBot for ChatGPT search, PerplexityBot for Perplexity, the Google Search index for Gemini grounding, and Bing’s systems for Copilot. Block the relevant bot or stay unindexed, and you lose your best shot at a citation.

Each company publishes its crawler rules. The table below is built only from those pages.

Bot or tokenOwner and jobObeys robots.txt?Blocking it means
GooglebotGoogle: crawls for Search, including AI Overviews and AI ModeYesOut of Google Search and its AI features
Google-ExtendedGoogle: a control token for Gemini model training and grounding, no separate crawlerYes (token only)Out of Gemini training and grounding; Google says Search inclusion and ranking are unaffected
OAI-SearchBotOpenAI: surfaces websites in ChatGPT searchYesOut of ChatGPT search answers; you may still show as a navigational link
GPTBotOpenAI: crawls content that may train foundation modelsYesSignals no training use; independent of OAI-SearchBot
ChatGPT-UserOpenAI: user-initiated visits, such as fetching a page to answer a questionMay not applyLittle; OpenAI says it does not decide search inclusion
PerplexityBotPerplexity: surfaces and links sites in Perplexity results, not for trainingYesOut of Perplexity search results
Perplexity-UserPerplexity: fetches a page when a user asks a questionGenerally noLittle; it is user-triggered

ChatGPT search

OpenAI’s publisher FAQ says any public website can appear in ChatGPT search, and that you should not block OAI-SearchBot if you want your content in ChatGPT summaries and snippets. OpenAI’s bots page adds that robots.txt changes take about 24 hours to show in search.

ChatGPT’s help centre says answers that use search carry inline citations and a Sources button listing every cited link. It also says ChatGPT may share disassociated search queries with third-party search providers such as Bing. That is why Bing indexing matters even if you only care about ChatGPT.

One trap: OpenAI says a page you disallow can still appear as a bare link and title if it is found through a third-party search provider. To stop that, use a noindex meta tag, and leave the page crawlable so the bot can read the tag.

Perplexity

Perplexity’s crawler docs describe PerplexityBot as the bot that surfaces and links websites in Perplexity search results, and say it is not used to crawl content for AI foundation models. Perplexity recommends allowing it, and says robots.txt changes can take up to 24 hours. It publishes IP ranges at perplexitybot.json, which helps your developer allowlist it in a firewall.

Gemini

Google’s crawler documentation explains that grounding in Gemini Apps means content from the Google Search index is supplied to the model at prompt time. In practice, your Google SEO is your Gemini visibility. If Googlebot cannot crawl you, Gemini cannot ground on you.

Microsoft Copilot and Bing

Microsoft launched AI Performance in Bing Webmaster Tools as a public preview on 10 February 2026. It reports how often your pages are cited in Microsoft Copilot, AI-generated summaries in Bing and select partner integrations. If you have never verified your site in Bing Webmaster Tools, this alone is reason to do it this week.

Microsoft also recommends IndexNow, a free protocol that notifies participating search engines when you add, update or delete a URL, so AI systems use the current version of a page. The IndexNow documentation says you host a key file at your root, can submit up to 10,000 URLs in one POST, and that participating engines share submissions with each other.

Pro tip: many Indian sites sit behind Cloudflare or a hosting firewall with aggressive bot rules. Google explicitly says to allow crawling in your CDN and hosting infrastructure, not just robots.txt. Ask your developer to check firewall logs for OAI-SearchBot and PerplexityBot hits before you assume robots.txt is enough.

Should you block GPTBot, OAI-SearchBot or Google-Extended in robots.txt?

For most Indian brands, the answer is no for search bots and a business decision for training bots. Keep Googlebot, OAI-SearchBot and PerplexityBot allowed so you stay eligible for citations. Blocking GPTBot or Google-Extended limits training and, for Google-Extended, Gemini grounding, without touching your Google Search rankings.

Pick the row that matches your goal. Each line goes in your robots.txt as its own group.

Your goalrobots.txt linesWhat happens
Maximum AI visibility (most SMBs, D2C, B2B services)User-agent: * Allow: /All search and AI bots can crawl
Cited in ChatGPT, not used for OpenAI trainingUser-agent: GPTBot Disallow: /GPTBot stays out; OAI-SearchBot is unaffected because OpenAI keeps the two independent
Out of Gemini training and groundingUser-agent: Google-Extended Disallow: /Google Search inclusion and ranking unaffected, but Gemini cannot ground answers on you
Out of ChatGPT search answersUser-agent: OAI-SearchBot Disallow: /No ChatGPT search citations; navigational links still possible
Out of Perplexity resultsUser-agent: PerplexityBot Disallow: /No Perplexity search listing; Perplexity-User may still fetch on request

Who should block training bots? News publishers, paid research firms and course creators whose content is the product. Who should not? Almost everyone selling a product or service. A D2C brand in Jaipur gains nothing by keeping its size guide out of an AI answer about lehenga sizing.

Blocking a search bot to protect your content is like pulling down your shop shutter on Diwali week. You keep the stock and lose the customers.

What is llms.txt and do you need one?

llms.txt is a proposed Markdown file, placed at /llms.txt, that gives AI agents a short curated guide to a site’s key pages. Jeremy Howard proposed it on 3 September 2024. It is not an official standard, and Google says no AI text files are needed for AI Overviews or AI Mode.

The llmstxt.org proposal, updated on 10 August 2026, sets out a simple format:

  • An H1 with the site or project name. This is the only required part.
  • A blockquote with a short summary.
  • Optional free-text sections with more detail.
  • H2 sections listing links in the form [name](url): notes.
  • An optional H2 called “Optional” for secondary links an agent can skip.

The proposal calls itself open for community input and names no standards body. Neither OpenAI’s publisher FAQ nor its bots page mentions llms.txt, and Google’s page rules out needing new machine-readable files.

Our position: it costs an hour, so a documentation-heavy SaaS or API company can add one for coding agents. For a typical Indian SMB, a clean sitemap, crawlable HTML and IndexNow will do far more. Do not let anyone bill you for “llms.txt optimisation”.

Team planning AI SEO content structure for answer engine optimization
Answer-first structure, named entities and original data are what AI answers lift.

What content gets cited in AI answers?

AI answers cite pages that state a clear answer early, name entities precisely, add something the rest of the web lacks such as original data, and use structure machines can parse: headings, tables, FAQs and structured data that matches the visible text. Thin, commodity rewrites give a system no reason to pick you.

Microsoft’s AI Performance announcement is unusually direct about this. It tells publishers to use clear headings, tables and FAQ sections, to back claims with examples, data and sources, to update content regularly, and to keep text, images and video consistent about the same entities. Here is how to turn that into a writing routine.

1. Lead with the answer

Put a 40 to 60 word direct answer right under each question heading. Then expand. An AI system pulling a sub-answer for a fan-out query wants a self-contained passage, not three paragraphs of context.

  • Weak: “Choosing the right clinic is an important decision, and there are many factors to consider.”
  • Strong: “Our Indiranagar clinic charges a flat fee for the first consultation, lists it on this page, and lets you book the slot on WhatsApp in under two minutes.”

2. Name entities fully and consistently

Write “Google Search Console”, not “GSC”, on first mention. Name the product, the company, the city, the regulation. Use the same brand name, address and phone everywhere, including your Google Business Profile, which Google explicitly asks you to keep current. Entities are how a model connects your page to the question.

3. Publish something original

If ten pages say the same thing, the model needs only one of them. Give it a reason to pick you: a pricing benchmark from your own invoices, a survey of 200 customers, a teardown with screenshots, a local example from Pune or Kochi. This is also the core of experience in E-E-A-T, which we unpack in Is E-E-A-T the secret sauce for top rankings?

4. Use structured data that matches the page

Google’s structured data guide recommends JSON-LD, says markup must describe content users can actually see, and never promises a rich result. Google’s AI features page adds that no special schema.org markup is required for AI Overviews or AI Mode.

So schema is hygiene, not a hack. Add Organization, Article, Product, FAQPage or LocalBusiness where they truthfully fit, then validate with the Rich Results Test and watch the rich result reports in Search Console.

5. Keep pages fresh and tell engines when they change

Update the date only when you update the substance. Then ping IndexNow so Bing and other participating engines pick up the change. Microsoft’s IndexNow page says many content management systems and CDNs have built-in support, so this is usually a toggle, not a project.

6. Write in the language your buyer asks in

With AI Mode now answering in Hindi, Bengali, Tamil, Telugu, Kannada, Malayalam, Marathi and Urdu, a well-made regional-language page faces far less competition than its English twin. Start with your top five money pages in the one or two languages your customers actually use.

7. Keep the important bits in text

Google asks you to keep important content available as text and to support it with good images and video. A price list trapped inside a JPG, or specs buried in a PDF brochure, is the first thing to fix. Put the numbers in an HTML table and keep the image as support.

How do you measure AI search visibility?

Measure AI search visibility in three places: Google Search Console, where AI Overviews and AI Mode traffic sits inside the Performance report under the Web search type; GA4, where you group AI assistant visits by referral source; and Bing Webmaster Tools, whose AI Performance report counts Copilot citations.

Google Search Console

Google says sites appearing in AI Overviews and AI Mode are counted in overall Search traffic in the Performance report, under the “Web” search type. There is no separate AI filter. Google also suggests tracking conversions and time on site in tools like Google Analytics, and claims clicks from results pages with AI Overviews are higher quality.

The practical move: watch long, conversational queries in the Queries tab. Those are the searches most likely to trigger AI features. Our list of Google Search Console features most marketers never use shows how to filter them with regex.

GA4: build an AI assistants channel

OpenAI says ChatGPT adds utm_source=chatgpt.com to referral URLs for sites that allow OAI-SearchBot. Other assistants show up as referral sources. Group them so they stop hiding inside “Referral”.

  1. Open channel groups. In GA4, go to Admin, then Data display, then Channel groups. You need Editor access at property level.
  2. Copy the default group. Click Create new channel group, name it “Default plus AI”, and save.
  3. Add an AI assistants channel. Click Add new channel, set the condition on Source with “matches regex”, and use a pattern such as chatgpt|openai|perplexity|gemini|copilot|claude.
  4. Move it above Referral. Google’s help page says traffic goes to the first channel it matches, so order matters. Save the group.
  5. Read the report. Open Acquisition, then Traffic acquisition, and switch the dimension to your new channel group. Google says custom groups apply to past data too.

A standard GA4 property allows two custom channel groups and 50 channels per group, per Google’s help page, so use the slot wisely. For the wider setup, see our GA4 guide.

Bing Webmaster Tools AI Performance

Microsoft’s report shows Total Citations, Average Cited Pages, a sample of grounding queries, page-level citation counts and trends over time. Microsoft notes citations count references, not placement or ranking. It is the closest thing to an official “AI rank tracker” any platform offers today.

A manual prompt panel

Neither OpenAI’s nor Perplexity’s publisher pages describe a citation report for site owners, so build your own panel. List 25 questions your buyers ask, in their words. Run them monthly in ChatGPT, Perplexity, Gemini, Copilot and Google AI Mode. Log three things: were you cited, were you mentioned, and who was cited instead. That last column is your content brief.

AI search visibility tracked in an analytics dashboard with GA4 referrals
Group ChatGPT, Perplexity and Gemini referrals into one channel so they stop hiding in Referral.

GEO checklist: 15 checks before you hit publish

Use this generative engine optimization checklist on every money page. It mixes Google’s, OpenAI’s, Perplexity’s and Microsoft’s published guidance with what we see work for Indian sites.

  1. The page is indexed in Google (URL Inspection) and in Bing Webmaster Tools.
  2. robots.txt allows Googlebot, OAI-SearchBot and PerplexityBot.
  3. Your CDN or firewall is not challenging those bots.
  4. The page is not set to nosnippet unless you truly mean it.
  5. Every question H2 opens with a 40 to 60 word direct answer.
  6. Key facts appear as text, not only inside images or PDFs.
  7. Entities are named in full on first mention.
  8. At least one original element: data, screenshots, a worked example or a quote from your own expert.
  9. A comparison table where buyers compare options.
  10. A short FAQ answering real follow-up questions.
  11. JSON-LD that matches visible content, validated in the Rich Results Test.
  12. An author with a real bio and credentials.
  13. Internal links from related pages so crawlers find it.
  14. A visible updated date, changed only for real updates, plus an IndexNow ping.
  15. A regional-language version for your top pages if your buyers search that way.

What are the biggest AI SEO mistakes and myths?

The biggest AI SEO mistakes are blocking search bots by accident, chasing tricks like llms.txt or special AI schema before fixing indexing, publishing generic AI-written rewrites, and judging success by clicks alone. The biggest myth is that AI search needs a separate playbook: Google says normal SEO best practices still apply.

MythRealityWho says so
You need special AI schema to show in AI OverviewsNo special schema.org markup is requiredGoogle Search Central
llms.txt is now a web standardIt is a 2024 proposal open for community inputllmstxt.org
Blocking GPTBot removes you from ChatGPT searchGPTBot and OAI-SearchBot settings are independentOpenAI
Blocking Google-Extended hurts Google rankingsIt is not used as a Search ranking signalGoogle
robots.txt stops every AI visitUser-triggered agents such as ChatGPT-User and Perplexity-User may not follow itOpenAI, Perplexity

Mistakes we see on Indian sites

  • Copy-paste robots.txt. A developer adds a “block all AI bots” snippet from a forum and wipes the site out of ChatGPT and Perplexity overnight.
  • Bing ignored. Teams verify Google Search Console and never touch Bing Webmaster Tools, even though Copilot reporting lives there and ChatGPT may use Bing as a search provider.
  • AI rewrites at scale. Fifty near-identical “best X in Bangalore” pages give a model nothing new to cite. One deep page with real data wins.
  • Only tracking clicks. A citation in an AI answer builds brand recall even without a click. Pair GA4 with your prompt panel.
  • Hiding prices. Buyers ask AI “how much does X cost in India”. If your page dodges the number, a competitor’s page answers it.

When traffic moves suddenly, check our Google algorithm updates tracker before you blame AI Overviews. A core update and an AI feature change look identical in a traffic graph.

Do ChatGPT ads or Google AI tools change AI search optimization?

Not the fundamentals. Treat paid placements and organic citations as separate tracks. ChatGPT ads, whose landing pages OpenAI checks with its OAI-AdsBot, buy visibility near answers, while citations are earned through crawlable, useful pages. Run both if budget allows, but never expect an ad budget to buy you citations.

OpenAI’s bots page says OAI-AdsBot only visits landing pages submitted as ads and does not train models. So if you advertise on ChatGPT, keep that bot allowed on your landing pages. If you already run ChatGPT ads, read our explainer on ChatGPT ads creative enhancements and how to opt out before your next campaign. The Go deeper section below has the full paid coverage.

The same split applies to Google AI tools. AI Mode, Search Live and Gemini are surfaces you earn organically. Google’s AI-powered ad products are a separate budget line, covered in our ads coverage rather than here.

Your 30/60/90-day AI search optimization plan

This plan assumes a small team: one marketer, part-time developer access and a founder who can share real data. Scale the page counts up if you have more hands.

WindowFocusTasksOutput
Days 1 to 30Access and baselineAudit robots.txt and CDN bot rules; verify Bing Webmaster Tools; set up IndexNow; build the GA4 AI assistants channel; run the 25-question prompt panel onceBaseline sheet: citations, mentions, AI referral sessions
Days 31 to 60Fix money pagesRewrite your top 10 pages answer-first; add tables, FAQs and matching JSON-LD; move key facts out of images and PDFs; add author bios10 upgraded pages, all pinged via IndexNow
Days 61 to 90Earn new citationsPublish two original-data pieces; launch regional-language versions of your top 5 pages; fill gaps your prompt panel exposedSecond panel run compared with baseline; next-quarter brief

Repeat the prompt panel at day 90 with the exact same questions. If you are cited in more answers, keep going. If not, look at who is cited instead and study what their page has that yours lacks.

PingTalks verdict

AI search optimization is not a new discipline with secret rules. It is SEO done properly, plus three things most Indian sites skip: allowing the right AI bots, showing up in Bing, and publishing original, answer-first pages. Ignore llms.txt hype and special-schema sellers. Fix access in week one, upgrade your ten best pages next, and measure citations, not just clicks.

Go deeper

These PingTalks guides pick up where this pillar stops:

What to do now

  • Open your robots.txt today and confirm Googlebot, OAI-SearchBot and PerplexityBot are allowed.
  • Ask your developer to check CDN or firewall logs for blocked AI bots.
  • Verify your site in Bing Webmaster Tools and open the AI Performance report.
  • Set up IndexNow through your SEO plugin or CDN.
  • Create the AI assistants channel in GA4.
  • Write your 25-question prompt panel and run it once to set a baseline.
  • Rewrite one money page answer-first this week.

Frequently asked questions

Is AI search optimization different from SEO?

Mostly no. Google says standard SEO best practices still apply to AI Overviews and AI Mode, with no extra requirements. The difference is emphasis: answer-first writing, allowing AI search bots such as OAI-SearchBot and PerplexityBot, and showing up in Bing.

How do I get my website cited by ChatGPT?

Allow OAI-SearchBot in robots.txt and your firewall, make sure the page is publicly accessible, and keep it indexed in Bing, since ChatGPT may use Bing as a third-party search provider. Then publish clear, specific, answer-first content that is better than what is already cited.

Does blocking GPTBot remove my site from ChatGPT search?

No. OpenAI says GPTBot is for model training and OAI-SearchBot is for search, and the two settings are independent. Block GPTBot if you do not want training use, but keep OAI-SearchBot allowed if you want ChatGPT search citations.

Do I need an llms.txt file for AI search?

No. llms.txt is a community proposal, not an official standard, and Google says no AI text files are needed for AI Overviews or AI Mode. It is a low-cost extra for documentation-heavy sites, not a priority for most businesses.

How can I see AI Overviews traffic in Google Search Console?

Google counts AI Overviews and AI Mode traffic inside the Performance report under the Web search type, with no separate AI filter. Track long conversational queries there and pair it with GA4 conversion data.

Is AI Mode available in Indian languages?

Yes. Google said on 8 October 2025 that AI Mode was adding Bengali, Kannada, Malayalam, Marathi, Tamil, Telugu and Urdu, on top of English and Hindi in India. That makes regional-language pages a real AI search opportunity.

Sources

Kousik BhattacharyaEditor, PingTalks

Digital Evangelist and Entrepreneur at Clickseek Digital. A results-oriented marketing leader with an 18+ year track record driving transformative growth for diverse global businesses. Expert in performance marketing, demand generation, customer acquisition, strategic partnerships, and market expansion within B2B SaaS, Consumer-Tech, Fintech, Travel/Hospitality, and CRM/ITES Solutions sectors. IIM alumnus with a strong analytical background and a string of accolades, including "Top 50 Most Influential E-Commerce Professional of India" and "Top 100 Digital Marketer of India."

Two stories a day. Zero noise.

What changed in marketing and AI, why it matters and exactly what to do. In your inbox at 11:00 IST.