Answer first: A UK or Isle of Man firm cannot buy a mention in Perplexity, Microsoft Copilot, or an Apple answer. It can choose, in public files it already controls, whether those products are allowed to crawl a page and, for Microsoft and Apple, whether a crawled page may be used in an AI answer or to train a model. Those choices are separate. Blocking the training control does not, on the vendor’s own pages, remove the site from that vendor’s search. ChatGPT and Google use a different set of tokens, covered in the AI search citations guide.

Quick answer

Perplexity, Microsoft, and Apple do not share a switch. Copilot follows Bing’s index:

  • Perplexity search uses PerplexityBot. Perplexity says this bot surfaces and links sites in Perplexity search results, and that it is not the crawler used for AI foundation models. Perplexity-User fetches a page when a person asks a question. Perplexity says that fetcher generally ignores robots.txt.
  • Bing and Copilot share Bing’s crawl and index. Bing’s webmaster guidelines say robots.txt controls crawl access, not indexing, and they tell owners not to block Bingbot on pages they want understood. noindex, noarchive, and nocache are the meta controls for whether a URL appears in Bing search and Copilot, or how much of it Copilot may use.
  • Apple search (Spotlight, Siri, Safari) uses Applebot. Applebot-Extended does not crawl. Apple says it is only the signal for whether pages Applebot already fetched may train Apple’s foundation models. A nosnippet tag is the separate opt-out Apple describes for broad world-knowledge answers that cite sources.

None of these settings is a citation. Microsoft’s guidelines say SEO does not guarantee rankings or traffic, and that generative engine optimisation does not guarantee grounding or citations. Recheck the vendor pages when you edit the file. This article records what they said for a reader on 12 October 2026.

Perplexity: two agents, two jobs

Perplexity’s crawler documentation says each robots.txt setting works independently, and that a change may take up to 24 hours to show up in their systems.

PerplexityBot is the search crawler. The docs say it is designed to surface and link websites in Perplexity search results, and that it is not used to crawl content for AI foundation models. If you want the site eligible for those results, Perplexity recommends allowing PerplexityBot and allowing requests from the IP ranges it publishes at perplexity.com/perplexitybot.json. Copying an old IP list out of a blog post is how a firewall ends up allowing the wrong addresses.

Perplexity-User is the user-action fetcher. When someone asks Perplexity a question, it may visit a page to answer and may include a link. Perplexity says this agent is not used for web crawling or for training foundation models, and that because a person requested the fetch it generally ignores robots.txt. The published ranges are at perplexity.com/perplexity-user.json.

A security tool can still return a block page after robots.txt says allow. Perplexity’s docs tell site owners to allow the bots in the firewall as well, and to match both the user-agent and the current IP list. The same failure hits a small firm on a locked-down host: the text file allows PerplexityBot, the edge challenges it, and nothing useful is fetched.

There is no Perplexity form in those docs that “submits” a small-business site. Allowing the search bot makes the public page eligible to be fetched. The page still has to say something worth quoting.

Bing and Copilot: one index, then the meta tags

Bing’s webmaster guidelines describe how Bing discovers, crawls, indexes, and surfaces content across Bing search, Copilot, and grounding API results. The line that matters for a small firm: Bing and Copilot search experiences rely on the same crawling, indexing, and ranking foundation as ordinary search. Practices that help discovery and clear content also support eligibility for grounding and citations. They do not promise either.

What the guidelines tell an owner to get right:

  • Let Bingbot crawl and render the pages that matter. Blocking Bingbot in robots.txt is on their list of things to avoid. Content that cannot be reliably rendered may not be indexed or selected for grounding.
  • Put the important facts in text. Images and video should reinforce the page, not be the only place a reader can learn what you do or where you work.
  • Keep one URL per topic, with the answer near the top, and use the same names for the business and the places you cover.
  • Use a sitemap of canonical URLs if you have one. Bing says sitemaps do not guarantee visibility.
  • When a URL is added, changed, or removed, Bing points owners to IndexNow so the index can refresh. The guidelines say timely notices help reduce outdated URL references in Copilot and grounding results. A notice is not a promise that Copilot will then cite you.

robots.txt is the crawl switch. It is not the indexing switch. Bing says a crawler has to be able to fetch the page to see a meta tag, so a Disallow in robots.txt hides the tag as well as the page.

The meta controls, from the same guidelines and from Bing’s robots meta tag help:

  • noindex means the URL should not appear in Bing search, Copilot experiences, or grounding API results. Bing has to be allowed to crawl the page to read the tag.
  • noarchive, in the webmaster guidelines, prevents the content being used in Copilot responses and grounding results. Bing’s meta tag help pairs “do not store a cached page” with “do not link in Chat and Copilot” and “do not use the content for training Microsoft’s generative AI foundation models.” The guidelines are the page that names the Copilot effect as noarchive. Read both before you add the tag, because nocache sits next to it and is easy to mix up.
  • nocache limits Copilot to the URL, title, and snippet. Bing says that if a page carries both nocache and noarchive, it is treated as nocache.
  • nosnippet and data-nosnippet stop Bing showing captions and may limit how useful a Copilot citation can be.

You can scope a tag to Bing with name="bingbot" instead of name="robots". Read your live source before you add one. A plugin that injects noarchive on every URL will quietly take the whole site out of Copilot answers, which may be what you wanted, and is a poor accident.

Bing’s guidelines also name the abuses that can cost grounding eligibility: cloaking, scraped pages, keyword stuffing, misleading structured data, and large amounts of automatically generated content with no editorial review. A stack of near-copy town pages sits in that last group. Microsoft uses “generative engine optimisation” for eligibility to be grounded and cited, and it says that work does not guarantee a citation.

Apple: Applebot crawls, Applebot-Extended does not

Apple’s About Applebot page, which showed a published date of 4 September 2026 when this article was written, separates search from training in the same spirit as the other vendors, with its own names.

Applebot crawls the web for search features in Spotlight, Siri, and Safari. Apple says allowing it in robots.txt lets site content appear in those search results. The same crawl may also be used to help train Apple foundation models that power Apple Intelligence and other generative features. Apple publishes address ranges at search.developer.apple.com/applebot.json. Verify a visitor against that list or against reverse DNS in applebot.apple.com before you treat a log line as Apple. The user-agent string is easy to fake.

Applebot-Extended is not a second crawler. Apple says it does not crawl webpages. It only decides how data already crawled by Applebot may be used. Disallow it when you do not want that content used to train Apple’s general-purpose foundation models. Apple says pages that disallow Applebot-Extended can still be included in search results, and that Applebot-Extended rules are not a ranking signal for Search.

nosnippet is a third, narrower choice. Apple says a nosnippet meta tag opts the page out of being used as extra context for broad world-knowledge answers in Siri and Search, including answers that show source links. Suggestions to visit the URL then carry the title only. Apple also says that disallowing Applebot-Extended and adding nosnippet still leaves the page discoverable in Spotlight, Siri, and Safari, as long as Applebot itself is allowed to crawl.

Two Apple rules catch firms that only edited the Google block:

  • If robots.txt never mentions Applebot, but it does mention Googlebot, Apple says Applebot follows the Googlebot instructions. A Googlebot disallow can therefore keep Apple from crawling, even though you never typed the word Applebot.
  • Applebot does not follow crawl-delay.

Apple supports the usual noindex, nofollow, and nosnippet meta tags, including an X-Robots-Tag header. noindex means the page will not be indexed and will not appear in Spotlight or Siri Suggestions.

What the page still has to say

Crawler access only makes a page eligible to be fetched. An assistant still needs a sentence it can repeat without inventing the rest.

Write that sentence in HTML, near the top of the one service page you most want understood. Name the service, the area you actually cover, and the next step (a form, a phone number, or an email). Do not leave the only clear sentence inside a slide, a PDF, or a picture of a van. Bing’s guidelines make the same point: essential information belongs near the top, each URL should stay on one topic, and the business and its locations should be named clearly and consistently. Structured data may help grounding only when it matches the visible text, and Bing says it does not guarantee visibility.

For a local firm, the coverage sentence is the GEO job. Say which towns, counties, or parts of the Island you cover, and say what you do not cover if callers often assume the wrong area. Use the same wording in the footer and on any profile you control. The GEO and citations guide is that consistency check. If Copilot or another assistant is still repeating last year’s area, the wrong business details guide is the correction route. Do not publish a page that argues with the assistant. Publish the correct fact.

A Manchester firm and a Douglas firm use the same crawler tokens. The tokens do not know the company’s home jurisdiction. A firm that only works on the Island should not publish a sentence that reads as UK-wide attendance. A firm that covers three postcodes should not publish a page per village with the town name swapped. Bing lists automatically generated content at scale, without editorial review, among the practices that can lose grounding eligibility.

Page shape, FAQ wording, and internal links sit in the answer-engine optimisation checklist and the service-page AEO guide. Halo’s visibility audit is the page-and-evidence review. This article is only the Perplexity, Microsoft, and Apple control review.

A robots.txt pattern to check, not to paste blindly

This is the shape the vendor docs describe for “allow search, refuse Apple’s training use.” Read the live file before you add it. A User-agent: * group that disallows everything still catches any bot that does not have its own group. Perplexity-User is not controlled by this file.

``` User-agent: PerplexityBot Allow: /

User-agent: Bingbot Allow: /

User-agent: Applebot Allow: /

User-agent: Applebot-Extended Disallow: / ```

If the business does not want to appear in Perplexity search results, disallow PerplexityBot as well. That is a real choice. It is not an SEO trick. Write down which choice you made and the date, because a plugin or a host panel will edit the file again.

A 30-minute check

Do this on the live site, logged out, on the one service URL you most want understood.

  1. Open /robots.txt. Look for User-agent: * with Disallow: /. Then look for separate groups for PerplexityBot, Bingbot, Applebot, and Applebot-Extended.
  2. If Googlebot is disallowed and Applebot has no group of its own, treat Apple search as blocked until you say otherwise. That is Apple’s stated fallback, not a guess.
  3. View source on the service URL. Confirm the coverage sentence is text, and that the page is not noindex, noarchive, or nosnippet unless you meant it to be. Check both the meta tag and any X-Robots-Tag header.
  4. Click through from the homepage. A service page with no internal link is harder for a person and for Bing, which lists crawlable internal links among the discovery basics.
  5. If you use Bing Webmaster Tools, inspect that URL and read whether Bing indexed it. If you do not have Webmaster Tools, set it up before you pay anyone to “track Copilot rankings”.
  6. If a firewall or bot fight sits in front of the site, check that PerplexityBot is not being challenged. Perplexity’s own note is that robots.txt is not enough when the edge blocks the request.
  7. Write down the date and what you changed. Judge later against that note. Perplexity says robots.txt changes may take up to 24 hours. Bing says crawlers can keep a cached copy of robots.txt for at least a few hours. Neither figure is a promise that you will then be cited.

What not to claim

  • That a Perplexity link, a Copilot answer, a Bing ranking, or a Siri mention is guaranteed.
  • That disallowing Applebot-Extended removes the site from Spotlight, Siri, or Safari search. Apple says the opposite.
  • That disallowing PerplexityBot stops Perplexity-User. Perplexity says that fetcher generally ignores robots.txt.
  • That noindex and noarchive are the same Bing switch. They are different controls, and Bing describes different effects.
  • That IndexNow, a sitemap, an llms.txt file, or FAQ schema forces any of these products to quote you.
  • A client result, a town you do not cover, or a “before and after” Halo has not published with evidence. This guide does not add one.

Where Halo fits

Search, AEO, and content is the related service. The boundary is the same as on the studio’s public description: no guaranteed rankings, citations, leads, or revenue. If you want a person to look at one URL and the robots.txt in front of it, start with the free audit and send the page, not a list of bots to “optimise”.

FAQ

If I block PerplexityBot, can someone still open my page from Perplexity?

PerplexityBot is the search crawler. Perplexity-User is the fetch that can run when a person asks a question, and Perplexity says that fetcher generally ignores robots.txt. A firewall can still block the request. Blocking PerplexityBot is the search opt-out Perplexity describes. It is not a promise that a Perplexity screen will never show the URL.

Does blocking Bingbot remove my business from Copilot?

Bing says Copilot search uses the same crawling and indexing foundation as Bing search, and it lists blocking Bingbot among the ways to keep important pages from being rendered and selected for grounding. That is a broad block, not a Copilot-only switch. For a page you still want in Bing, the guidelines point to noarchive and nocache for Copilot and grounding use, and to noindex when the URL should not appear in Bing search, Copilot, or grounding at all.

Will disallowing Applebot-Extended remove me from Siri and Spotlight?

No. Apple says Applebot-Extended does not crawl pages, and that pages which disallow it can still appear in search results. It is the opt-out for training Apple’s foundation models. Search depends on Applebot. The nosnippet tag is the separate opt-out Apple describes for broad world-knowledge answers that include source links.

Are noarchive and noindex the same thing on Bing?

No. Bing says noindex is for a URL that should not appear in Bing search, Copilot experiences, or grounding API results. It says noarchive prevents the content being used in Copilot responses and grounding results. If a page has both nocache and noarchive, Bing’s meta tag help says the page is treated as nocache, which limits Copilot to the URL, title, and snippet. Bing must be allowed to crawl the page to read any of these tags.

Does an Isle of Man firm need different crawler rules from a UK firm?

No. The tokens read the public site, not the company’s home jurisdiction. What must be different is the sentence about where you work. A Douglas firm that only works on the Island should not publish UK-wide attendance. Local wording is covered in the GEO and citations guide.

Does allowing these crawlers mean I will be cited?

No. Perplexity, Microsoft, and Apple describe access and eligibility. Microsoft says outright that SEO does not guarantee rankings or traffic, and that generative engine optimisation does not guarantee grounding or citations. Allowing a crawler removes one reason a citation is impossible. It does not place the firm in an answer.

Sources