Deep ResearchAI Visibility24 min read

MedTech Website AI Readability in 2026: What a 188-Site Audit Can Tell You

An 188-domain audit tests crawler policies, page access and metadata, then turns bounded source panels into a practical MedTech website release checklist.

DJ Fang

Founder @ VayoMed, RAC · September 6, 2026

TL;DR

Before commissioning another AI visibility dashboard, inspect the pages a buyer or search service needs to read. VayoMed fetched 188 tracked life-sciences domains on 5 September 2026. Of these, 157 returned usable home pages; 31 returned errors, challenge pages or failed connections. Among the 121 retrieved, parseable robots.txt files in the reachable group, six contained a root disallow for at least one tracked AI token. None met that screen for OAI-SearchBot, Claude-SearchBot, Claude-User, ChatGPT-User or PerplexityBot. This describes our requests and a narrow rule screen, rather than proving those engines can access every page [1].

Reparsing the saved HTML found JSON-LD on 58 of 157 home pages (36.9%) and an Organization-related type on 45 (28.7%). These are optional descriptive signals, not an AI admission test. Google explicitly says its AI search features require no special schema or AI text file [2]. The practical priority is accessible, accurate page content; relevant markup can make that content more explicit.

Two stored citation datasets suggest useful pages to inspect next. In 144 five-entry source-domain panels, 181 of 720 entries (25.1%) match the tracked primary domain or its subdomains. That is a bounded panel statistic, not the company's share of all AI citations. Separately, 15 of 37 primary-domain matches among 110 stored top-page records are PDFs. Reference documents already appear in the evidence; replacing them indiscriminately with new marketing pages would miss the opportunity [3].

Our recommendation is to fund a small, testable release: resolve verified retrieval failures, improve one important product page and its reference links, validate applicable markup and localization, then rerun the same buyer queries. This audit identifies checks worth performing. It does not estimate the citation lift from any one change.

What the audit measures, and what it leaves open

The reader decision is how to allocate website work when the company's name or official documentation appears inconsistently in AI-assisted research. There are several possible explanations: an inaccessible page, missing information, weak internal links, an ambiguous brand name, a query that never called for a manufacturer source, or a retrieval system choosing a different reference. A website audit can test some of these explanations. A count of JSON-LD blocks cannot settle the others.

Our tracked list contains 119 medical-device companies, 42 diagnostics companies, 20 imaging companies and seven other life-sciences businesses. It is the cohort behind VayoMed's AI Visibility Reports, rather than a random industry sample [1][4]. Categories describe the tracking list; a diversified business may operate across several of them.

For each domain, the crawl requested the home page, robots.txt, the root sitemap.xml and llms.txt. It used curl with a Chrome-desktop user-agent string, followed redirects and retried some failures through alternate host or scheme forms. It did not execute JavaScript. Calling this a browser-identified request distinguishes the request header from a real browser session or a verified AI crawler.

The home-page analysis records presence: JSON-LD scripts, Organization-related types, HTML hreflang links, sitemap signals and selected response headers. It does not certify that the markup is complete, the language relationships are reciprocal, a sitemap's URLs are valid, or the content matches a regulatory authorization. During this publication review, a proper HTML parser and recursive JSON walk recovered two JSON-LD pages and three Organization detections missed by the first script. The corrected values are used throughout [1].

Two independent observations matter commercially. A failed page request is a reason to investigate delivery. A missing factual answer on a successfully retrieved product page is a reason to improve content. Both are closer to a buyer's problem than a composite readability score that awards points for optional files.

Crawler policies: distinguish training, search and user requests

A crawler token identifies a particular operator function. Training, search indexing and visits initiated by a user are separately documented functions at several providers. They should have separately considered policies [5][6][7].

Named directives in 121 retrieved robots.txt files

Six files contained a root disallow for at least one tracked AI token. None met that screen for the five listed search/user tokens; path restrictions and network access require separate checks.

VayoMed technical crawl of 188 tracked MedTech websites — VayoMed analysis, accessed September 2026

Among the 121 robots files in the reachable panel, 13 explicitly name at least one tracked AI token. Six meet our root-disallow screen. Three name GPTBot, ClaudeBot and several other AI-related tokens with root disallows; the other three target Bytespider in that screen. No root disallow was detected for the five search/user tokens listed in the summary. Path-specific restrictions can still apply [1].

The saved Baxter, Fisher & Paykel Healthcare and Shinva files contain blocks labelled as Cloudflare-managed content. They are evidence of the directives served on the capture date, not evidence that three management teams independently chose identical policies or misunderstood the bots [8][9][10]. A company can reasonably limit training uses while keeping search discovery available. The audit did not interview these operators or measure their actual bot traffic.

Interpret the effective rules for the URL

The Robots Exclusion Protocol defines user-agent groups and path-matching rules. A crawler may use a matching specific group or fall back to a wildcard group. Multiple applicable groups, path lengths and allow exceptions matter. Robots.txt is not a security boundary; access control belongs in authentication and server policy [11].

Our chart uses a simpler question: does the applicable group include a root disallow without an allow-all rule? It is useful for screening explicit broad directives. It is not a complete evaluator of every wildcard, encoded path, more-specific exception or implementation difference. A site with a partial restriction could still block its most valuable product directory. A site without a retrieved robots file might have returned a transient server error that a particular crawler handles conservatively.

Before changing policy, select the actual product, reference and regional URLs that matter. Evaluate those paths for the intended bots and inspect server logs for genuine requests. An explicit allow group can also override restrictions that were previously inherited from a wildcard group. Review the resulting policy as a whole before copying an example into production.

What operator documentation supports

OpenAI separates GPTBot's training-use preference from OAI-SearchBot search eligibility. Its documentation says a site opting out of OAI-SearchBot is excluded from ChatGPT search answers, although navigational links can remain. ChatGPT-User supports user actions, and robots rules may not apply to those visits [5].

Anthropic distinguishes ClaudeBot, Claude-SearchBot and Claude-User and describes its robots compliance. Google's Google-Extended control covers Gemini training and certain grounding uses independently of Google Search. Perplexity distinguishes its search crawler from Perplexity-User, whose user-requested fetches generally ignore robots rules [6][12][7].

What each AI user agent controls, according to its operator

Choose training permissions separately from search discoverability. User-initiated retrieval has operator-specific controls; robots.txt is not an access-control system.

TokenPurposeEffect or limitation of robots disallow
GPTBotOpenAI trainingTraining-use preference; independent of ChatGPT search eligibility
OAI-SearchBotChatGPT searchOpted-out sites excluded from search answers; navigational links may remain
ChatGPT-UserUser-initiated visitsRobots rules may not apply; not a search-inclusion control
ClaudeBotAnthropic trainingHonors robots directives; manage separately from search
Claude-SearchBotClaude searchRestricts search crawling and may reduce search visibility
Claude-UserUser-requested visitsHonors robots directives for requested pages
Google-ExtendedGemini training and some groundingIndependent of Google Search, AI Overviews and AI Mode
GooglebotGoogle Search crawlingDisallow blocks crawling; it is not equivalent to noindex
PerplexityBotPerplexity searchAllow robots and verified bot traffic if search inclusion is desired
Perplexity-UserUser-requested visitsOperator says robots rules are generally ignored

OpenAI, Anthropic, Google and Perplexity crawler documentation, read September 2026

Keep the policy decision narrower than the marketing promise. Allowing a crawler makes a request eligible to proceed through that layer. It does not promise a citation. Blocking one provider's crawler also cannot erase information about the company from other sources or from prior model knowledge.

For Google, robots disallow and noindex have different jobs. Disallow restricts crawling, while an indexed URL may remain known through other links. A noindex instruction must be accessible to the crawler to be processed. Google's snippet controls govern how eligible page content can appear in its search features; they should not be presented as universal instructions honored by every AI service [13].

The useful handoff is a short policy record: permitted purpose, bot token, affected paths, verification method, policy owner and review date. Training permission can then be decided on its own merits, without confusing it with a guarantee of search inclusion or a way to protect confidential material published on an open website.

Home-page signals: improve description without inventing an AI requirement

The corrected detection rates show room for cleaner website metadata. They also show why the denominator and test matter. “No JSON-LD detected in the initial home-page response” is a reproducible observation. “No structured data anywhere on the website” would require a much broader crawl, including other structured-data formats and rendered templates.

Detected website signals and separately collected web references

JSON-LD was detected on 58 of 157 retrieved home pages after HTML reparsing. Different years, clients and sampling rules make Web Almanac a reference, not a matched control.

SignalTracked sampleWeb Almanac reference
robots.txt parsed121 / 157 (77.1%)85% HTTP 200; 2025 SEO
JSON-LD script detected58 / 157 (36.9%)43% of home pages; 2024 Structured Data
Organization-related JSON-LD type45 / 157 (28.7%)No matched reference
HTML hreflang link58 / 157 (36.9%)No matched reference
Root sitemap or robots declaration113 / 157 (72.0%)No matched reference
llms.txt text response17 / 157 (10.8%)2.13% desktop; 2025 GenAI
HSTS header90 / 157 (57.3%)36% mobile; 2025 Security
CSP header59 / 157 (37.6%)21.9%; 2025 Security

VayoMed technical crawl (September 2026) and HTTP Archive Web Almanac 2024–2025 — VayoMed analysis, accessed September 2026

Of 157 usable home-page responses, 99 had no detected JSON-LD script (63.1%). An Organization-related JSON-LD type was detected on 45; it was not detected on 112 (71.3%). Some of these sites may describe the same entity using microdata, RDFa, nested content rendered by JavaScript, or a different page. The audit covers initial HTML and the specific parser rules documented here [1].

HTTP Archive's 2024 Structured Data chapter reports JSON-LD on 43% of home pages [14]. It is a useful external reference, collected with a different methodology and at a different time. The 2025 SEO, Generative AI and Security chapters provide separate robots, llms.txt and security-header references [15][16][17]. We place these alongside the audit as a table so that missing comparisons and differing measurement scopes remain visible. They are not a matched control group from which to infer a MedTech performance deficit.

Make the company and product facts easy to verify

Organization markup can state the company name, URL and other supported properties. Use a type that describes the actual entity. A device manufacturer need not call itself a MedicalOrganization merely because it sells medical products. The markup should agree with visible content, and sameAs links should point only to verified identities [18][19].

For example, an illustrative organization block can be deliberately modest:

json
{
  "@context": "https://schema.org",
  "@type": "Organization",
  "@id": "https://example.com/#organization",
  "name": "Example Device Manufacturer",
  "url": "https://example.com/"
}

This example uses a reserved demonstration domain and contains no invented clearance, clinical indication or third-party profile. In a real deployment, an editor must reconcile legal name, trading name, regional entity and official product ownership before adding more fields. Markup makes a statement explicit; it does not independently prove that statement.

Schema.org also defines MedicalDevice, with medically relevant properties [20]. Product and medical-device markup should be evaluated on actual product templates, using supported properties and values. A home page without Product markup is not a failed product-page implementation. Our initial home-page sample was never designed to measure product-template coverage, so it supplies no product-page adoption rate.

For a buyer comparing two device models, visible content should identify the precise model and version, applicable territory, intended purpose, source document and its revision, relevant specifications, and a route to the correct local contact. Where regulatory status is stated, link to evidence for that jurisdiction and date. Do not ask a schema field to substitute for that work.

Use localization annotations where they apply

The crawl detected HTML hreflang links on 58 of the 157 home pages. Hreflang can help Google identify language or regional alternatives, and Google also supports declarations through HTTP headers and sitemaps. This audit checked home-page HTML links, so it can miss those other implementations [1][21].

A single-language website with no alternate version does not need a language-alternative map. Nor does hreflang validate local clinical indications or guarantee that an AI answer selects the right jurisdiction. On a multilingual site, first verify that the alternatives exist and offer the relevant local information. Then validate the language codes, canonical URLs and reciprocal references. Use x-default when a fallback page is appropriate; do not treat it as a universal pass condition.

Home-page signals by tracked category

Device companies had JSON-LD on 38/97 pages (39.2%), diagnostics on 11/36 (30.6%), and imaging on 5/17 (29.4%). The small categories support site-level follow-up rather than a sector ranking.

CategorySitesrobots.txtSitemapJSON-LDOrganization schemahreflangllms.txtHSTS
Medical Devices9776713831361052
Diagnostics36252211712417
Medical Imaging171313537114
Other life sciences77744327
All retrieved pages1571211135845581790

VayoMed technical crawl of 188 tracked MedTech websites — VayoMed analysis, accessed September 2026

The category breakdown helps focus an audit without grading an entire sector. Device companies have JSON-LD on 38/97 retrieved pages (39.2%); diagnostics on 11/36 (30.6%); imaging on 5/17 (29.4%). Imaging's 14/17 HSTS detections and 3/17 Organization detections describe different properties, not a contradiction or proof of budget priorities [1].

Our judgment is to improve the templates a visitor actually uses. If a company has one home-page schema block but broken country navigation and inconsistent model names, adding another markup type is unlikely to be the first useful release. The 40-market localization model addresses the separate question of which markets deserve that editorial investment [22].

What the stored source panels suggest about reference content

The domain panels and top-page records are different datasets. Keeping them separate changes the conclusion from “AI gives corporate websites only a quarter of citations” to a narrower, usable observation: many stored source entries point away from the tracked primary domain, while some official reference assets appear among stored top-page results.

Domain entries are a bounded sample

Each of the 144 nonempty source panels contains exactly five domain entries, and every stored citations field equals one. The 720-entry total therefore describes a fixed-width derived record set. We did not recover an uncapped denominator of all underlying answers or validate these lists against every original answer [3].

Domain matching inside 144 stored five-entry source panels

181 of 720 stored entries (25.1%) match the tracked primary domain or a subdomain. The remaining entries include external sources and unmerged corporate domains. This is not a share of all AI citations.

VayoMed AI Visibility dataset (June 2026 snapshot) — VayoMed analysis, accessed September 2026

Matching normalized hosts to each tracked primary domain and its subdomains yields 181 matches (25.1%) and 539 other-host entries (74.9%). The median panel has one primary-domain match out of five. “Other hosts” includes publishers, video services and clinical sources, but can also include country domains, product brands, subsidiaries or acquired businesses that the primary-domain rule does not merge. These percentages are not a corporate ownership share.

Recurring other-host entries in the stored source panels

YouTube appears in 124 panels, PubMed Central in 73 and English Wikipedia in 48. These are occurrences in bounded source lists, not traffic, unique answers or citation-frequency measurements.

VayoMed AI Visibility dataset (June 2026 snapshot) — VayoMed analysis, accessed September 2026

YouTube occurs in 124 other-host entries, PubMed Central in 73 and English Wikipedia in 48. The counts say which host names recur in these bounded panels. They do not identify the contents of every linked page, the number of unique user answers, the authority assigned by an engine, or referral traffic. Some stored results also reflect ambiguous brand terms. That is a reason to improve query and identity matching before evaluating a company's visibility.

The business use is source discovery. Inspect the actual pages behind the relevant panel for one product category, check whether they describe the correct company, and identify the factual question each page answers. A distributor listing, a journal paper and a manufacturer instruction document play different roles. Treating all three as interchangeable “citations to win” obscures that difference.

The top-page sample contains reference documents

The separate February snapshot contains 110 stored top-page records for 11 brands, with 37 URLs matching their tracked primary domains. An unweighted URL-path classification finds 15 PDF URLs, 12 news/blog/article paths, two patient-education paths and eight other paths in those 37 matches [3].

URL-path classes in 110 stored top-page records

15 of the 37 primary-domain matches are PDF URLs. None matches the product-path rule; eight remain other. Path-based labels can miss product pages and do not establish which format earns more citations.

VayoMed AI Visibility dataset (February 2026 snapshot) — VayoMed analysis, accessed September 2026

No primary-domain match triggers the product-path rule. This is not proof that product pages earn zero citations: product content can live at unrecognized paths, and a top-page list omits the long tail. The sample also includes pharmaceutical and diagnostic brands. It is not a census of device-manufacturer product pages.

Useful examples in the stored records include Medtronic reimbursement guides, Lilly prescribing information and instructions, and Quest Diagnostics preparation information. Their existence helps choose pages for inspection. The records do not establish why a model selected them or that a newly published HTML version would outperform the PDF [3].

Keep the reference asset, improve the route to it

PDFs are already visible in the sample, which is counter-evidence to any blanket claim that AI systems cannot read them. Where extraction or navigation is poor, a companion HTML page can help human readers reach and understand the authoritative document. It should preserve the source document, revision date and jurisdiction, with a clear link to the complete version.

For an IFU, the goal is controlled access to current instructions, not an improvised clinical summary. Have the responsible reviewer define which information can be reproduced, how the full approved document remains available, and how both formats will be updated. If an authorized PDF changes, a stale HTML excerpt creates a second maintenance problem.

Choose one reference asset using existing demand, support questions and applicable rights. Inspect text extraction and table order; check that the URL is stable; add a meaningful internal link from its product page; and show the document's purpose and revision before the download. Only then decide whether a parallel HTML representation solves a real reader problem.

Delivery failures deserve a separate investigation

The crawl classified 31 of 188 domains as not returning usable home-page content to this client. Sixteen were HTTP 403 or manually identified challenge responses; seven were connection failures; eight were other errors or empty responses [1].

How 188 MedTech sites answered a plain browser-identified request

31/188 domains did not return a usable home page to this client. Only 16 were classified as HTTP 403 or challenge responses; seven connection failures and eight other responses had different causes.

VayoMed technical crawl of 188 tracked MedTech websites — VayoMed analysis, accessed September 2026

The 16-response group represents 8.5% of the full panel. It would be inaccurate to call all 31 failures firewall blocks. A failed connection can reflect DNS, transport, routing, timeout or service conditions. HTTP 403 is an access refusal, but its status code alone does not identify the particular policy that caused it.

The saved Intuitive response is an Imperva/Incapsula interstitial with a noindex meta instruction. Terumo illustrates a different separation: robots.txt was retrieved while the home-page request returned 403. Neither observation proves that the companies block verified search-engine IP addresses. The requests came from our client, and a properly configured bot-management system can treat genuine search crawlers differently [1].

Cloudflare's July 2025 announcement described changes to AI crawler controls for new customers [23]. Its 2025 Radar report measures crawler traffic on Cloudflare's own network [24]. Those sources explain why edge policy warrants attention. They do not identify the cause of every failure in our MedTech sample, or the revenue impact of allowing a particular bot.

A useful investigation combines application and security evidence. Obtain the final response body for the important URL, review the security event at that time, and compare genuine bot requests with the operator's documented verification method. Perplexity's guidance, for example, combines the user-agent with its published IP ranges [7]. An allow rule based only on an easily copied user-agent string is weak identification.

Keep the scope of any exception narrow and check access to public documents after the change. Sensitive customer or clinical information belongs behind access controls. Allowing a verified crawler to read public content should not create access to authenticated material.

Where llms.txt fits

Seventeen retrieved sites returned a non-HTML text response at llms.txt that passed the original length check (10.8%). That detector establishes a text response, not semantic compliance with the proposal. The llms.txt proposal describes a curated information directory for language-model use [1][25].

Treat it as an optional experiment with an owner and a maintenance cost. The operator documentation reviewed for this report does not establish a general search-inclusion requirement for it. Google explicitly says no new AI text file is needed for its AI search features [2]. A useful existing reference index should not be delayed while a team debates whether to create this additional file.

A release checklist that separates detection from validation

The audit's percentages are detection rates. They are not pass rates against the stronger checks below. In particular, finding a JSON-LD script says nothing about factual completeness, and finding a Sitemap declaration says nothing about the health of every URL in the file.

CheckEvidence to collectWhat a useful result establishes
Retrieve the key pageFinal status, body and redirect chain from a GET requestThe requested content arrived for that client
Check actual bot trafficVerified crawler identity, access logs and security eventsWhether the intended service reaches the relevant public URLs
Evaluate robots rulesApplicable user-agent groups and specific page pathsWhich cooperative crawling requests policy permits
Inspect index controlsRobots meta, HTTP directives and canonical URLWhether intended indexing and snippet controls are consistent
Follow sitemap discoveryRoot response or robots declaration, then fetched XML and sample URLsThe sitemap is usable and contains the intended canonical pages
Inspect initial and rendered textModel, territory, reference links and factual answersImportant information is available beyond interactive presentation
Validate entity markupParsed JSON, supported types and agreement with the pageThe explicit entity statement is technically valid and accurate
Inspect product templatesApplicable product fields and primary-source linksThe model and jurisdiction can be verified on the correct page
Validate language alternativesExisting alternate URLs, codes and reciprocal relationshipsThe language map describes real alternatives
Check reference documentsRevision, access, extracted text and download linksThe current authoritative document is findable and usable
Review security headersDeployed transport and content-security configurationHeader policy matches the site's security requirements
Repeat the buyer query panelSame prompts, model/service, locale, date and stored answersChanges can be compared on an explicitly bounded basis

Start with a GET request, not a header-only check. A 200 response can still contain a challenge or a soft-error page. The following example is a retrieval aid rather than a complete audit:

bash
curl --location --max-time 30 --dump-header page.headers \
  --output page.html https://example.com/products/example-device

Inspect page.html and the final response headers. Use a browser to compare rendered content, then use the relevant search-console tools and verified logs to investigate actual indexing and crawler behavior. Changing a curl user-agent to a bot name does not reproduce that service's network identity.

For structured data, parse the entire JSON document and recurse through nested objects, rather than searching only for one quoted script attribute or a top-level type. Our own correction from 56 to 58 JSON-LD detections shows why a seemingly small parser choice can change the published benchmark [1].

What to fund first

Choose the next release from the observed failure, with an owner and acceptance evidence. A site that delivers a challenge page needs a different first task from a site that is accessible but lacks a current reference document. The data does not justify fixed industry-wide implementation times or promised citation returns.

Observed problemFirst deliverableEvidence before calling it done
Verified crawler cannot reach a key public URLTargeted delivery or bot-policy correctionGenuine bot logs and successful content retrieval
Buyer cannot verify model or territoryRevised product page with current source linksProduct/regulatory review and rendered-page inspection
Useful PDF is poorly linked or difficult to readClear reference landing page and maintained downloadCorrect revision, readable extraction and navigation
Existing language variants conflictCorrected content and alternate-link mapLocale review, canonical and reciprocal-link checks
Entity markup is invalid or missing where usefulAccurate shared template updateParser validation and agreement with visible company facts
Visibility measurement mixes identities or capped listsFixed query and source-record definitionsStored prompts, raw answers, inclusion rules and clear denominators

VayoMed's judgment is to connect the technical and editorial work in one small release. Select the product page, its reference asset and relevant local version together. Approve the facts, fix the delivery path, and record the before-and-after measurement. If the page already works, do not rebuild it merely to improve a tool's schema score.

The same discipline applies to content distribution. The press-release distribution benchmarks examine evidence from actual campaigns, while AI Recommends Life Sciences explains the limits of capped recommendation panels [26][27]. Neither visibility nor distribution can substitute for accurate product evidence on the company's own website.

For teams needing implementation support, VayoMed's Done-for-You service combines website delivery and operations, LinkedIn, domain management, source-grounded content and PR distribution. The service can implement and maintain this work; it does not guarantee that a particular engine will cite or recommend a device. A useful initial handoff is the priority product URL, intended market, current reference documents and the buyer questions the site should answer.

Methodology and limits for interpreting the results

The crawl is a single-date, single-network snapshot of a selected tracking cohort. The 157-page denominator excludes failed or manually identified challenge responses. Their unobserved markup could have either higher or lower adoption, so there is no justified directional correction to an industry-wide rate. The seven businesses outside the three main categories remain in the full panel and are shown separately in the category table [1].

JSON-LD detection used an HTML parser followed by JSON parsing and recursive type inspection. Organization-related detection covers the enumerated Organization, Corporation, MedicalOrganization and MedicalBusiness types. Other schema types, microdata, RDFa, dynamically injected markup and deeper templates remain outside that measure. The crawl detects headers and HTML hreflang links, without validating security-policy strength or multilingual completeness.

Sitemap discovery combines an XML response at the root with a Sitemap declaration in robots.txt; declared endpoints were not all fetched. The robots chart reproduces a root-rule screen and selected named tokens, rather than full RFC conformance or real-engine access testing. Raw responses and reproducibility code support the internal review, while reader-facing charts show the aggregate findings.

The June source-domain panels contain five entries for each of 144 brands. Every stored citations value is one; consequently we count entries and avoid claims about all answer citations, impressions or click-through. Primary-domain matching can miss alternate corporate domains. The February top-page dataset covers 11 brands and 110 records; URL-path classification is heuristic and records are unweighted. Neither dataset is joined to September page delivery or markup to estimate an association, much less a causal effect [3].

These boundaries leave a useful result: a reproducible set of website checks, a distinction between crawler functions, and a concrete reference-content inspection path. The next evidence should come from the company's own important URLs and a controlled buyer-query panel, not an extrapolation from our cohort to every MedTech website.

Sources

1. VayoMed, original technical crawl of 188 tracked life-sciences domains, 5 September 2026; HTML reparsed during independent publication review. Counts describe the frozen requests and detection rules documented in this report.

2. Google Search Central, AI features and your website, accessed September 2026 - Official guidance explaining indexing prerequisites and snippet controls for Google AI Overviews.

3. VayoMed, AI Visibility Reports, stored source-domain panels dated 25 June 2026 and top-page records dated 16 February 2026; VayoMed analysis. Five source-domain entries per included brand; no population-wide citation denominator.

4. VayoMed, AI Visibility Reports, accessed September 2026 - Company benchmark platform; selected tracking panel and category definitions are described in this report.

5. OpenAI, Overview of OpenAI crawlers, accessed September 2026 - Official technical documentation defining operational roles for OAI-SearchBot, GPTBot, and ChatGPT-User.

6. Anthropic, Does Anthropic crawl data from the web, and how can site owners block the crawler?, accessed September 2026 - Official documentation defining ClaudeBot, Claude-SearchBot, Claude-User, and robots.txt compliance.

7. Perplexity, Perplexity crawlers, accessed September 2026 - Technical documentation defining PerplexityBot indexing and Perplexity-User real-time fetch behaviors.

8. Baxter, robots.txt, as served 5 September 2026 - Saved robots exclusion file; root-rule screen and managed-content block examined in the original crawl.

9. Fisher & Paykel Healthcare, robots.txt, as served 5 September 2026 - Saved robots exclusion file; root-rule screen and managed-content block examined in the original crawl.

10. Shinva, robots.txt, as served 5 September 2026 - Saved robots exclusion file; root-rule screen and managed-content block examined in the original crawl.

11. IETF, RFC 9309: Robots Exclusion Protocol, September 2022 - Internet standard defining parsing syntax, record grouping, and longest-match path evaluation for robots.txt.

12. Google Search Central, Google's common crawlers, accessed September 2026 - Technical specification detailing Google-Extended, Googlebot, and model training controls.

13. Google Search Central, Robots meta tag, data-nosnippet, and X-Robots-Tag specifications, accessed September 2026 - Developer documentation governing fine-grained snippet extraction controls and indexing parameters.

14. HTTP Archive, Web Almanac 2024: Structured Data, November 2024 - Global web benchmark documenting JSON-LD presence on 43.0% of desktop and mobile home pages.

15. HTTP Archive, Web Almanac 2025: SEO, November 2025 - Global crawl statistics recording robots.txt HTTP status distributions, AI crawler mention frequencies, and llms.txt adoption rates.

16. HTTP Archive, Web Almanac 2025: Generative AI, November 2025 - Benchmark study analyzing robots.txt AI directives across 12.9 million websites and enterprise adoption trends.

17. HTTP Archive, Web Almanac 2025: Security, November 2025 - Web security benchmark measuring HSTS adoption, CSP implementation rates, and SSL configuration across mobile and desktop.

18. Schema.org, Organization, accessed September 2026 - Standard entity schema specification defining corporate identity properties and authority linkages.

19. Google Search Central, Introduction to structured data markup in Google Search, accessed September 2026 - Official developer specification recommending JSON-LD for machine-readable web annotations.

20. Schema.org, MedicalDevice, accessed September 2026 - Vocabulary and inherited properties for describing a medical device; validate each proposed property against the specification.

21. Google Search Central, Tell Google about localized versions of your page, accessed September 2026 - Technical specification governing hreflang annotations for international targeting.

22. VayoMed, Which Markets Should MedTech Localize First? A 40-Market Model (2026), 2026 - Strategic localization model ranking forty international healthcare markets for digital expansion.

23. Cloudflare, Cloudflare Just Changed How AI Crawlers Scrape the Internet-at-Large, 1 July 2025 - Corporate announcement establishing default AI crawler blocking for new customers and pay-per-crawl mechanisms.

24. Cloudflare Blog, Cloudflare Radar 2025 Year in Review, December 2025 - Global internet traffic study detailing AI bot request volumes, crawl-to-refer ratios, and scraper trends.

25. llmstxt.org, The /llms.txt file, accessed September 2026 - Open proposal document outlining a standardized markdown file format for large language model ingestion.

26. VayoMed, Medical Device Press Release Distribution Benchmarks 2026, 2026 - Research report on published press-release campaigns and distribution benchmarks.

27. VayoMed, AI Recommends Life Sciences 2026, 2026 - Category-level citation analysis identifying sources behind generative AI product recommendations.

DJ Fang

DJ Fang

Founder @ VayoMed, RAC

DJ is a Regulatory Affairs Certified (RAC) professional with deep expertise in life sciences go-to-market strategy. He helps medical device and healthcare companies navigate the intersection of regulatory compliance and digital visibility, ensuring brands are positioned for success in both traditional and AI-powered search environments.

Next step

Need the engine that comes after clearance?

VayoMed builds and runs your website, content, LinkedIn, domains, PR, and AI visibility as one fully managed $24K/year annual subscription.