This is the technical companion to The AI Visibility Debate Is Starting at the Wrong End. It separates the planned protocol, the fields recorded during collection, and the rules used in the final analysis.
Prefer the complete narrative and exhibits? Download the full 19-page research report.
How to interpret the findings
Three details define the scope of the findings:
- H2 should be read as less confirmatory than H1. We had not inspected individual rows, but a live progress display showed an unreadable rate of roughly 17% when the 15% threshold was set.
- The applied analysis uses the fields available in the original crawl. The title-presence field was not stored, and the direct
blocks_ai_crawlersfield was blank. We used the pre-specified word threshold and recovered the crawler-access subgroup from the recorded crawlability component, without recrawling. - The scores do not measure citation. They measure traditional SEO checks and a page’s readiness for non-rendering retrieval and reuse. Source selection by an AI engine is outside the study.
We retained every row, report all pre-specified subgroups, and publish the data needed to reproduce the numerical findings.
1. Research question
Does traditional SEO quality predict whether a page is readable and usable by a non-rendering AI system?
The study focuses on delivery and extractability. It does not ask whether an AI engine will select, rank, quote, or cite the page.
2. Hypotheses and results
| Hypothesis | Falsification rule | Result |
|---|---|---|
| H1: SEO quality does not strongly predict AI extractability among qualifying content pages. | Falsified if Spearman’s ρ ≥ 0.4 and the lower bound of the 95% bootstrap CI is also ≥ 0.4. | Not falsified. ρ = 0.264, 95% CI 0.175–0.351, n = 476. |
| H2: At least 15% of domains returning HTML provide fewer than 50 body words. | Falsified if the observed share is below 15% and the upper bound of its 95% CI is also below 15%. | Not falsified by the stated rule. 100/576 = 17.4%, 95% CI 14.5–20.7%. |
H1 does not say the scores are unrelated. The confidence interval excludes zero, so the association is real in this sample; it is simply weak. A squared-correlation approximation gives roughly 7% shared variance.
H2 also needs restraint. Its interval crosses 15%, and the live progress display provided context about the running unreadable rate when the threshold was set. We therefore treat H2 as less confirmatory than H1. At the published sensitivity thresholds, 19.8% of HTML responses fell below 100 words and 24.7% fell below 200 words.
The roughly 50% raw ok rate was not a hypothesis. It combines the behavior of websites with dead domains, infrastructure endpoints, bot walls, and network failures, so we report it descriptively rather than treating it as a single AI-readiness result.
3. Sample
| Field | Value |
|---|---|
| Source | Tranco top one million, list 5674N |
| List generated | August 30, 2026 |
| Aggregation window | August 1–30, 2026 |
| Ranking method | Dowdall rule, pay-level domains |
| Rank strata | 1–1,000; 1,001–10,000; 10,001–100,000; 100,001–1,000,000 |
| Sample per stratum | 250 domains; 1,000 total |
| Random seed | 20260831; each stratum used seed + stratum.min |
| Page requested | Homepage only |
The list permalink and seeded sampling rule make the sample reproducible. Because Tranco ranks pay-level domains rather than only human-facing websites, the sample includes advertising, CDN, identity, and other infrastructure domains.
4. What we measured
Primary: observable response signals
- Request outcome and status
- Body word count
- HTML response size
- Signs that client-side rendering might explain an empty response
- Presence of JSON-LD
- robots.txt rules affecting named AI crawlers
- Pass, partial, or fail status for each audit check
These signals are the foundation of the findings because they can be checked directly against a response.
Secondary: composite scores
The SEO Score and AI Visibility Score are AIO Mapper measurement instruments. They are useful for structured comparisons, but they are not neutral facts and are not the primary evidence for the category-level claim.
5. Analysis plan
Correlation
We used Spearman rank correlation with a 95% confidence interval from 10,000 seeded bootstrap resamples. The two 0–100 scores contain ties and are not expected to have a normal, linear relationship, making Pearson’s r a poor fit.
Pages without valid scores appear in the raw outcome table but are excluded from the primary correlation. As a sensitivity check, we assigned unreadable pages an AI score of zero; the result agreed with the primary calculation to three decimal places.
Applied content threshold
The planned content-page rule required an HTML response, a <title>, and at least N = 50 body words. The title field was not stored in the crawl output, so the applied rule used HTML plus the word threshold only.
We borrowed the 50-word lower bound from the document-length filter described for MassiveText in the Gopher paper. We used it as a permissive way to separate substantive responses from near-empty ones—not as a universal standard for useful or high-quality content.
We repeated the analysis at N = 100 and N = 200 to show how the result changes under stricter definitions.
Confidence intervals
- Proportions use 95% Wilson score intervals.
- Spearman correlations use 10,000 seeded bootstrap resamples.
- Group means are descriptive unless explicitly identified otherwise.
Pre-specified subgroups
We reported all four planned comparisons:
- Tranco rank band
- Content versus infrastructure classification at each word threshold
- Presence versus absence of JSON-LD
- Whether robots.txt blocked at least one named answer-time AI crawler
Exclusions
None. All 1,000 requested domains remain in the published dataset with their recorded outcomes.
6. Collection
- Crawler:
scripts/research-crawl.ts - User agent:
AIOMapperResearch/1.0 (+https://www.aiomapper.com/research) - Requests: one per domain, with no recrawl for the analysis
- Rendering: JavaScript was not executed
- robots.txt: checked before the homepage request; a 5xx robots response was treated as a refusal
- Run window: September 5, 2026, from 11:38:27 to 12:08:29 UTC
Following robots.txt was specific to this unsolicited research crawl. A user-initiated audit of their own page is a different context and may use different retrieval behavior.
7. Raw crawl outcomes
| Recorded outcome | Count | Share |
|---|---|---|
ok | 501 | 50.1% |
unreadable | 75 | 7.5% |
robots_disallowed | 43 | 4.3% |
fetch_failed | 381 | 38.1% |
These are collection outcomes, not four judgments of website quality. In particular, fetch_failed combines DNS failures, timeouts, infrastructure domains, rate limits, bot protection, and other unsuccessful requests.
Across the 576 rows that returned HTML, the applied word thresholds produced:
| Threshold | Below threshold | Share | 95% CI | Qualifying pages |
|---|---|---|---|---|
| 50 words | 100 | 17.4% | 14.5–20.7% | 476 |
| 100 words | 114 | 19.8% | 16.7–23.2% | 462 |
| 200 words | 142 | 24.7% | 21.3–28.3% | 434 |
8. Limitations
- Extractability is not citation. The study does not observe source selection, answer generation, rankings, or citations in an AI product.
- Only homepages were tested. Results may differ for articles, documentation, product pages, or other templates.
- The crawler made one non-rendering request. The result is a point-in-time observation, not a claim about every user agent, geography, retry, or browser session.
- Missingness is not random. Bot-protected and unreachable domains may differ systematically from scored pages.
- The composite scores are product-defined instruments. Two of the nine AI components showed very little variation in dry runs. Observable response signals deserve more weight than the score alone.
- The restricted-crawler group is small. The 26.3-point mean AI-score difference is descriptive and should not be read as a causal treatment effect.
- The H2 threshold was set with context from the live progress display. Its result is less confirmatory than H1.
9. Method notes
| Planned approach | Applied approach |
|---|---|
| Set H2’s threshold before reviewing the final analysis. | The 15% threshold was set while a live progress display showed a running unreadable rate of roughly 17%; individual rows had not been inspected. H2 is therefore treated as less confirmatory than H1, whose correlation had not been calculated or displayed. |
Define a content page using a <title> plus N body words. | The crawl output stored body word count but not title presence. We used HTML plus N words without recrawling, making the qualifying group slightly more permissive than planned. |
| Report 95% intervals for proportions. | The plan did not name an interval method. We applied Wilson score intervals and state that choice explicitly. |
Compare pages by the direct blocks_ai_crawlers field. | That field was blank in the published output. We recovered the subgroup from aiomapper_ai_bot_crawlability < 100, which indicates that at least one answer-time crawler was not allowed. |
| Assess client-rendering evidence where missing text required an explanation. | We report this measure only within the 75 unreadable responses. Other pages are left unclassified on this measure rather than being treated as “not client-rendered.” |
10. Reproduce the study
Download the complete CSV. It contains all 1,000 rows, including failures.
The repository analysis command is:
TS_NODE_COMPILER_OPTIONS='{"module":"CommonJS","moduleResolution":"Node"}' \
node -r ts-node/register -r tsconfig-paths/register \
scripts/research-analyze.ts data/study-2026-09.jsonl
The command should report 1,000 rows, 576 HTML responses, 100 below the 50-word threshold, and 476 qualifying pages for the primary analysis.
The sample uses Tranco list 5674N. For the source-list methodology, see Le Pochat et al., “Tranco: A Research-Oriented Top Sites Ranking Hardened Against Manipulation”.