Back to Blog

Technical readability: what does a retriever receive?

Check technical readability: complete HTML, accurate HTTP status codes and consistent crawler controls.

Editorial review:

Reviewed factual claims and sources; added distinct examples and acceptance criteria. Observations, methodological recommendations and unconfirmed causes are distinguished.

TL;DR: Key Takeaways

An HTTP 200 response is not enough. The response must actually contain the service, heading, conditions and contact path.

  • Initial response: No service information in the response body yet.
  • Server-rendered content: The essential statement is available without a script.
  • Failure case: Incomplete page; fix the error and return the appropriate status.

Inspect the HTTP response body first, then the browser view. Server-rendered service text is immediately available. An empty shell that loads data through JavaScript depends on the retriever’s capabilities. Google can process JavaScript; that does not establish identical capabilities across search and AI services.

Context and scope

Consent walls, blocked scripts and failed API requests can change the accessible content. A 200 response from an interrupted PHP template is not a successful page. Check content, status, canonical URL, indexability and internal links together.

Distinguish documented bots for search, user-triggered retrieval and model training. robots.txt guides cooperative crawlers but does not protect confidential data. Private content needs actual access control. Make training decisions separately from desired search visibility.

SOURCE/01 records rendering dependencies, blocked agents, incorrect status codes and conflicts between sitemap and canonical URLs. A complete service page should remain usable without animation or optional analytics.

Example: a visible card or delivered information?

Fictional before-and-after test: a page initially returns an empty element. JavaScript later displays a service card. In the improved version, the first response already includes the heading and text; JavaScript only adds an expandable detail view. The browser views may look similar, but their failure dependencies differ.

Review pointExample / subjectInterpretation
Initial response<div id="services"></div>No service information in the response body yet.
Server-rendered content<section><h2>Office cleaning</h2><p>Area and scope …</p></section>The essential statement is available without a script.
Failure caseHTTP 200 containing “Fatal error”Incomplete page; fix the error and return the appropriate status.

How to apply this

  • Save the canonical URL’s HTML, status and headers.
  • Disable JavaScript and check essential content and links.
  • Check sitemap, robots.txt and canonical declarations for conflicts.
  • After a fix, test the complete page rather than HTTP 200 alone.

Sources and further reading

The next step

Next step

The free short check provides an initial assessment of one domain, three buying questions and three observations. An introductory call establishes whether deeper analysis is useful.

Request an introductory call

Frequently Asked Questions

Are JavaScript websites inherently invisible?

No. Processing capabilities depend on the service. Server-rendered core content reduces avoidable dependencies.

Does robots.txt protect internal documents?

No. A Disallow rule is not access control.

Why check content as well as status?

A template can fail after output begins while still returning a success status. A content check detects the incomplete response.