How LucidCrawler Works
Everything you need to understand the scoring methodology and get the most out of your AI readiness reports.
What is LucidCrawler?
LucidCrawler is an AI readiness scanner that analyzes public websites and scores them on how well they can be understood, parsed, and extracted by AI systems — including large language models, search AI agents, and automated web crawlers.
As AI-powered search and retrieval becomes the default way people find information, websites that aren't structured for machine readability will be left behind. LucidCrawler gives you a concrete score and actionable fixes so you know exactly where to improve.
Scoring Categories
Every scan evaluates six independent dimensions of AI readiness.
Structured Data
Checks for JSON-LD, Microdata, and RDFa markup. Structured data helps AI systems understand the semantic meaning of your content — product info, articles, organizations — without ambiguity.
Semantic HTML
Evaluates use of HTML5 landmark elements: <article>, <nav>, <main>, <section>, <aside>, and proper heading hierarchy. Semantic structure lets AI crawlers navigate and extract content accurately.
Metadata Quality
Analyzes <title>, <meta description>, Open Graph, and Twitter Card tags. Rich metadata gives AI systems context about each page before they parse body content.
Content Clarity
Measures text-to-HTML ratio, paragraph density, and readability. Clear, dense prose is easier for language models to extract and summarize than content buried in markup.
Accessibility
Inspects alt text, ARIA labels, form labels, and language attributes. Accessibility signals double as machine-readability signals — what's clear to screen readers is clear to AI.
AI Extractability
Looks for canonical URLs, robots directives, hreflang, and link structure. These signals tell crawlers what to index, how content relates, and what the authoritative version is.
How Scoring Works
Each of the six categories is scored independently on a 0–100 scale. The overall AI Readiness Score is a weighted average of all six categories.
Excellent
90–100
Good
70–89
Needs Work
50–69
Critical
0–49
Issues within each category are classified as Critical, High, Medium, or Low severity, and each carries a point penalty proportional to its impact.
Getting Started
- 1
Scan a URL
Go to the Scan page, enter any public URL, and click "Scan Now". The analysis typically takes 5–15 seconds.
- 2
Review Your Results
The results page shows your overall score, per-category breakdown, and a prioritized list of issues with detailed explanations.
- 3
Fix Issues
Each issue includes a description of the problem and what to do about it. Start with Critical and High severity items for the biggest score gains.
- 4
Re-scan After Changes
Once you've made improvements, run another scan to verify your score improved. Track your progress over time in Scan History.