SaaS product
LucidCrawl
A web application that crawls public websites and audits what AI systems can access, understand, trust and answer. Every finding links to captured evidence.
- Context
- Independent product
- Date / status
- 2026
- Work
- SaaS product
The problem
Traditional SEO tools do not show which answers a website can support for AI systems. Audit findings also need to be traceable to the exact pages and passages that produced them.
Scope and architecture
My responsibilities
- Product definition and UX
- Vue.js console and reporting interface
- Go API, worker and crawler design
- PostgreSQL schema, deployment and operations
System
- Vue.js and TypeScript web console
- Go modular monolith with separate API and worker processes
- PostgreSQL for audits, evidence and credit ledger
- Docker and Caddy production stack on Hetzner
Key decisions
- 01
Store evidence before producing an interpretation.
Every finding and answer points back to a captured URL and passage. The product can explain what it found, what it could not prove and why a recommendation exists.
- 02
Treat crawling as an untrusted network operation.
Targets and redirects are revalidated against private networks while page count, depth, response size, render time and same-origin discovery stay bounded and visible in the report.
- 03
Separate request handling from durable work.
The API accepts and validates work while a separate worker claims audit jobs with renewable leases. Atomic credit reservation and idempotent settlement make retries recoverable.
What I built
I designed and built the product with a Vue and TypeScript frontend, a modular Go API and worker, PostgreSQL, safe web crawling, browser capture, deterministic analysis, versioned reports, Answer Lab, Fix Studio and a credit ledger. It runs in Docker behind Caddy on Hetzner.
- Go
- Vue.js
- PostgreSQL
- Docker
- Web crawling
Result
Live web application with evidence-backed reports and fixes
- Audit findings linked to captured page and passage evidence
- Grounded answers, prioritized fixes and versioned report comparison
- Production deployment with separate API, worker and PostgreSQL services
Contact
Have a project in mind?
Tell me what you’re building, where it stands and what you need help with. LinkedIn is the best place to reach me for now.
Start a conversation