I keep seeing the same “innovation” pitch: scrape your own site (or half the internet), stuff it into a chatbot, park it on ai.yourwebsite.com, and call it the future. Why would you deliberately build something that competes with your main website? That is not smart. That is a second front door that steals attention from the one you already paid to make work.
Entities are not a side quest. Your brand is becoming a file machines read — that essay — and the canonical place for that truth should strengthen the primary property, not open a rival tab that confuses buyers and bots.
What goes wrong with the scraped AI subdomain
- You compete with yourself. Answer engines and customers now have two URLs to trust. Rankings, citations, and brand queries split. You paid for a revenue engine and then built a diversion — engines, not brochure twins.
- Scraped is stale by definition. If the chatbot only knows what a crawler last saw, it is a funhouse mirror of your site — late, lossy, and proud of it. Canonical facts belong in structured sources you own ( how to ship entity.md, /entity.md), not in a scrape residue.
- It teaches the wrong architecture. “AI lives over there on a subdomain” trains the company to bolt intelligence on the side instead of into the layer — bolt-on vs. built.
- Cardboard risk. Seeded or half-scraped corpora fail the reality test — is this data real?
When people sell you this anyway
When a vendor needs a demo that looks like magic in fifteen minutes. When an agency wants a new monthly line item without touching your real conversion paths. When someone confuses “we have a chatbot URL” with “we have an AI strategy.” Make them open the curtain — one infrastructure question — and ask where the entity source of truth lives when the subdomain disagrees with the homepage.
What smart looks like instead
One primary web property, fully optimized for the pages it has — launch complete, not forever-fog SEO. Entity files, llms.txt, and schema that make the main site legible to machines. Agents and answer experiences that read the same canonical facts your humans publish — not a scraped shadow. Intelligence in the operating layer; the site as the trusted surface, not a competitor you invented for yourself.
Professionals use AI to build the tool that serves the business. They do not rent a second website that argues with the first.
Failure modes if you already shipped the subdomain
- Support hears “your AI said X” while the service page says Y.
- Ads send traffic to the main site; the chatbot cites last year’s pricing scrape.
- You fund two content programs and neither compounds.
Fix: pick a canonical home. Fold the useful answers back into the primary entity layer. Retire the rival door — or demote it to an internal lab, not a public competitor.
Machine customers deserve one coherent entity story — the same facts on the homepage, in llms.txt, in schema, in the agent. A scraped subdomain is a second story that drifts. Drift is how brands lose answer-engine trust one contradiction at a time.
If you already bought the subdomain experiment, treat it as a lab notebook — not a second brand. Migrate the true answers into the primary entity layer, then turn the rival door off before it trains Google, Perplexity, and your customers to argue with your homepage.
Action
If a partner proposes ai.yourdomain.com on scraped pages, ask: “How does this help the main site win — and what happens when the two disagree?” If they cannot answer without a logo reel, walk. For entity work done right, start with /entity.md and the framework essay. For a partner who builds revenue engines instead of shadow sites, Work with Jennifer. For the stage version of this argument, book Jennifer.