GitHub Copilot Suggests Methods That Don't Exist: How to Catch Hallucinated APIs

GitHub Copilot suggests methods that do not exist because it generates the most plausible next code, and a plausible name is not always a real one. The suggestion fits the rest of the file and fails only when it runs: TypeError: x is not a function, a module that cannot be found, or a 400 from an API that never had that field. You catch these by checking packages against your lockfile and the registry, running strict type checks, giving Copilot the real API through instructions files and MCP, and validating external API calls against the real schema before they are sent.

This article covers the three kinds of hallucinated APIs, what research says about how often they happen, why Copilot produces them, a checklist for catching them, and how to set up Copilot's agent mode so invented API calls fail before they run. Checked against GitHub and VS Code documentation and published research in October 2026.

Three kinds of hallucinated APIs

How often do code models hallucinate APIs?

The largest study of package hallucination, "We Have a Package for You!" by Spracklen and colleagues (arXiv 2406.10279), generated 576,000 code samples in Python and JavaScript with 16 popular models. On average, at least 5.2% of packages suggested by commercial models and 21.7% of those suggested by open-source models did not exist. The authors counted 205,474 unique hallucinated package names.

That matters beyond broken builds. If a model repeats the same invented name, an attacker can publish a package under it, and the next developer who accepts the suggestion installs the attacker's code. The authors describe this as a new form of package confusion attack.

For tool and API calls, the ParamBench study (arXiv 2608.03071, 2026) found that only 2.1% of failed calls from seven frontier models violated the schema. Most failures were well-formed calls with wrong values. Generated code that looks structurally correct is no evidence that it uses the API correctly.

Why Copilot suggests methods that don't exist

• It predicts code from patterns. A method named findByEmail on a user service is likely in general, whether or not your service has it.
• Training data mixes versions. Methods renamed or removed in recent versions still appear in older public code.
• Context is partial. Inline completions see the open file and nearby context, which may not include the installed type definitions or the API's current reference.
• Similar libraries blur together. A method from one HTTP client or ORM gets applied to another with a similar API.
• Agent mode raises the stakes. In agent mode, Copilot can run terminal commands and call MCP tools, so an invented call can execute instead of waiting in a diff for review.

Checklist: catching hallucinated APIs in Copilot suggestions

1. Check every new import against the lockfile. If the package is not already a dependency, treat it as unverified.
2. Before installing an unfamiliar package, check its registry page for age, maintainers, download history, and source repository.
3. Turn on strict type checking (TypeScript strict mode, or mypy or Pyright for Python) and run it in CI, so invented methods on typed libraries fail the build.
4. Confirm method names and options in the documentation for the exact version in your lockfile.
5. Write a test that runs the code path against a sandbox or test account before merging.
6. For REST calls, compare the path, method, parameters, and body against the provider's API reference or OpenAPI spec.
7. In agent mode, require approval for terminal commands and tool calls that write data.
8. Validate external API calls against the real schema at execution time, so anything the review missed fails before it is sent.

GitHub's documentation advises reviewing and testing Copilot's suggestions before accepting them. The checklist turns that advice into checks that run whether or not someone remembers.

Give Copilot the real API

Copilot reads repository instructions from .github/copilot-instructions.md. VS Code also supports AGENTS.md and path-specific .instructions.md files with an applyTo glob. Use them to point Copilot at the source of truth for external APIs.

<!-- .github/copilot-instructions.md -->
## External APIs

- Do not add a dependency that is not already in the lockfile without asking.
- Check the installed version before using any SDK method.
- For third-party API calls, use the Swytchcode MCP tools: discover the
  operation, read its schema with info, then exec. Preview writes with
  dry_run=true.
- Never invent an endpoint, parameter, or field. If validation fails,
  report the error instead of trying another endpoint.

For MCP servers, VS Code reads a workspace .vscode/mcp.json file and a portable .mcp.json at the workspace root, and forwards eligible servers to Copilot's agent host. VS Code's documentation recommends the portable locations for new servers.

How Swytchcode catches hallucinated API calls in Copilot

Swytchcode is an execution layer between agents and the APIs they call. Copilot's agent describes the job, gets real canonical IDs from the Swytchcode catalog, reads the resolved input schema, and runs the call through validation, policy checks, credential resolution, execution, and response normalization. An invented operation or a field the schema does not define fails before any request is sent.

Running swy init with the copilot editor option adds Swytchcode guidance to .github/copilot-instructions.md and merges a stdio MCP entry into .vscode/mcp.json.

# Initialize the project for Copilot in sandbox mode
npm install -g swytchcode
swy init --editor=copilot --mode=sandbox
swy doctor

# The loop Copilot follows for any third-party call
swy discover "send a message to a team channel" --json
swy get <project>
swy add <canonical_id>
swy info <canonical_id>
swy exec <canonical_id>

What Swytchcode catches:

• Invented operations. Only operations enabled in tooling.json can run. Anything else exits with code 2.
• Invented or missing fields. Inputs are validated against the resolved schema. Unknown fields, wrong types, and missing required fields exit with code 1 before the request is sent.
• Silent failures. A 200 response with an error in the body is treated as a failure, and errors include a category and whether they are retryable.
• Risky writes. Copilot can preview any call with dry_run=true. Allow and deny rules (Pro plan and above) and approval rules (Business and Enterprise) add control in production.
• Credentials. Keys are attached at execution time and never pass through the model's context, so Copilot never needs to see or write an API key.

Where this does not help

Swytchcode does not check imports, packages, or method calls inside your own code. Package hallucinations and invented SDK methods are caught by lockfile review, registry checks, and type checking. Swytchcode covers calls to external APIs, and to internal APIs you add from an OpenAPI spec, which type checkers cannot see.

FAQ

Why does GitHub Copilot suggest methods that don't exist?
Copilot predicts the most plausible code from patterns it learned. When the real API differs from the common pattern, or a method was renamed in a newer version, the plausible suggestion can be wrong.

What is slopsquatting?
Slopsquatting is registering a package under a name that AI models tend to hallucinate, so developers who accept the suggestion install the attacker's package. Checking every unfamiliar dependency before installing it is the main defense.

Does TypeScript catch Copilot's hallucinated methods?
Strict type checking catches many invented methods and arguments on typed libraries. It does not catch invented REST endpoints, query parameters, or JSON fields, because those are strings and objects sent over HTTP.

Can Copilot use MCP tools?
Yes. In VS Code, Copilot's agent mode can use MCP servers configured in .vscode/mcp.json, a portable .mcp.json at the workspace root, or the user configuration.

Does Swytchcode work with GitHub Copilot?
Yes. Run swy init --editor=copilot to add the Swytchcode MCP server to .vscode/mcp.json and guidance to .github/copilot-instructions.md.

Swytchcode resources

• Why your AI agent calls the wrong API: https://www.swytchcode.com/content/why-your-ai-agent-calls-the-wrong-api-and-how-to-fix-it
• Why Claude Code guesses API endpoints: https://www.swytchcode.com/content/why-claude-code-guesses-api-endpoints
• Why Cursor writes outdated API calls: https://www.swytchcode.com/content/cursor-outdated-api-calls
• Credential brokering for AI agents: https://www.swytchcode.com/content/ai-agent-credential-brokering
• AI agent execution layer architecture: https://www.swytchcode.com/content/ai-agent-execution-layer-architecture
• Swytchcode MCP server docs: https://docs.swytchcode.com/cli/mcp/
• CLI command reference: https://docs.swytchcode.com/reference/commands/
• Supported APIs: https://www.swytchcode.com/apis