Running a personal agent on your own machine

People choose a local setup for privacy. Then they run the agent against a hosted model and assume their data stayed home.

It did not. But the part that left is smaller and more controllable than most people realise, and knowing exactly which part it is turns out to be the whole skill.

This post separates the three decisions that get muddled together, then tells you what to do about each.

Three separate questions

When someone says they run their agent locally, they could mean any of three things. They are independent, and you can pick differently on each.

1. Where the tools run. The code that calls Gmail or reads a file. Local means the HTTP request to Gmail leaves your machine, not a vendor's.
2. Where the credentials sit. Your OAuth tokens and API keys. Local means they are on your disk and no third party holds a copy.
3. Where the model runs. The inference itself. Local means a model on your own hardware, through something like Ollama or llama.cpp.

The common and sensible setup is local on the first two and hosted on the third. Tools and credentials on your machine, inference at Anthropic or OpenAI.

That setup is a real privacy improvement. It is also not private in the way people sometimes describe it, so be precise about what you have.

What leaves, exactly

With a hosted model, what goes to the provider is the prompt. Nothing more and nothing less.

Your files do not get uploaded. Your mailbox is not synced. Your tokens stay put. What goes is whatever text the agent assembled before asking the model what to do next, which includes the content of any tool result it decided to include.

So if your agent reads forty emails and puts all forty in the prompt, forty emails left. If it reads forty and puts a count and three subject lines in the prompt, that is what left.

This is good news, because it means the exposure is a design decision rather than a property of the architecture. You control what goes in the prompt.

Keep the data out of the prompt

Four habits, in rough order of how much they help.

Filter before you fetch. Use the API's own query parameters so the data you do not need never arrives. A Gmail search that excludes verification emails is better than fetching everything and ignoring some of it.

Compute locally, summarise into the prompt. If the task is arithmetic over a spreadsheet, do the arithmetic in code and put the result in the prompt. Models are bad at long column addition anyway, so this makes the answer better as well as more private.

Return the fields the task needs. A tool that returns a whole API response puts every field in the context, including ones nobody asked for. Trim the response before it reaches the model.

Redact identifiers when they are not load bearing. Account numbers, addresses and full names often add nothing to the reasoning. Replace them with placeholders and map them back locally after.

The first habit does double duty, because trimming context also makes the agent work better. Too many tools and too much data degrade tool selection, which is a separate problem with the same fix. Nango wrote up the mechanics of that in how to expose tools to AI agents without clogging their context.

Where credentials should sit

This is the part of local that matters most and gets the least attention.

A token in a .env file is on your machine, which sounds local and fine. It is also readable by any process you run, it gets copied when you copy the project folder, and it is one bad git add from being public.

Worse, if the token is in an environment variable the agent can read, the agent can put it in the prompt. Then your credential has left your machine inside a conversation, and you will not notice.

Keep credentials in an encrypted store outside the project, and attach them to the request after the model has decided what to call. The model never sees them, so no conversation log can contain one. That requirement list is in what a credential vault needs to do for AI agents, and the mechanism is in keep API keys out of the LLM.

Should you run the model locally too?

You can, and for some tasks you should. Be clear about the trade.

A model running on your laptop means nothing leaves at all. It also means weaker reasoning, shorter usable context, slower responses, and noticeably worse tool calling than a frontier model. Multi-step agent work is exactly where small models struggle most, because one wrong tool choice derails the rest.

A reasonable middle path is to split by sensitivity. Run a local model over the data you would rather not send anywhere, such as classifying or redacting, then send the cleaned-up version to a hosted model for the harder reasoning. You get good output on most of the work and full privacy on the sensitive part.

If you do go fully local, check your hardware honestly. A model that fits in memory but runs at three tokens a second is not something you will keep using, and an agent you stop using protects nothing.

Setting up the local-tools version

npx swytchcode
swy login
swy init

# Credentials land in a local encrypted store, not in your project.
swy get gmail
swy auth connect gmail
swy auth status

swy discover "list messages matching a query" --project gmail
swy add method <canonical_id>

# Calls run from this machine. Credentials are attached at call time.
swy audit network               # see exactly what left and where it went

That last command is the one to actually use. It lists every call with its arguments and response, which is how you verify your assumptions rather than trusting them. If you believed the agent was only reading two days of mail, the log will tell you whether that is true. See how to log every API call an AI agent makes.

Pair this with a local MCP server over stdio and there is no network listener anywhere in the setup. The options are compared in self-hosted vs managed MCP servers for personal agents.

What local does not fix

Three things stay exactly as dangerous on your own machine.

Prompt injection. If the agent reads attacker-supplied text and has capabilities, running on your laptop changes nothing about that. The text is still untrusted and the capability is still there.

Over-permissioned access. A local agent with a send-email method can still email the wrong people. Local is about where things run, not about what is allowed.

Irreversible actions. Deleting a file from your laptop deletes it just as thoroughly as deleting it from a server. The limits in what a personal AI agent should never be allowed to do apply unchanged.

Local is a good answer to "who holds my credentials" and a partial answer to "who sees my data." It is not an answer to "what can this thing break."

FAQs

If I run an AI agent locally, does my data stay on my machine?

Your files and credentials do. Whatever the agent puts in the prompt goes to the model provider, unless you also run the model locally. Narrow what goes in the prompt and you narrow the exposure.

Is a local model good enough for agent work?

For classification, extraction and redaction, often yes. For multi-step tool use, usually not yet, because tool selection is where smaller models fall down and one bad choice compounds. Splitting the work by sensitivity tends to beat picking one model for everything.

Can I check what actually left my machine?

Yes. swy audit network shows every API call with arguments and responses. For what went to the model provider, check your client's own conversation history or logs. Looking at both once is worth more than a week of assuming.

Is a .env file good enough for a personal setup?

It works, and it is the thing most likely to leak. Any process you run can read it, it travels with the folder, and an agent that can read environment variables can put your token in a prompt. Use an encrypted store instead.

Does a model provider train on my prompts?

It depends on the provider and the plan, and it has changed more than once. Read the current terms for the specific plan you are on rather than relying on what was true last year, and check the retention period as well as the training question.

Related reading

• Self-hosted vs managed MCP servers for personal agents
• What a credential vault needs to do for AI agents
• Keep API keys out of the LLM: credential brokering for AI agents
• What a personal AI agent should never be allowed to do
• Browse all 300+ API integrations