← Blog

Open-Source AI Agent: 9 Checks Before You Install One

Open-Source AI Agent: 9 Checks Before You Install One

An open-source AI agent is an AI agent whose source code is published under a license that lets you read, run, change and redistribute it. Before you install one, check nine things: the license, the installer, model freedom, where data lives, defaults, limits, a stop button, the handling of untrusted input, and how it is updated.

This guide gives you each check as a question you can answer in a few minutes from the repository, and shows the answer for POLYROB, an open-source (MIT) self-hosted AI agent. Every POLYROB statement below comes from the public repository at release v1.7.9. Where POLYROB is weaker or a thing is unproven, the text says so.

Why does “open source” alone not tell you enough?

A public repository tells you that you can read the code. It does not tell you what the agent does when you are not watching, what it is allowed to touch on your machine, or who can talk to it. An agent has tools: a shell, files, a browser, mail. The risk sits in what those tools do by default and what stops them.

So treat “open source” as the first check, not the last. The other eight decide whether you want to run the thing on a machine that holds your keys.

Check 1: What is the license, and does it cover the whole thing?

Open the LICENSE file before the README. Look for a standard license name, and for whether parts of the repository fall outside it.

POLYROB is MIT licensed: “MIT License, Copyright (c) 2026 The POLYROB Authors”. MIT lets you use, copy, modify and redistribute the code. One related limit: the name POLYROB is a trademark with its own rules in TRADEMARK.md, so a fork has to be renamed.

Check 2: Can you read the installer before it runs?

Many agents install with curl ... | bash. That is fine only if you can read what you pipe. The POLYROB installer is a plain shell script, and the README tells you to read it first:

curl -fsSL https://polyrob.dev/install.sh -o install.sh && less install.sh && bash install.sh

The script clones the source into ~/.polyrob/src, builds a virtualenv, puts polyrob on your PATH and starts the setup wizard. It accepts --commit REF, which checks out an exact commit or release tag such as v1.7.4, so you can pin what you run. --uninstall removes the command and the PATH entry.

If you already manage Python tools, skip the script:

pipx install polyrob
polyrob setup
polyrob doctor

Check 3: Are you tied to one model provider?

An agent that works with one provider ties your bill and your privacy to that company. Look for a list of built-in providers and a way to declare your own endpoint.

POLYROB has OpenAI, Anthropic, Google Gemini, DeepSeek, OpenRouter and NVIDIA NIM built in. Any OpenAI-compatible or Anthropic-compatible endpoint can be declared in ~/.polyrob/providers.yaml, which is how a local server such as Ollama, LM Studio or vLLM is reached. /model <name> switches the model live from the next step. When no provider is named, the default is the first provider that has a key present, in this order: OpenRouter, Anthropic, OpenAI, Gemini, NVIDIA, DeepSeek.

POLYROB does not download or start a local model for you. You run the model server and POLYROB connects to it. A small local model will plan and use tools less reliably than a large hosted one, so test your model on a real task first.

Check 4: Where do your data and keys live?

Look for a plain statement of where memory, state and keys are stored, and for what leaves the machine.

In POLYROB, configuration and keys live in ~/.polyrob/.env. Memory, goals, sessions and skills live under ~/.polyrob/data, and memory and state are local SQLite files. Your prompts reach the model provider you configured, and the tools you switch on reach their own services. A hosted model sees your prompts; a local one does not.

One more honest detail: since version 1.7.5 the installer sends one anonymous count after a successful fresh install (version, OS and install method, no ID). Pass --no-telemetry or set DO_NOT_TRACK=1 to turn it off. An agent that makes a bigger privacy claim than that deserves a second look.

Check 5: What is on by default?

This is the check that matters most. Read the configuration reference and note which flags default to on.

For POLYROB, the reference is docs/CONFIGURATION.md, and polyrob doctor --flags prints the same table with the values resolved on your machine. The defaults to know:

Check 6: Is there a cap on what a run can do?

An agent that runs without you watching needs a ceiling. Ask: is there a step limit, a spend limit, and who sets them?

POLYROB has a step cap and a spend cap on every autonomous run. AUTONOMOUS_MAX_STEPS defaults to 150, and no tool argument or goal payload can change it: the operator owns it. AUTONOMOUS_RUN_BUDGET_USD defaults to $2.00 per run (0 turns it off). When a run hits a cap, it stops and says why. The spend check is a guard, not a guarantee: it fails open if the cost probe errors.

Know what this is not. It is a cap per autonomous run, not a daily or monthly budget, and interactive chat has separate step budgets. For how goals, jobs and the autonomy levels fit together, read what an autonomous AI agent is and how to keep control. Lower the spend cap while you learn what your tasks cost:

polyrob config set AUTONOMOUS_RUN_BUDGET_USD 0.5

Check 7: Is there a stop button you control?

If the agent misbehaves, you need a stop that is a command, not a polite request. POLYROB’s /pause (the old name /halt still works) is a command you send, not a message to the model. The configuration reference says plain chat text is never parsed as a pause or resume. You can scope it:

/pause background
/pause deploying for 6h
/pause for 2d

From a shell, polyrob autonomy pause and polyrob autonomy resume do the same job. Plain words such as “stop” typed to the agent are only messages to the agent, never commands. The agent can pause its own work, but only you can lift a pause.

Check 8: How does it treat untrusted input and strangers?

An agent reads web pages, mail and tool output, and any of them can contain text written to manipulate it. Ask what keeps that text from becoming a command.

POLYROB passes web pages, mail and tool results to the model inside an untrusted-content frame, as data. On a chat surface only the owner steers. A third party the agent contacted can only return data, and while that data is in the session the high-impact tools are denied until the owner’s next turn. A stranger who messages the agent gets a pairing code and no access until you approve them with the /pair owner command. With pairing off, a paired sender has no access on any chat app.

Skills follow the same split. A skill your own agent writes during your turn, a job or the background reviewer is live at once, with the earlier text kept so you can roll back; one written from a chat that holds someone else’s message stays a draft. A skill you install is scanned and goes to quarantine when it comes from a community tap, a git spec or a URL. You read it and run polyrob skills approve <name> to activate it. A name from an official or trusted tap, or a local folder you install with --trust local, is active after a clean scan.

The security model page is candid about the limit: these gates run in the same process as the agent, and none of them is a hard security boundary. The only hard boundary is the operating system or a container. On a server, run code execution in Docker (CODE_EXEC_BACKEND=docker); the local subprocess backend is a convenience for your own machine. This reduces prompt-injection risk. It does not remove it.

Check 9: How is it updated, and can you roll back?

Self-hosting makes you the operator, so look at the upgrade path. For a git install, polyrob update --apply takes a snapshot, installs, migrates and verifies, with automatic rollback if verification fails. polyrob update --rollback restores the most recent snapshot. For a pip or pipx install, polyrob update prints the command to run instead. polyrob service install keeps the agent running as a systemd user unit or a launchd agent.

Goals and scheduled jobs live in SQLite and survive restarts, and updates snapshot first.

Where is POLYROB weaker?

Be fair to yourself when you compare. POLYROB supports eight chat surfaces: Telegram, email, WhatsApp, Discord, Slack, Signal, Feishu/Lark and DingTalk. It has no native phone or desktop apps. It does not set up a local model for you. Its serverless sandbox backends (Modal, Daytona, Vercel Sandbox, Singularity) have been tested against fakes only. It runs on Linux, macOS and Windows through WSL2, with Python 3.11 or newer. It is also a young project with a small community. A dated comparison lives on the compare page.

FAQ

What is the best open-source AI agent?

It depends on your answers to the nine checks. If you want a personal agent that lives in your chat apps, keeps memory across sessions, works on standing goals and runs on your own machine inside caps you set, POLYROB is built for that case. Test any candidate on one real task before you trust it with a goal.

Is an open-source AI agent free?

The software can be. POLYROB is MIT licensed, so there is no license fee. You pay your model provider per token, or nothing with a local model, and you supply the hardware.

Can an open-source AI agent run without an API key?

Yes, with a local model server you run yourself, declared in ~/.polyrob/providers.yaml. Quality depends on the model.

Should I give an open-source AI agent access to my shell and files?

Treat that as a decision to earn, not a default. Keep autonomy off until you have set the caps, keep code execution off or in Docker on a server, read any installed skill before you approve it, and start with one task you can check.

Install it and check for yourself

Read the installer, then run the preflight:

curl -fsSL https://polyrob.dev/install.sh | bash
polyrob doctor

Install POLYROB, read the step-by-step self-hosting guide, and see the capabilities page for the full list. Reusable procedures the agent loads on demand are on the skills page.

Sources

Fetched from the public repository at release v1.7.9 (2026-11-13):

POLYROB is an open-source AI agent you run and own. Install it or read the docs.