Security & AI
Secure AI agents through permissions and verifiable controls
Tool-using agents can affect real systems. Security therefore covers inputs, information access and permitted effects, with application controls beyond written instructions.
Map inputs and effects
Identify messages, documents and services read by the agent, then list permitted outcomes. This exposes boundaries where outside information could influence an action.
Classify operations by consequences. Internal reading and external transmission need distinct conditions tied to the user’s intent.
Enforce bounded tools and authorisation
Check identity, organisation and records in the services performing actions. Avoid broad administrative credentials where restricted capabilities suffice.
Validate parameters and separate proposed changes from consequential execution. Generated persuasion must not serve as approval.
Separate information from authorised instructions
A document can contain instructions aimed at the assistant. Its content must not become authority to export information or change rules.
Restrict capabilities and data around external sources. Do not depend on a single message asking the model to ignore malicious content.
Keep useful, protected operational records
Log relevant tools, states, failures and approvals without automatically retaining secrets or whole documents. Define access and useful retention.
Provide stop controls and an incident process, including credential renewal or revocation. Teams should be able to disable a faulty capability independently.
Test boundaries after changes
Use contradictory content, inaccessible data, incorrect parameters and duplicate actions in a suitable environment. Inspect actual effects rather than promises in generated text.
Repeat sensitive cases after model, instruction or tool changes. Assign incident review and expand scope only with verifiable boundaries.
Your checklist
- Map inputs and permitted effects.
- Enforce service-side permissions.
- Separate external content from authority.
- Protect operational traces.
- Test boundaries and stop controls.
Frequently asked questions
Are written safety instructions sufficient?
No. Applications and services must enforce permissions, parameters and effects independently.
Should every conversation be logged?
Not automatically. Retain useful diagnostic and accountability records with appropriate access and retention.
A project to explore
Hush Messenger
Hush Messenger’s identity and access design illustrates why explicit boundaries matter when connecting an agent to operational tools.
Explore the project