AI AGENT · ARCHITECTURE AND SECURITY
AI agent architecture, security, and evaluation
A safer agent is built around bounded tools, server-side enforcement, untrusted inputs, human decisions, and reproducible evaluation.
01 · DIRECT ANSWER
Direct answer
An AI agent is not secure merely because its system prompt is strong. Treat data, memory, and tool output as untrusted; grant least privilege; validate arguments server-side; require human approval for consequential actions; and test every version on normal, failing, and adversarial cases.
02 · CRITERIA
Criteria before development
Apply these criteria to the specific workflow, data, and cost of error. They are not a universal readiness checklist.
Agent boundary
Allowed tasks, tools, data, limits, and stopping conditions are explicit.
Untrusted context
Documents, pages, messages, and tool output cannot elevate permissions.
Deterministic enforcement
The server validates arguments, access, state, limits, and idempotency before action.
Human decision
Irreversible, financial, legal, and other consequential actions require explicit approval.
03 · PROCESS
Validation sequence
- 01
Threat model
Map assets, roles, trust boundaries, attacker goals, misuse, and consequences.
- 02
Tool contracts
Separate read and write, minimise data, and define safe failure.
- 03
Evaluation set
Create normal, edge, failing, and adversarial cases with expected behaviour.
- 04
Production controls
Add traces, alerts, rate limits, versions, incident procedures, and rollback.
04 · DECISION
Decision table
Permissions should follow error cost rather than model capability.
| Action | Control | Reason |
|---|---|---|
| Retrieval and draft | Sources, access filters, and disclosure | The output stays informational and reviewable. |
| Reversible state change | Validation, idempotency, and audit trail | Failure can be found and corrected. |
| External communication | Preview and human confirmation | The action affects a third party. |
| Irreversible or consequential decision | Do not delegate to the agent | Residual risk exceeds the autonomy benefit. |
05 · MEASUREMENT
What to measure
Record the baseline workflow and collection method first. Then compare equivalent scenarios without presenting a planned target as measured impact.
Task success
Correct end-to-end outcome rather than plausibility of one response.
Tool safety
Tool choice, arguments, access, repeated calls, forbidden actions, and safe failure.
Attack resistance
Prompt and tool injection, data leakage, privilege escalation, and memory manipulation.
Operations
Latency, cost, integration errors, human interventions, and quality changes between versions.
06 · BOUNDARIES
Risks
Prompt injection
Untrusted content attempts to change the goal, expose data, or invoke a tool.
Excessive agency
Broad tools and permissions turn a model error into a real action.
Incomplete logs
Without version, context, and tool result, an incident cannot be reproduced.
Evaluation drift
An old set misses new tools, data, users, and attack methods.
WHAT CANNOT BE CLAIMED
Limitations of the conclusion
- Threat models and attacks depend on the actual architecture.
- No evaluation proves the absence of every vulnerability.
- Human review works only with sufficient context, time, and real authority to stop the action.
07 · PORTFOLIO
Related projects and honest stage
Legal Support MCP is a human-in-the-loop concept, not an autonomous legal agent. It illustrates the separation of sources, recommendations, and expert decisions.
Legal Support MCP
The MVP scope is documented: an MCP server for rules, sources, templates, and versions, plus a thin AI-client skill. The intended output is a structured package of prioritised remarks, questions, source links, and draft wording, with material or ambiguous terms escalated to a lawyer.
Open project review08 · METHODOLOGY
Primary sources
These sources support the methodology and definitions. They do not validate AI SaaS Solution project outcomes.
Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile
Open sourceOWASP Top 10 for Large Language Model Applications
Open sourceAdversarial Threat Landscape for Artificial-Intelligence Systems
Open sourceArtificial Intelligence Risk Management Framework (AI RMF 1.0)
Open source