The Security Checklist for AI-Built Apps

AI-generated code does not need a special kind of security theatre. It needs the same fundamentals as any internet product, plus careful controls around prompts, model output, and agent actions.

This checklist is an editorial starting point based primarily on the OWASP Top 10 for Large Language Model Applications. It is not a penetration test, legal advice, or a guarantee of safety. Products handling health records, financial assets, children’s data, critical infrastructure, or other high-impact decisions need qualified review appropriate to their risk.

Start with what can go wrong

Write down the product’s valuable assets: accounts, private data, money, paid content, model credits, API access, administrator actions, and business records. Then identify who or what could misuse them: an anonymous visitor, a standard user, a compromised account, malicious uploaded content, an untrusted integration, or the model itself producing an unsafe instruction.

Keep the first threat model to one page. For every asset, ask:

• Can someone read it without permission?
• Can someone change or delete it without permission?
• Can someone make the owner pay for unwanted usage?
• Can untrusted text cause a privileged action?
• Would a failure be visible and reversible?

This turns “make it secure” into tests the team can actually perform.

1. Remove secrets from every client and prompt

Browser code, mobile bundles, public repositories, generated previews, logs, screenshots, and shared prompts are not secret stores. Search the final build for API keys, database URLs containing credentials, private tokens, signing secrets, and test accounts. Rotate anything that was exposed, even briefly.

Call paid or privileged services through a server you control. Store secrets in the deployment platform’s encrypted secret manager, give separate credentials to development and production, and grant each credential the smallest required scope. Add usage limits and billing alerts where providers support them.

Do not paste customer records or production credentials into a coding assistant to debug an issue. Redact the example or reproduce the structure with synthetic data.

2. Enforce identity and authorization server-side

Authentication answers “who is this?” Authorization answers “may this identity perform this action?” Both decisions belong on the trusted side of the system.

For every endpoint that reads or changes non-public data:

• reject missing, invalid, and expired sessions;
• load the current identity from the verified session, not a browser-supplied user ID;
• verify ownership or role for the specific record;
• prevent users from assigning themselves elevated roles;
• make reset and verification tokens short-lived and single-use;
• invalidate sessions when the relevant security state changes.

Test with two ordinary accounts. Capture a request for account A, replace its record identifier with one owned by account B, and verify that the server refuses it. Repeat for downloads, invoices, messages, saved prompts, and admin-like operations.

3. Treat prompt injection as untrusted input

OWASP places prompt injection at the top of its LLM application risks. An attacker may put instructions in a chat message, web page, document, image, tool result, or retrieved knowledge source. The model can mistake that content for authority.

System prompts alone are not a security boundary. Reduce the impact of a successful injection:

• separate instructions from untrusted content in the application design;
• allowlist tools and parameters rather than accepting arbitrary commands;
• keep retrieval sources scoped to the current user and task;
• remove secrets from model context;
• require deterministic server validation after model output;
• ask for explicit human confirmation before sending, buying, deleting, publishing, or changing access.

Test with malicious instructions inside every format the app accepts, including uploaded files and content fetched from the web. The safe result is not merely a polite refusal; it is the absence of unauthorized data or action.

4. Validate model output before using it

Model output is untrusted, even when the prompt is trusted. OWASP calls this risk improper output handling. Never place generated HTML directly into a page without sanitizing it. Never concatenate generated text into a database query, shell command, file path, or dynamic code evaluator. Use parameterized APIs and strict schemas.

When a model returns structured data, parse it against a narrow schema and reject extra fields. Apply the same business rules that a hand-written form would face: prices cannot become negative, quantities have limits, email recipients must be allowed, and account roles cannot be invented.

Display uncertainty honestly. A fluent answer can still be wrong; the 2025 Stack Overflow survey found 46% of respondents distrust AI accuracy. For consequential output, provide the underlying source and a human review path.

5. Limit agent authority

An agent that can read email, query customer data, modify a repository, or issue a refund combines model uncertainty with real permissions. OWASP describes this as excessive agency when the system grants functionality, permissions, or autonomy beyond what the task needs.

Give each tool one narrow capability. Prefer create_refund_request over unrestricted payment-provider access, and draft_message over send_any_email. Bind tool execution to the authenticated user, re-check authorization at execution time, cap cost and frequency, and record who approved the action.

Use a preview-and-confirm step for irreversible or external effects. The confirmation must describe the actual resolved action — recipient, amount, resource, and consequence — not a generic “continue?” prompt.

6. Protect data through its full lifecycle

Classify what the product collects and avoid collecting what it does not need. Encrypt transport with HTTPS and use the storage provider’s encryption controls. Restrict database access, backups, analytics, support tools, and model-provider logs. A privacy promise is only as strong as the broadest downstream copy.

Check for sensitive information disclosure in model responses. Try requests that ask for system instructions, other users’ content, hidden context, secrets, or memorized examples. Prevent cross-user retrieval by filtering at the database or search layer before content reaches the model.

Define retention and deletion. Verify that deleting an account addresses primary records, stored files, vector indexes, cached copies, and queued jobs as required by the product’s policy. Document exceptions such as legally required transaction records.

7. Review dependencies and generated packages

AI tools can suggest packages that are outdated, compromised, misspelled, or nonexistent. OWASP’s LLM risk list includes the supply chain because models, datasets, adapters, libraries, and external services all affect the application.

For every dependency, confirm that the package name and publisher are real, the repository and release history are credible, and the requested version has no known critical advisory. Remove packages used for trivial tasks the platform already supports. Lock versions, keep the lockfile in version control, and make updates reviewable.

Apply the same diligence to model providers and remote templates. Know where they run, what data they receive, which version can change underneath the app, and what happens when they are unavailable.

8. Bound cost and resource consumption

An unauthenticated generation endpoint can become an open bill. Set request size limits, timeouts, concurrency caps, per-account quotas, and provider budgets. Rate-limit authentication, password reset, generation, upload, search, and any action that triggers paid work.

Reject oversized files before expensive processing. Limit recursive agents and repeated tool calls. Cache only where results can safely be shared, and prevent one user from consuming the capacity needed by everyone else. OWASP groups these failures under unbounded consumption.

9. Log security events without logging secrets

Record authentication failures, permission denials, role changes, secret rotation, payment-state changes, agent tool calls, confirmations, and administrator actions. Include a correlation identifier so an incident can be traced across services.

Do not record passwords, session tokens, raw payment data, API keys, or unnecessary private prompts. Restrict access to logs and set retention. Test alerts with a controlled event before launch; an alert rule that has never fired is only a hypothesis.

The 2025 DORA report emphasizes the surrounding delivery system. For security, that means small deploys, fast feedback, visible failures, and a practiced recovery path matter as much as the first code review.

10. Rehearse recovery

Back up valuable data and prove that restoration works. Document how to revoke sessions, rotate credentials, disable a compromised integration, stop an agent, reverse an unsafe action where possible, and deploy the last known-good version.

Assign an incident owner and a contact channel. Prepare a short template that tells affected users what happened, what data or action was involved, what has been contained, and what they should do next. Accurate communication is part of the security response.

The launch decision

Do not ask whether an AI-built app is “100% secure.” Ask whether its most valuable assets have explicit controls, whether those controls were tested, and whether the team can detect and contain a failure.

A model can propose a security control. Only a test against the deployed system can show whether that control exists.

If a high-impact action is still performed directly from the browser, a model can call unrestricted tools, or one user can retrieve another user’s data, stop the release. Those are not polish items. They are product boundaries.