LLMShield

The problem

AI applications can receive prompt-injection attempts and sensitive data, and can return content that needs inspection. LLMShield explores putting a policy layer between an application and its model provider so the request and response pass through one place.

What is implemented

• A FastAPI chat endpoint and health check.
• Heuristic checks for common prompt-injection phrases, email and phone patterns, and selected secret formats.
• A central policy decision with block, review, redact, and allow outcomes, configurable through validated YAML policy rules with category-specific actions and severity thresholds.
• A deterministic mock provider, so the default flow does not call an external model.
• A React/Vite dashboard, Docker Compose setup with policy-file mounting, optional shared-key authentication, and SQLite audit metadata that avoids storing raw prompt/response text by default.
• An authenticated operational metrics endpoint and dashboard summaries for recent decisions, finding categories, and severities across up to 200 audit events.

Operational audit metrics

The dashboard can request aggregate counts from an authenticated /admin/metrics endpoint. It summarizes up to 200 recent audit events by policy decision, finding category, and severity. This gives a quick view of what the gateway has been recording without displaying raw prompt or response content in the metrics.

These are event and finding counts—not rates, detection accuracy, or proof that an attack was stopped. Access requires the configured API key, and the endpoint uses the same metadata-only audit records as the rest of the application.

Testing and evaluation

The repository includes scanner and API tests, linting, type checks, and a reproducible frontend install/build in CI. A separate synthetic evaluation harness reports precision, recall, false-positive rate, and false-negative rate independently for prompt-injection, PII, and secret detection. Its cases include benign text, prompt-injection variants, email and phone patterns, and token-shaped strings.

This evaluation is a small regression aid, not a real-world benchmark or production security score. The cases are synthetic and do not represent real traffic; broader performance claims would require a carefully labeled public dataset with pinned versions and documented licensing.

Reproducible frontend setup

The frontend dependency lockfile is committed and CI uses npm ci, making clean installs more reproducible than resolving dependency versions afresh on each build.

Policy configuration

Policy behavior can be configured with validated YAML rules rather than relying only on fixed defaults. Rules can set actions by detector category and severity threshold, and the Compose workflow documents how to mount the policy file into the service. Configuration is loaded and validated at startup; policy behavior is covered by tests.

Evaluation trade-offs

Reporting metrics per detector category makes it easier to see when one kind of detection behaves differently from another—for example, whether a PII match is incorrectly counted as a prompt-injection result. The current dataset is intentionally small, so the output is useful for regression checks rather than estimating real-world accuracy.

A trade-off I made

The initial version uses explicit regular expressions and a mock provider rather than forwarding requests to a real model by default. This makes the behavior easier to inspect and test, but pattern-based detection is limited: attackers can rephrase content, and benign text can trigger a rule.

Limitations

• Detection is heuristic and can miss attacks or flag harmless text.
• Real provider forwarding is disabled; upstream allowlisting, SSRF protections, response validation, and integration tests are still required.
• The review outcome is a hold message, not a full human-review queue.
• Redaction is not comprehensive anonymization, and shared-key authentication is not a complete identity system.

Read the code, tests, and security notes ↗