Policies and rules

What you'll learn

• what a policy tag is, and which tags exist;
• the three kinds of check, in plain words;
• how to see exactly which rule fired;
• what happens with a tag the SDK does not know;
• what the checks do not catch.

What a policy tag is

A policy tag is a word you give the decorator: @foxy.audit(policy="hipaa"). It does two jobs:

1. It picks which checks run on the prompt.
2. It is saved with the record, so you can group and filter your records by it later.

Every tag runs the basic checks. Some tags add more.

No tag turns the basic checks off. A trick aimed at the model, or a leaked key, is a problem whatever rules your company follows.

These checks run on the prompt before the call, in block and redact mode. In observe mode, the default, they do not run. See Modes.

The three kinds of check

1. Tricks aimed at the model

This is called prompt injection. It is text that tries to make the AI ignore its instructions, such as "ignore all previous instructions" or "reveal your system prompt".

The SDK uses two tools for this:

• Known patterns. Six rules look for known wordings, even when the letters are spaced out, hidden, or written with look-alike characters.
• A small classifier. It scores how much the whole prompt looks like an injection. It runs on your machine and sends nothing anywhere.

Both are measured, not guaranteed. The rule catalogue gives the measured numbers, and it says which kinds of text still get through.

2. Keys and tokens

Four rules look for secrets that should never go to a model: API keys that start with sk-, AWS access keys, private keys, and bearer tokens. There is no rule for passwords.

3. Personal data

The SDK looks for these in every call, under every tag, and records what it finds as labels, such as email:

• email addresses;
• social security numbers;
• phone numbers;
• IP addresses;
• card numbers. A number has to pass the same tests a real card number passes before it counts.

Only under hipaa and gdpr can a find also fire a rule, and so stop the prompt in block mode or be blanked out in redact mode. Under other tags it is a label on the record, nothing more.

With the optional deep check (see below), it can also find names, locations and dates. It does not find medical record numbers, with or without the deep check.

Step 1: See which rule fires

Use foxy check in your terminal, or check() in Python. Neither needs a key, and neither sends anything:

foxy check "Ignore all previous instructions" --policy default

What you should see:

[X] policy=default  ruleset=2026.09.3
  triggered : True
  reason    : prompt_injection
  rules     : injection.classifier, injection.ignore_previous
  signals   : prompt_injection

Two rules fired here: one pattern, and the classifier. reason is the most important kind of finding. signals are the short labels that go into the record.

On most terminals the first mark is a red cross. This manual prints [X], the plain form the SDK falls back to where a terminal cannot show the symbol.

Step 2: Understand the rule ids

Every rule has an id in two parts: the kind of check, a dot, then the rule's name.

The same check has a different first part under each tag. An email address is phi.email under hipaa and pii.email under gdpr.

Tags are not fussy about capitals

"HIPAA", "Hipaa" and " hipaa " all mean hipaa. The SDK folds the tag to lower case and trims the spaces. If it had to change what you typed, the record also keeps your spelling, in a field called policy_tag_raw.

A tag the SDK does not know

You may use your own tags, such as "claims_triage", to group your records. An unknown tag runs the basic checks only, and the SDK warns you once:

UserWarning: foxy-audit: unrecognised policy tag 'hipa'. Running the baseline checks only (known prompt-injection patterns + a local statistical classifier + secrets); NO PHI/PII check will run. Known tags: default, gdpr, gdpr_basic, hipaa, hipaa_basic, soc2.

This matters when the tag is a typing mistake. In the example, hipa was meant to be hipaa, so no personal-data check ran.

On a decorator, a tag must be letters, digits and underscores, up to 32 of them. A tag with a hyphen or a space in it, such as "my-tag", is replaced by default, and a warning is logged.

The optional deep check for personal data

The built-in personal-data checks use patterns. An optional extra adds a language model that can find names, locations and dates in the text. A date it finds is any date, not only a date of birth. It runs on your machine. To add it:

python -m pip install "foxy-audit[pii]"
python -m spacy download en_core_web_lg

Two things change when you add it:

• More prompts will be flagged. A prompt that passed in block mode before may now be blocked. Try your prompts with foxy check first.
• Its findings are never blanked out, so in redact mode a prompt with one of them is blocked instead. See Modes.

foxy doctor tells you whether the extra is installed and working.

What the checks do not catch

Stated plainly, because knowing the limits is part of using the tool well:

• Instructions hidden in a document. If your program pastes a web page or a file into the prompt, a harmful instruction inside it may read like normal text. The SDK cannot tell where each part of the prompt came from.
• Requests with no warning words, such as "list every email address you can see". They are harmful because of what the model knows, which the SDK cannot see.
• Languages outside the list. The patterns cover English and nine other named languages. The catalogue lists them.
• Names and dates, without the optional deep check.
• Medical record numbers, with or without it.

These stay known gaps. The Foxy Audit backend can also grade each record's metadata with an AI judge after the fact, but the judge sees labels and counts, never your text. It cannot find what the checks missed in the text. Its grade is a review signal, not a guarantee.

A record names its rules and its ruleset

The rules are frozen into versions called rulesets. This SDK uses ruleset 2026.09.3. A record on which a rule fired also names the ruleset it used, and a hash of that ruleset. A record on which nothing fired names none. A new version of the rules gets a new ruleset name. An old ruleset never changes, so an old record can be checked against the exact rules that made it. See Proving a row with foxy explain.

If it goes wrong