Troubleshooting

What you'll learn

• the one command to run first;
• how to find why records are not arriving;
• why the checks stop too little, or too much;
• what the SDK's warnings and log messages mean;
• where to get help.

Start with foxy doctor

foxy doctor

It checks your machine first, then the backend, then the chain, and it says what is wrong at each step. Its exit code is 0 only when nothing on this machine needs you, a key is set, and the backend accepts it. Lines that start with an arrow are notices, and they need nothing from you. The foxy command explains every line.

foxy doctor runs as a new program, so it cannot see the background thread inside your own program. If that thread's last pass failed, only your program's debug log says so at once. foxy doctor reports the problem when a record has waited an hour.

To see more of what the SDK is doing, turn on its debug log. Most messages below are only shown at this level:

import logging

logging.basicConfig()
logging.getLogger("foxy_audit").setLevel(logging.DEBUG)

My records are not reaching the ledger

Go through these questions in order. Stop at the first "no".

1. Is an API key set where the program runs? Print foxy.enabled. If it is False, the SDK sends nothing, on purpose. Set FOXY_API_KEY in the program's own environment, not only in your terminal.
2. Is the backend address right? Print foxy.cfg.endpoint. For Foxy Audit's service it must be https://app.foxyaudit.tech. With no address set, the SDK uses http://127.0.0.1:8000, a backend on your own machine.
3. Does foxy doctor say records are waiting? Read the lines under the delivery line. The last delivery error says why: - 401 means the key is wrong or was revoked. Set the right key for new records. The waiting ones stay under the old key: move them with foxy spool rekey, as in question 4. - 402 means the workspace cannot record right now: it is under review, out of credits, or its trial has ended. The dashboard's Billing page says which. - ConnectionError means the address is wrong, or there is no network. When the network comes back, the waiting records go out on their own. If the address was wrong, see question 4. - A redirect status, in the 300s, means the address points somewhere that forwards elsewhere. Use the final address, then see question 4. - NotAcknowledged means something other than the Foxy backend answered: a proxy, a captive portal or a gateway in front of it. The records are kept and sent again. Point the address at the backend itself, or have the proxy let the request through.
4. Did you change your key or your address? Records made with the old ones are kept, but your new settings do not send them. Use foxy spool drain with the old key, or foxy spool rekey (with --from-backend <the old address> if the address changed). See The spool and delivery.
5. Is the spool on a disk that lasts? In a container, the spool must be on a volume that survives restarts. Set FOXY_SPOOL_PATH.
6. Was the program killed hard? Records waiting at the time are still in the spool. They go out when the program next starts.

Records are never thrown away because a send failed. When the network or the backend is back, they go out on their own. After a change of key or address, they wait until you move them, as in question 4. A lost key is the one case with no way out: its records can never be sent.

The checks are not stopping anything

1. Is the mode observe? It is the default. In observe, the prompt is not checked before the call, and nothing is stopped. Set mode="block" or FOXY_MODE=block.
2. Is the mode spelled right? A misspelled mode falls back to observe.
3. Did the SDK find the prompt? It looks for a keyword argument called prompt, user_prompt, message, messages, contents, text, input or query, then for the first positional argument that is a string. A prompt under any other keyword name is not found, and the SDK checks an empty prompt. Pass your prompt by one of those names.
4. Does your tag run the check you expect? Only hipaa and gdpr, and hipaa_basic and gdpr_basic, stop a prompt for personal data. Under other tags, personal data is only a label on the record. Check with foxy check "<your prompt>" --policy hipaa.
5. Is it something the checks do not look for? Names, locations and dates need the optional deep check. Medical record numbers are not found, with or without it. See Policies and rules.

The checks stop too much

1. Find the rule. Run foxy check on the prompt. It names every rule that fired.
2. Look it up in the rule catalogue. It says what each rule matches, and the mistakes it is known to make.
3. Common cases: - phi.credit_card or pii.credit_card on a long ID number. A few ID numbers pass every test a card number passes. The catalogue gives the measured rate. - phi.phone or pii.phone on a number that is not a phone. Any 10 to 13 digit number with the usual separators looks like one. - injection.* on a normal sentence such as "ignore the previous draft". The catalogue explains this known mistake. - response_sql.destructive on an answer that explains a database command. The check matches the words, not the meaning.
4. While you decide, set that function back to observe. The records keep coming, and nothing is stopped.

Redact mode is blocking calls

That is on purpose. After blanking out a finding, the SDK checks the prompt again. If something still fires, it blocks the call rather than send it. The error names the rules that still fired. The usual causes are a classifier finding, a finding from the deep personal-data check, or a value in a part of the prompt that is not text. See Modes.

A block came from nowhere

Your workspace's enforcement setting is block. The error says so, and names the setting. Change it in the dashboard under Settings, or set FOXY_ORG_POLICY=off for that program. The record says blocked_by_org_policy, so an auditor can see where the decision came from.

foxy explain cannot prove a row

• hash_mismatch: the prompt or the key is not the one the record was made with. A record made before version 1.18.0 with no key set used your API key, so a changed API key gives this answer for it. Use the old key. A newer record used the SDK's key file. The foxy command also gives this answer for a prompt that was a list of messages, or a prompt file whose line endings changed. See Proving a row with foxy explain.
• salt_unavailable: the record was made with a salt, and the salt file you gave does not hold it. Point --sidecar at the file the SDK wrote.
• ruleset_unrecorded: the record lists rules but names no ruleset. Your delivery logs and your SDK version tell you why.

Proving a row with foxy explain lists every answer.

Answers

• Streamed answers feel slower: under block, each piece is checked before it is passed on. Use observe if speed matters more.
• Every record says response_scan.degraded: your model library returns a shape the scan does not know. It never blocks on this. Return plain text from your function if you can.
• A script came through in a stream: under observe, nothing is stopped. Under block, pieces already passed on cannot be taken back: the flagged piece and the ones after it are stopped, and the ones before it are not. See Response scanning.

Errors and receipts

• audit_required raised an error, but the record is in the ledger: the record was saved before the wait began, and the confirmation came late. This is expected. The one exception is a full spool: then the record was refused and never saved, and the error's cause is SpoolFull.
• A receipt is missing: two things give no receipt. With audit_required=True, a call whose confirmation did not come in time raises AuditRequiredError instead. And if your on_event function raises an error, the SDK ignores it and notes it only in its debug log, so check the function on its own. An async def function cannot be the cause: it is refused with a TypeError when you make the client. In an async program, on_event runs on a helper thread.

Warnings and log messages

Getting help

• For a security problem, write to security@foxyaudit.tech. How we handle reports is on the Report Abuse page.
• For anything else, use the contact page.

When you write, include what foxy doctor printed and your SDK version. Never include your API key or a real prompt.