Python CLI Guide

Python CLI Guide

Setup

cd python_cli
python -m venv venv

# Linux / macOS
source venv/bin/activate
# Windows:
venv\Scripts\activate

pip install -r requirements.txt

requirements.txt includes prompt-toolkit, needed for the interactive shell below.

Usage

Run with a subcommand for one-shot scripted use, or with no subcommand to drop into the interactive shell:

python cli.py                                                 # enters the interactive shell
python cli.py status                                          # one-shot scripted command

Both modes dispatch to the exact same commands.py functions -- nothing is duplicated between them.

# User management
python cli.py list                                          # list all users
python cli.py add --uid 04AABBCCDD --name "John Smith" --valid-days 30
python cli.py add --uid 5AF73581 --name "Hyace"             # ADMIN badge (no expiry)
python cli.py add                                          # prompts interactively
python cli.py remove --uid 04AABBCCDD
python cli.py remove --force                                # wipe ALL users (prompts for backup format: json/bin)
python cli.py remove --force --no-backup                    # wipe ALL users, skip backup
python cli.py remove --except 04AABBCCDD,5AF73581           # keep only these UIDs
python cli.py rename --uid 04AABBCCDD --name "J. Smith"
python cli.py find --uid 04AABBCCDD                       # find user by exact UID (O(log n) lookup)
python cli.py find --name "Smith"                           # find users by name (partial match)

# Batch import / export
python cli.py export users.json                             # export to JSON
python cli.py export users.bin                              # export to .bin (byte-for-byte device format)
python cli.py export users.txt                              # still JSON -- only .bin gets raw binary
python cli.py export users.json --json-transport             # fallback: older per-user JSON 'list' transport
python cli.py import users.json                             # import from JSON, CSV, or .bin (auto-detected)
python cli.py import users.bin                              # import .bin directly (no re-encoding)
python cli.py import users.json --dry-run                   # validate file without writing
python cli.py import users.json --clear                     # backup (prompts json/bin), wipe DB, then import
python cli.py import users.json --clear --no-backup         # wipe DB directly, no backup, then import
python cli.py import users.json --json-transport            # fallback: older per-batch JSON transport
python cli.py sync users.json                               # make device match file exactly (merge-diff)
python cli.py sync users.bin --dry-run                       # show the remove/add/replace diff, change nothing

# Tag renewal
python cli.py tag-renew 30 --quota 10                       # renew tags, stop after 10
python cli.py tag-renew 0.01 --quota none                   # renew until Ctrl+C

# Device info
python cli.py status                                        # DB path + LittleFS storage usage
python cli.py netstatus                                     # Wi-Fi connected? SSID? IP? signal?
python cli.py ntp-time                                      # show device's current local time
python cli.py ntp-sync                                      # force NTP resync

# Wi-Fi provisioning
python cli.py configure -w "MyWiFi" -p "MyPassword"

# Timezone (persisted on device, no reflash needed)
python cli.py timezone --offset 3600                        # UTC+1
python cli.py timezone --offset 3600 --dst 3600              # UTC+1 with DST

# Card scanning
python cli.py scan                                          # present a card, prints its UID
python cli.py scan --infinite                               # scan cards forever, Ctrl+C to stop

# Debug
python cli.py list-ports                                    # list all serial ports
python cli.py --port COM5 list                              # override auto-detection

Interactive Shell

Running python cli.py with no subcommand (optionally with --port) drops into a git/psql-style shell: auto-detects and connects to the device once, then lets you run commands without re-specifying --port or re-establishing the connection each time. Requires prompt_toolkit and rich; if either is missing it prints the exact pip install command and exits without erroring into a stack trace.

$ python cli.py
rfid(COM5)> list
rfid(COM5)> add --uid 04AABBCCDD --name "John Smith" --valid-days 30
rfid(COM5)> find 04AABBCCDD
rfid(COM5)> disconnect
rfid> connect --port COM7
rfid(COM7)> exit

• Every scripted subcommand works verbatim inside the shell (same commands.py handlers, same flags).
• ls/rm/? are shell-only aliases for list/remove/help.
• Shell-only meta-commands: help, history, version (CLI, firmware, protocol, and Python versions), clear, connect, disconnect, reconnect, exit/quit.
• TAB-completes UIDs for find/remove/rename against the device's current user list.
• Command history persists across sessions in ~/.rfid_cli_history.
• All shell output goes through Rich (tables, panels, colored status); prompt_toolkit is used only for line editing and TAB completion. The scripted (non-shell) CLI shares the same utils.py rendering and falls back to plain-text output if rich isn't installed.

User Schema and Expiration

Each user is stored on the device as:

{
  "uid": "A43FE5S4",
  "name": "Azrael",
  "registered": "2024-04-06",
  "valid_days": 30
}

• registered -- ISO-8601 date (YYYY-MM-DD), stamped automatically from the CLI machine's local date at add-time.
• valid_days -- days from registered the badge stays valid. Accepts decimals (0.01 = ~14 minutes). Counts from midnight UTC of the registered date, not from the moment of creation.

Expiration is evaluated on the device:

expiration_date = registered + valid_days

if current_date_time <= expiration_date:
    Access Granted
else:
    Access Denied (Expired)

If the ESP32 hasn't synced NTP, it fails safe -- every normal card is denied with "No Time Sync".

The device's timezone (default UTC+0, see Config.h's NTP_GMT_OFFSET_SEC/NTP_DAYLIGHT_OFFSET_SEC) must match the timezone of the CLI machine -- set it at runtime with python cli.py timezone --offset SECONDS (see below), no reflash needed.

Admin badges

Omit --valid-days for an admin card (no expiration, always granted):

python cli.py add --uid 5AF73581 --name "Hyace"

Admin badges work even without NTP sync. Stored with sentinel values (registered="", valid_days=-1).

Batch Import / Export

The import command reads a JSON, CSV, or .bin file and sends all users to the device. By default it uses a raw binary transfer (import_bin): no ArduinoJson parsing on the device, no per-user JSON encoding on the host -- the exact on-disk record format goes over the wire as-is:

python cli.py import users.json           # encode + binary transfer
python cli.py import users.bin            # byte-for-byte pass-through (fastest)

A .bin input file is sent directly without re-encoding. A JSON/CSV input is encoded into the binary format on the host first, then sent as one blob. If the file is already on the device (CRC32 match), the transfer is skipped entirely. One serial connection, one flash write at the end via import_begin/import_end.

The --json-transport flag falls back to the older per-batch JSON pipeline (batch_add, ~100 users per round-trip) for older firmware or if the binary path misbehaves on specific hardware.

Names longer than 48 UTF-8 bytes (MAX_NAME_LEN) are rejected, not truncated: convert.name_fits_device() is the single source of truth for this policy, and every JSON/CSV/.bin entry point -- CLI add, CLI import, and the standalone convert.py encoder -- agrees on it. Only the low-level encode_record() (for direct library callers) stays permissive and truncates instead of raising, by design.

CSV files are also supported (auto-detected by extension, with cp1252/latin-1 encoding fallback for non-UTF-8 files):

uid,name,registered,valid_days
04AABBCCDD,Alice,2025-01-15,30
5AF73581,Bob,,

The export command dumps the device database. Output format is auto-detected from the file extension: only .bin gets a self-contained binary file (7-byte header + records, same format as users.bin on the device); any other extension (.json, .txt, etc.) always produces JSON. The wire transport is export_bin (raw binary) by default, with --json-transport falling back to the older list command. A .bin export can be re-imported directly with import -- no re-encoding needed. There is never hidden binary output under a misleading filename:

python cli.py export backup.json          # JSON output
python cli.py export backup.bin           # byte-for-byte device format
python cli.py export backup.txt           # still JSON -- never raw binary under a wrong extension

Sync

sync makes the device database exactly match a local JSON/CSV/.bin file -- unlike import, which is additive (or fully destructive with --clear), sync computes the minimal remove/add/replace diff and applies only that:

python cli.py sync users.json             # make the device match the file exactly
python cli.py sync users.bin --dry-run    # show the diff, change nothing

How it works:

1. The host computes the local file's canonical db_crc32 (same algorithm as the device's status/sync_begin response) and asks the device for its own db_crc32. If they already match, nothing is transferred.
2. Otherwise the host downloads a compact manifest (uid + per-record CRC32 for every user on the device) and diffs it against the local file:
   • uid on the device but not in the file → remove
   • uid in the file but not on the device → add
   • uid on both sides but with a different record CRC32 → replace (whole record, never a field-level patch)
3. The host streams the remove list, then the add records, then the replace records, in one raw binary transfer -- no per-user round trips.
4. The device applies everything in RAM, does one flash write, and returns its new db_crc32. The host compares that against the local file's own crc32 and reports a match or a mismatch explicitly, rather than just trusting an "ok" status.

Duplicate UIDs within the input file are resolved before diffing (last occurrence wins, with a note printed for how many were collapsed) -- the diff needs exactly one entry per uid on each side.

If the transfer stalls or the final flash write fails, the device reloads its in-RAM state from what's actually still on flash (nothing was written until the very end) and reports the failure; re-running sync is always safe.

User Limit

Maximum 70000 users (MAX_USERS in Config.h). User records use a PSRAM-backed allocator (PsramAllocator) so the 8MB external PSRAM holds the full database instead of the ~300KB SRAM heap. Fixed-capacity strings (FixedStr<N>) store UID/name/dates inline so no per-field SRAM allocations leak out of the PSRAM vector.

The on-disk format is a fixed-width binary file (users.bin) with a header (magic "RUD1", version, record size) and per-record CRC32. The in-RAM array is kept physically sorted by UID at all times, enabling binary search lookups and targeted save strategies:

Operation	Save strategy
renameUser() / renewUser()	Single-record seek+write (saveSingleRecord_)
addUser()	Suffix rewrite from insertion point onward (saveSuffixFrom_)
removeUser()	Full rewrite (no in-place truncate on this LittleFS)
clearAll() / import_end	Full rewrite

A corrupted record is skipped on load, not the whole database. Existing users.json databases are migrated automatically on first boot and kept as users.json.bak.

The CLI pre-checks the limit before importing. Use --clear to wipe first:

python cli.py import users.json --clear

Tag Renewal

The tag-renew command puts the device into renewal mode. Present cards one by one to update their registered date to today and reset valid_days:

python cli.py tag-renew 30 --quota 10       # renew 10 tags with 30-day validity
python cli.py tag-renew 0.01 --quota none   # renew until Ctrl+C (~14 min validity)

LCD shows "RENEWING NFC TAG / Present Card..." during the process. Only tags already in the device database are renewed. Ctrl+C exits cleanly.

Wi-Fi Provisioning, NTP Time Sync, and Timezone

1. python cli.py configure -w "MyWiFi" -p "MyPassword" provisions credentials (stored in NVS, persist across reboots).
2. Device reconnects and re-syncs NTP on every boot.
3. Re-sync every 6 hours (NTP_RESYNC_INTERVAL_MS).
4. python cli.py timezone --offset 3600 sets the device's GMT offset (here, UTC+1) and persists it in NVS -- no reflash needed. Add --dst SECONDS for daylight saving on top of --offset. Applied immediately (the device re-syncs NTP against the new offset before confirming) and must match the timezone of the machine running this CLI, same as before -- see User Schema and Expiration above. Until timezone is run once, the device uses Config.h's NTP_GMT_OFFSET_SEC/NTP_DAYLIGHT_OFFSET_SEC as the default.

python cli.py ntp-time                    # Device time: 2026-07-03 19:43:20 (epoch: ...)
python cli.py ntp-sync                    # Force NTP resync
python cli.py timezone --offset 7200      # UTC+2, no DST
python cli.py timezone --offset 3600 --dst 3600   # UTC+1 with a 1h DST bump