firefly-importer
Bash scripts to automate importing financial data into a self-hosted Firefly III instance via the Firefly Data Importer (FIDI).
How it works
incoming/
jerickdiscover.csv ← raw CSV dropped here
checking.csv
↓ watch-imports.sh
↓ (flips 4th column sign on matching files)
imports/
jerickdiscover.json ← FIDI config (permanent)
jerickdiscover.csv ← processed CSV (staged)
checking.json
checking.csv
↓ import.sh
↓ (POST to FIDI /autoimport — FIDI reads the directory itself)
Firefly III ✓
CSV files are paired with their FIDI config by base name: checking.json + checking.csv.
FIDI scans the mounted import directory and imports all JSON+CSV pairs it finds.
Requirements
- bash
- curl
- python3 (standard on Ubuntu)
watch-imports.shuses polling rather than inotify, so it works on NFS and other network filesystems.
Setup
git clone <repo-url> firefly-importer
cd firefly-importer
cp .env.example .env
# Edit .env with your FIDI URL and credentials
mkdir -p incoming imports
chmod +x import.sh watch-imports.sh
# Place your FIDI JSON config files in imports/
# e.g.: imports/checking.json, imports/jerickdiscover.json
FIDI Docker configuration
FIDI must be able to read the imports/ directory. Mount it as a volume and whitelist it:
docker-compose.yml:
fidi:
image: fireflyiii/data-importer:latest
restart: always
env_file: .fidi.env
volumes:
- /path/to/.fidi.env:/var/www/html/.env:ro
- /var/vault/Finances/firefly-importer/imports:/imports:ro
ports:
- 8082:8080
.fidi.env (add these lines):
CAN_POST_AUTOIMPORT=true
AUTO_IMPORT_SECRET=<your-secret>
IMPORT_DIR_WHITELIST=/imports
Note: PHP-FPM does not inherit Docker container environment variables. The
.fidi.envfile must be volume-mounted as/var/www/html/.envinside the container (in addition to being referenced byenv_file).
After updating docker-compose.yml, restart FIDI:
docker compose up -d fidi
Configuration
All config lives in .env (never committed to git):
| Variable | Required | Default | Description |
|---|---|---|---|
FIDI_URL |
Yes | — | URL of your FIDI instance, e.g. http://localhost:8080 |
FIDI_SECRET |
No | — | FIDI AUTO_IMPORT_SECRET value |
FIDI_ACCESS_TOKEN |
No | — | Firefly III Personal Access Token |
FIDI_IMPORT_DIR |
No | /imports |
Path to imports dir inside the FIDI container (must match volume mount target and IMPORT_DIR_WHITELIST) |
IMPORT_DIR |
No | ./imports |
Local directory with JSON configs and staged CSVs |
INCOMING_DIR |
No | ./incoming |
Drop zone for raw CSV files |
AUTO_IMPORT |
No | false |
Run import.sh automatically after a file is staged |
POLL_INTERVAL |
No | 10 |
Seconds between directory scans |
FILE_STABLE_AGE |
No | 3 |
Seconds a file must be unmodified before processing (upload guard) |
Usage
Watch for new files (continuous)
Monitors incoming/ and processes any CSV that arrives:
./watch-imports.sh
Process existing files (one-shot)
Useful for batch runs or cron jobs:
./watch-imports.sh --once
Run the importer
Tells FIDI to import all staged JSON+CSV pairs from the mounted directory:
./import.sh
Preview what would be sent without making any requests:
./import.sh --dry-run
Typical workflow
# Drop your CSVs into incoming/, then:
./watch-imports.sh --once && ./import.sh
Or set AUTO_IMPORT=true in .env and just run ./watch-imports.sh — it will stage and import automatically each time a file lands.
CSV sign-flip
Some bank exports report credits as negative and debits as positive (the reverse of what Firefly III expects). The following files have their 4th column sign automatically flipped during staging:
jerickdiscover.csvpaigediscover.csv
To add more files, edit the FLIP_FILES array near the top of watch-imports.sh.
Running as a systemd service
A unit file is included at firefly-importer.service. Edit the User and path values, then install it:
# 1. Edit the unit file
nano firefly-importer.service
# Set User= and both path references to your actual install path
# 2. Install and start
sudo cp firefly-importer.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now firefly-importer
# 3. Check status / logs
sudo systemctl status firefly-importer
sudo journalctl -u firefly-importer -f
Project structure
firefly-importer/
├── import.sh # Batch importer — triggers FIDI /autoimport for the mounted directory
├── watch-imports.sh # File watcher — processes and stages incoming CSVs
├── firefly-importer.service # systemd unit file
├── .env.example # Config template
├── .gitignore
├── imports/ # JSON configs (committed) + staged CSVs (gitignored)
└── incoming/ # Drop zone for raw CSV files (gitignored)