first commit

2026-03-04 08:33:10 -05:00
commit d8ed5d5dce
6 changed files with 494 additions and 0 deletions
--- a/README.md
+++ b/README.md
@@ -0,0 +1,145 @@
+# firefly-importer
+
+Bash scripts to automate importing financial data into a self-hosted [Firefly III](https://firefly-iii.org/) instance via the [Firefly Data Importer (FIDI)](https://docs.firefly-iii.org/how-to/data-importer/about/).
+
+## How it works
+
+```
+incoming/
+  jerickdiscover.csv   ← raw CSV dropped here
+  checking.csv
+
+        ↓  watch-imports.sh
+        ↓  (flips 4th column sign on matching files)
+
+imports/
+  jerickdiscover.json  ← FIDI config (permanent)
+  jerickdiscover.csv   ← processed CSV (staged)
+  checking.json
+  checking.csv
+
+        ↓  import.sh
+        ↓  (POST each JSON+CSV pair to FIDI /autoimport)
+
+Firefly III ✓
+```
+
+CSV files are paired with their FIDI config by base name: `checking.json` + `checking.csv`.
+
+## Requirements
+
+- bash
+- curl
+- python3 (standard on Ubuntu)
+- inotify-tools — for continuous file watching: `sudo apt-get install inotify-tools`
+
+## Setup
+
+```bash
+git clone <repo-url> firefly-importer
+cd firefly-importer
+
+cp .env.example .env
+# Edit .env with your FIDI URL and credentials
+
+mkdir -p incoming imports
+chmod +x import.sh watch-imports.sh
+
+# Place your FIDI JSON config files in imports/
+# e.g.: imports/checking.json, imports/jerickdiscover.json
+```
+
+## Configuration
+
+All config lives in `.env` (never committed to git):
+
+| Variable            | Required | Default       | Description                                              |
+|---------------------|----------|---------------|----------------------------------------------------------|
+| `FIDI_URL`          | Yes      | —             | URL of your FIDI instance, e.g. `http://localhost:8080`  |
+| `FIDI_SECRET`       | No       | —             | FIDI `AUTO_IMPORT_SECRET` value                          |
+| `FIDI_ACCESS_TOKEN` | No       | —             | Firefly III Personal Access Token                        |
+| `IMPORT_DIR`        | No       | `./imports`   | Directory with JSON configs and staged CSVs              |
+| `INCOMING_DIR`      | No       | `./incoming`  | Drop zone for raw CSV files                              |
+| `AUTO_IMPORT`       | No       | `false`       | Run `import.sh` automatically after a file is staged     |
+
+## Usage
+
+### Watch for new files (continuous)
+
+Monitors `incoming/` and processes any CSV that arrives:
+
+```bash
+./watch-imports.sh
+```
+
+### Process existing files (one-shot)
+
+Useful for batch runs or cron jobs:
+
+```bash
+./watch-imports.sh --once
+```
+
+### Run the importer
+
+Posts all staged JSON+CSV pairs to FIDI:
+
+```bash
+./import.sh
+```
+
+Preview what would be sent without making any requests:
+
+```bash
+./import.sh --dry-run
+```
+
+### Typical workflow
+
+```bash
+# Drop your CSVs into incoming/, then:
+./watch-imports.sh --once && ./import.sh
+```
+
+Or set `AUTO_IMPORT=true` in `.env` and just run `./watch-imports.sh` — it will stage and import automatically each time a file lands.
+
+## CSV sign-flip
+
+Some bank exports report credits as negative and debits as positive (the reverse of what Firefly III expects). The following files have their 4th column sign automatically flipped during staging:
+
+- `jerickdiscover.csv`
+- `paigediscover.csv`
+
+To add more files, edit the `FLIP_FILES` array near the top of [watch-imports.sh](watch-imports.sh).
+
+## Running as a systemd service
+
+A unit file is included at [firefly-importer.service](firefly-importer.service). Edit the `User` and path values, then install it:
+
+```bash
+# 1. Edit the unit file
+nano firefly-importer.service
+#    Set User=  and both path references to your actual install path
+
+# 2. Install and start
+sudo cp firefly-importer.service /etc/systemd/system/
+sudo systemctl daemon-reload
+sudo systemctl enable --now firefly-importer
+
+# 3. Check status / logs
+sudo systemctl status firefly-importer
+sudo journalctl -u firefly-importer -f
+```
+
+## Project structure
+
+```
+firefly-importer/
+├── import.sh                   # Batch importer — POSTs JSON+CSV pairs to FIDI
+├── watch-imports.sh            # File watcher — processes and stages incoming CSVs
+├── firefly-importer.service    # systemd unit file
+├── .env.example                # Config template
+├── .gitignore
+├── imports/                    # JSON configs (committed) + staged CSVs (gitignored)
+└── incoming/                   # Drop zone for raw CSV files (gitignored)
+```