TehRiehlDeal
a8a47e38c1
CI / secrets-scan (push) Successful in 8s
CI / sast (push) Successful in 13s
CI / vuln-scan (push) Successful in 15s
CI / test (push) Failing after 29s
CI / lint (push) Failing after 31s
CI / build-images (push) Has been skipped
CI / image-scan (push) Has been skipped
CI / push (push) Has been skipped
Lets a user upload a CSV, OFX/QFX, or PDF bank statement, parses the transactions, flags duplicates and possible transfers against existing records, and bulk-creates the accepted rows after a final read-only confirmation step. Backend - New `statements` module with CSV (papaparse), OFX/QFX (node-ofx-parser), and PDF (pdf-parse) parsers behind a `StatementParser` strategy interface; format detected by content sniffing + extension. - `DuplicateDetectorService` checks FITID/externalId exact matches first, then a date+amount+description Jaro-Winkler heuristic, then cross-account transfer pairing. - New `POST /statements/parse` (multipart, in-memory, 10MB cap) returns the parsed preview without writing, including per-row `status` (`new`, `duplicate`, `needs_review`, `possible_transfer`) and any `needsMapping` payload when CSV headers are unrecognized. - `POST /transactions/bulk` accepts up to 500 rows, chunks them 50 at a time inside `prisma.$transaction`, applies balance deltas, and writes a single `ActivityLog` row per chunk instead of one per transaction. - Schema: nullable `external_id` column on `Transaction` plus composite indexes on `(account_id, external_id)` and `(account_id, date)` for fast dedupe-window queries. Not encrypted — it's an opaque bank ID used as a lookup key. Frontend - `ImportStatementDialog` runs a 4-step wizard: Upload → Column Mapping (if needed) → Review (editable table with duplicate/transfer badges) → Confirm (read-only summary with projected per-account balance impact and count-bearing primary button). The Confirm step gates the actual write, and Back to Review preserves all edit/checkbox state. - New `bulkCreateTransactions` action on the transactions store. - "Import Statement" button added next to Export on the Transactions page, with a success toast and a refresh of the transactions + accounts stores. Tests - 306 backend tests (29 suites), 195 frontend tests (31 suites), all green. - Fixtures under `test/fixtures/statements/` cover three CSV sign conventions (signed-amount, debit/credit, credit-card), OFX 1.x SGML, OFX 2.x XML, and a credit-card QFX with the CCSTMTRS branch. Versions bumped to 0.4.0 on both packages per the lockstep rule. NOTE: the Prisma migration in `prisma/migrations/20260527203542_add_transaction_external_id/` still needs to be applied to the live database with `prisma migrate deploy` — the DB at 10.0.3.82 wasn't reachable from the dev environment. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
10 lines
638 B
SQL
10 lines
638 B
SQL
-- Add external_id column for statement-import dedupe via bank-provided transaction IDs (e.g. OFX FITID).
|
|
-- Not encrypted: needs to be indexed for fast lookup, and is an opaque bank ID rather than sensitive PII.
|
|
ALTER TABLE "transactions" ADD COLUMN "external_id" TEXT;
|
|
|
|
-- Speeds up duplicate detection via externalId (statement re-imports).
|
|
CREATE INDEX "transactions_account_id_external_id_idx" ON "transactions"("account_id", "external_id");
|
|
|
|
-- Speeds up the windowed date+amount duplicate-detection query and the existing date-range filters.
|
|
CREATE INDEX "transactions_account_id_date_idx" ON "transactions"("account_id", "date");
|