Feature/statement parsing #1

Merged

TehRiehlDeal merged 8 commits from feature/statement-parsing into main 2026-05-27 15:14:07 -07:00

Conversation 0 Commits 8 Files Changed 47

+5133 -12

8 Commits

Author	SHA1	Message	Date
TehRiehlDeal	9f0af6bfb8	Fix backend lint errors in the statement-parser code ... The new statements module was hitting type-safety errors from typescript-eslint's recommendedTypeChecked config: - parse-statement.dto.ts: tighten the Transform decorator's signature so the JSON.parse path returns a typed object or undefined, not `any`. - duplicate-detector.service.ts: drop the unused WEAK_DESCRIPTION_SIMILARITY constant left over from earlier logic. - csv.parser.ts and ofx.parser.ts: the parse() methods were `async` without any `await` (require-await). Convert them to non-async functions that return a Promise — wrap parseSync() in a try/catch so thrown errors still surface as rejected promises for spec callers that use `.rejects.toThrow()`. - ofx.parser.ts: replace `require('node-ofx-parser')` with a typed `import * as ofxLib`, backed by a hand-written declaration file at src/types/node-ofx-parser.d.ts that captures the bank + credit-card transaction shapes we consume. - pdf.parser.ts: import the typed `PDFParse` class from pdf-parse directly instead of lazy-requiring it as `any`. Keep the test seam but back it with a typed PdfTextExtractor function instead of the ad-hoc `any` shape. Also pulls in the prettier reformat that `eslint --fix` produced across the touched files and their specs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-27 14:57:22 -07:00
TehRiehlDeal	6a6d629bcf	Replace pnpm/action-setup with corepack to actually pin the pnpm version ... pnpm/action-setup@v4 was ignoring its `version` input on this runner and installing pnpm 10.x no matter what value we passed. That's why every attempt to land an onlyBuiltDependencies allowlist failed — ERR_PNPM_IGNORED_BUILDS kept blocking the install. Switch to corepack, which Node 22 ships with, and `corepack prepare pnpm@9.14.4 --activate`. Same mechanism the Dockerfiles use. Adds an explicit `pnpm --version` line so future CI runs make the actual installed version visible. Dropped `cache: pnpm` from actions/setup-node@v4 since pnpm isn't on PATH yet at that step — the pnpm store cache wasn't doing much for us on first runs anyway. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-27 14:49:18 -07:00
TehRiehlDeal	a79ee5f479	Pin CI to pnpm 9.14.4 to dodge the strict ERR_PNPM_IGNORED_BUILDS gate ... The Install dependencies step has been failing with ERR_PNPM_IGNORED_BUILDS no matter where I put the onlyBuiltDependencies allowlist (package.json#pnpm, pnpm-workspace.yaml, project .npmrc) and no matter which pnpm 10.x is installed. The strict build-script gate was introduced in pnpm 9.15 / 10.0; pnpm 9.14.4 predates it and just runs postinstall scripts the way pnpm has for years — matching what the Dockerfiles already do via corepack `pnpm@9`. Also reverts the short-lived `--ignore-scripts` install workaround, which skipped @prisma/client's postinstall and left runtime files missing so `prisma generate` couldn't complete. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-27 14:47:08 -07:00
TehRiehlDeal	aefa5abf95	Pin pnpm 10.33.0 in CI so onlyBuiltDependencies is honored ... pnpm/action-setup@v4 with `version: 9` was actually resolving to pnpm 10.x in CI (confirmed by the pnpm-10-only WARN about the package.json "pnpm" field and by the "Verifying lockfile against supply-chain policies" step). pnpm 10 reads onlyBuiltDependencies from pnpm-workspace.yaml — our config has been correct there since the first fix — but whichever 10.x the action picked apparently didn't, so every install failed with ERR_PNPM_IGNORED_BUILDS. Pin to 10.33.0 explicitly. That's the version where I verified locally that pnpm-workspace.yaml's onlyBuiltDependencies is read correctly and the install completes cleanly. Dockerfiles still pin pnpm@9 via corepack, which reads the legacy package.json#pnpm.onlyBuiltDependencies (still in place), so production image builds remain unaffected. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-27 14:42:11 -07:00
TehRiehlDeal	ca371d93c3	Move onlyBuiltDependencies into .npmrc so CI's pnpm 10 honors it ... pnpm-workspace.yaml's onlyBuiltDependencies (pnpm 10+) wasn't being read in CI even though that's where pnpm 10 docs say it should live — the install still bailed with ERR_PNPM_IGNORED_BUILDS. .npmrc is the long- established, version-agnostic location pnpm honors regardless of the installed major. Add the allowlist there. The duplicate entries in pnpm-workspace.yaml and package.json#pnpm stay in place — they're harmless and serve as documentation for anyone running older pnpm. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-27 14:37:53 -07:00
TehRiehlDeal	a551ec06b4	Keep onlyBuiltDependencies in both package.json and pnpm-workspace.yaml ... The previous commit relied on pnpm-workspace.yaml alone, but that field is a pnpm 10+ feature. CI and the Dockerfiles both pin pnpm 9 (via corepack / pnpm/action-setup), and pnpm 9 only reads onlyBuiltDependencies from the package.json "pnpm" field. Without it, ERR_PNPM_IGNORED_BUILDS blocked the install. Keep both definitions in sync: pnpm 10 reads the workspace file (and emits a benign warning about the package.json field), pnpm 9 reads package.json. Also includes msw, a new transitive of vitest 4.x that now needs the explicit allow. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-27 14:32:53 -07:00
TehRiehlDeal	cf7e8a5d3f	Move pnpm onlyBuiltDependencies into pnpm-workspace.yaml ... pnpm 10.x no longer reads the "pnpm" key in package.json and now hard-fails the install with ERR_PNPM_IGNORED_BUILDS when build scripts are ignored. Migrate the existing allowlist to pnpm-workspace.yaml and add msw (pulled in transitively by vitest 4.x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-27 14:28:07 -07:00
TehRiehlDeal	a8a47e38c1	Add bulk statement-import feature for transactions ... Lets a user upload a CSV, OFX/QFX, or PDF bank statement, parses the transactions, flags duplicates and possible transfers against existing records, and bulk-creates the accepted rows after a final read-only confirmation step. Backend - New `statements` module with CSV (papaparse), OFX/QFX (node-ofx-parser), and PDF (pdf-parse) parsers behind a `StatementParser` strategy interface; format detected by content sniffing + extension. - `DuplicateDetectorService` checks FITID/externalId exact matches first, then a date+amount+description Jaro-Winkler heuristic, then cross-account transfer pairing. - New `POST /statements/parse` (multipart, in-memory, 10MB cap) returns the parsed preview without writing, including per-row `status` (`new`, `duplicate`, `needs_review`, `possible_transfer`) and any `needsMapping` payload when CSV headers are unrecognized. - `POST /transactions/bulk` accepts up to 500 rows, chunks them 50 at a time inside `prisma.$transaction`, applies balance deltas, and writes a single `ActivityLog` row per chunk instead of one per transaction. - Schema: nullable `external_id` column on `Transaction` plus composite indexes on `(account_id, external_id)` and `(account_id, date)` for fast dedupe-window queries. Not encrypted — it's an opaque bank ID used as a lookup key. Frontend - `ImportStatementDialog` runs a 4-step wizard: Upload → Column Mapping (if needed) → Review (editable table with duplicate/transfer badges) → Confirm (read-only summary with projected per-account balance impact and count-bearing primary button). The Confirm step gates the actual write, and Back to Review preserves all edit/checkbox state. - New `bulkCreateTransactions` action on the transactions store. - "Import Statement" button added next to Export on the Transactions page, with a success toast and a refresh of the transactions + accounts stores. Tests - 306 backend tests (29 suites), 195 frontend tests (31 suites), all green. - Fixtures under `test/fixtures/statements/` cover three CSV sign conventions (signed-amount, debit/credit, credit-card), OFX 1.x SGML, OFX 2.x XML, and a credit-card QFX with the CCSTMTRS branch. Versions bumped to 0.4.0 on both packages per the lockstep rule. NOTE: the Prisma migration in `prisma/migrations/20260527203542_add_transaction_external_id/` still needs to be applied to the live database with `prisma migrate deploy` — the DB at 10.0.3.82 wasn't reachable from the dev environment. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-27 14:23:45 -07:00