feat: rebuild fiscal clone architecture and harden coolify deployment

docs/REBUILD_DECISIONS.md

@@ -0,0 +1,78 @@
+# Fiscal Clone Rebuild Decisions
+
+This document records the ground-up design choices for the 2026 rebuild so every major decision is explicit and reviewable.
+
+## 1) Architecture: split frontend and API
+- Decision: keep `Next.js` in `frontend/` and a dedicated high-throughput API in `backend/`.
+- Why: clean separation for scaling and deployment; web rendering and data ingestion do not contend for resources.
+- Tradeoff: more services to run locally.
+
+## 2) Runtime choice: Bun + Elysia for API
+- Decision: use Bun runtime with Elysia for low overhead and fast cold/warm request handling.
+- Why: strong performance profile for IO-heavy workloads (quotes, SEC fetch, queue polling).
+- Tradeoff: narrower ecosystem compatibility than plain Node in some libraries.
+
+## 3) Auth standard: Better Auth only
+- Decision: use Better Auth end-to-end and remove legacy JWT/NextAuth patterns.
+- Why: single auth surface across API and Next.js clients, DB-backed sessions, less custom auth code.
+- Tradeoff: schema must align closely with Better Auth expectations.
+
+## 4) Persistence: PostgreSQL as source of truth
+- Decision: keep Postgres for all domain entities and task durability.
+- Why: transactional consistency, mature operational tooling, simple backup/restore.
+- Tradeoff: queue throughput is lower than specialized brokers at massive scale.
+
+## 5) Long-running jobs: durable DB queue
+- Decision: implement a durable `long_tasks` queue table plus dedicated worker process.
+- Why: supports multi-minute jobs, retries, result persistence, and survives API restarts.
+- Tradeoff: custom queue logic is more code than dropping in a broker library.
+
+## 6) Async-first API for heavy workflows
+- Decision: filing sync, filing analysis, and portfolio insights are queued and polled via `/api/tasks/:id`.
+- Why: avoids request timeouts and keeps the UX responsive.
+- Tradeoff: frontend must handle job lifecycle states.
+
+## 7) AI integration contract for OpenClaw/ZeroClaw
+- Decision: use an adapter that targets an OpenAI-compatible chat endpoint (`OPENCLAW_BASE_URL`) with model override (`OPENCLAW_MODEL`).
+- Why: works with OpenClaw/ZeroClaw deployments while keeping provider lock-in low.
+- Tradeoff: advanced provider-specific features are not exposed in v1.
+
+## 8) SEC ingestion strategy
+- Decision: fetch filings from SEC submissions API and enrich with company facts metrics.
+- Why: stable machine-readable endpoints with less brittle parsing than HTML scraping.
+- Tradeoff: facts can lag specific filing publication timing.
+
+## 9) Market pricing strategy
+- Decision: use Yahoo Finance chart endpoint for quote snapshots and periodic refresh.
+- Why: good coverage and straightforward integration for portfolio mark-to-market.
+- Tradeoff: endpoint reliability/quotas can vary; provider abstraction retained for future switch.
+
+## 10) API shape: domain modules + strict schemas
+- Decision: organize routes by domain (`portfolio`, `watchlist`, `filings`, `ai`, `tasks`) with Zod-style schema validation via Elysia types.
+- Why: predictable contract boundaries and safer payload handling.
+- Tradeoff: slight boilerplate cost.
+
+## 11) Security posture
+- Decision: all business endpoints require authenticated session resolution through Better Auth session API.
+- Why: prevents cross-user data access and removes implicit trust in client-supplied user IDs.
+- Tradeoff: each protected route performs auth/session checks.
+
+## 12) Frontend rendering model
+- Decision: use Next.js App Router with client-heavy dashboards where live polling is required.
+- Why: server rendering for shell + interactive client zones for real-time task/market updates.
+- Tradeoff: more client-side state management in dashboard screens.
+
+## 13) Design language: terminal-futurist UI system
+- Decision: build a clear terminal-inspired design with grid scanlines, mono + geometric type pairing, and neon cyan/green accent palette.
+- Why: matches requested futuristic terminal aesthetic while remaining readable.
+- Tradeoff: highly stylized branding may not fit conservative enterprise environments.
+
+## 14) Performance defaults
+- Decision: optimize for fewer round trips (batched fetches), async processing, indexed SQL, and paginated list endpoints.
+- Why: improves p95 latency under concurrent load.
+- Tradeoff: slightly more complex query/service code.
+
+## 15) Operations model
+- Decision: run three processes in production: frontend, backend API, backend worker.
+- Why: isolates web traffic from heavy background processing and enables independent scaling.
+- Tradeoff: additional deployment/health-check wiring.