rebuild app as turbopack-first single-stack with internal api and openclaw tasks

README.md

@@ -1,60 +1,30 @@
-# Fiscal Clone 2.0
+# Fiscal Clone 3.0 (Turbopack Rebuild)
 
-Ground-up rebuild of a `fiscal.ai`-style platform with:
-- Better Auth for session-backed auth
-- Next.js frontend
-- high-throughput API service
-- durable long-running task worker
-- OpenClaw/ZeroClaw AI integration
-- futuristic terminal UI language
+Ground-up rebuild into a single Next.js 16 application that runs with Turbopack and internal API routes.
 
-## Feature Coverage
+## What changed
 
-- Authentication (email/password via Better Auth)
-- Watchlist management
-- SEC filings ingestion (10-K, 10-Q, 8-K)
-- Filing analysis jobs (async AI pipeline)
-- Portfolio holdings and summary analytics
-- Price refresh jobs (async)
-- AI portfolio insight jobs (async)
-- Task tracking endpoint and UI polling
+- Removed hard runtime dependency on the external backend for core app workflows.
+- Added internal `app/api/*` services for watchlist, portfolio, filings, tasks, and health.
+- Added durable local data store at runtime (`frontend/data/store.json`).
+- Added async task engine with retry support for:
+  - `sync_filings`
+  - `refresh_prices`
+  - `analyze_filing`
+  - `portfolio_insights`
+- Added OpenClaw integration through OpenAI-compatible `/v1/chat/completions`.
+- Enforced Turbopack for both development and production builds.
 
 ## Architecture
 
-- `frontend/`: Next.js App Router UI
-- `backend/`: Elysia API + Better Auth + domain routes
-- `backend/src/worker.ts`: durable queue worker
-- `docs/REBUILD_DECISIONS.md`: one-by-one architecture decisions
+- `frontend/`: full app (UI + API + task engine)
+- `frontend/app/api/*`: route handlers
+- `frontend/lib/server/*`: storage, task processors, SEC/pricing adapters, OpenClaw client
+- `frontend/data/store.json`: generated local runtime state (git-ignored)
 
-Runtime topology:
-1. Frontend web app
-2. Backend API
-3. Worker process for long tasks
-4. PostgreSQL
+The legacy `backend/` folder is retained in-repo but no longer required for the rebuilt local workflow.
 
-## Local Setup
-
-```bash
-cp .env.example .env
-```
-
-### 1) Backend
-
-```bash
-cd backend
-bun install
-bun run db:migrate
-bun run dev
-```
-
-### 2) Worker (new terminal)
-
-```bash
-cd backend
-bun run dev:worker
-```
-
-### 3) Frontend (new terminal)
+## Run
 
 ```bash
 cd frontend
@@ -62,74 +32,43 @@ npm install
 npm run dev
 ```
 
-Frontend: `http://localhost:3000`  
-Backend: `http://localhost:3001`  
-Swagger: `http://localhost:3001/swagger`
+Open: [http://localhost:3000](http://localhost:3000)
 
-## Docker Compose
+## Build (Turbopack)
 
 ```bash
-docker compose up --build
+cd frontend
+npm run build
+npm run start
 ```
 
-This starts: `postgres`, `backend`, `worker`, `frontend`.
+## OpenClaw setup
 
-## Coolify
-
-Deploy using the root compose file and configure separate public domains for:
-- `frontend` on port `3000`
-- `backend` on port `3001`
-
-Use the full guide in `COOLIFY.md`.
-
-Critical variables for Coolify:
-- `FRONTEND_URL` = frontend public URL
-- `BETTER_AUTH_BASE_URL` = backend public URL
-- `NEXT_PUBLIC_API_URL` = backend public URL (build-time in frontend)
-
-## Core API Surface
-
-Auth:
-- `ALL /api/auth/*` (Better Auth handler)
-- `GET /api/me`
-
-Watchlist:
-- `GET /api/watchlist`
-- `POST /api/watchlist`
-- `DELETE /api/watchlist/:id`
-
-Portfolio:
-- `GET /api/portfolio/holdings`
-- `POST /api/portfolio/holdings`
-- `PATCH /api/portfolio/holdings/:id`
-- `DELETE /api/portfolio/holdings/:id`
-- `GET /api/portfolio/summary`
-- `POST /api/portfolio/refresh-prices` (queues task)
-- `POST /api/portfolio/insights/generate` (queues task)
-- `GET /api/portfolio/insights/latest`
-
-Filings:
-- `GET /api/filings?ticker=&limit=`
-- `GET /api/filings/:accessionNumber`
-- `POST /api/filings/sync` (queues task)
-- `POST /api/filings/:accessionNumber/analyze` (queues task)
-
-Task tracking:
-- `GET /api/tasks`
-- `GET /api/tasks/:taskId`
-
-## OpenClaw / ZeroClaw Integration
-
-Set these in `.env`:
+Set in environment (for example `frontend/.env.local`):
 
 ```env
 OPENCLAW_BASE_URL=http://localhost:4000
-OPENCLAW_API_KEY=...
+OPENCLAW_API_KEY=your_key
 OPENCLAW_MODEL=zeroclaw
+SEC_USER_AGENT=Fiscal Clone <support@fiscal.local>
 ```
 
-The backend expects an OpenAI-compatible `/v1/chat/completions` endpoint.
+If OpenClaw is not configured, the app falls back to local analysis responses so task flows remain testable.
 
-## Decision Log
+## API surface
 
-See `docs/REBUILD_DECISIONS.md` for the detailed rationale and tradeoffs behind each major design choice.
+- `GET /api/health`
+- `GET /api/me`
+- `GET|POST /api/watchlist`
+- `DELETE /api/watchlist/:id`
+- `GET|POST /api/portfolio/holdings`
+- `PATCH|DELETE /api/portfolio/holdings/:id`
+- `GET /api/portfolio/summary`
+- `POST /api/portfolio/refresh-prices`
+- `POST /api/portfolio/insights/generate`
+- `GET /api/portfolio/insights/latest`
+- `GET /api/filings`
+- `POST /api/filings/sync`
+- `POST /api/filings/:accessionNumber/analyze`
+- `GET /api/tasks`
+- `GET /api/tasks/:taskId`