Francy51/Neon-Desk
2
0
Fork 0
Code Issues 22 Pull Requests Actions Packages Projects Releases Wiki Activity

feat: rebuild fiscal clone architecture and harden coolify deployment

Browse Source
This commit is contained in:
francy51
2026-02-23 21:10:39 -05:00
parent cae7cbb98f
commit 04e5caf4e1
61 changed files with 3826 additions and 2923 deletions
Download Patch File Download Diff File Expand all files Collapse all files

412
README.md
View File

@@ -1,367 +1,135 @@
# Fiscal Clone
# Fiscal Clone 2.0
Financial filings extraction and portfolio analytics powered by SEC EDGAR.
Ground-up rebuild of a `fiscal.ai`-style platform with:
- Better Auth for session-backed auth
- Next.js frontend
- high-throughput API service
- durable long-running task worker
- OpenClaw/ZeroClaw AI integration
- futuristic terminal UI language
## Features
## Feature Coverage
- **SEC Filings Extraction**
- 10-K, 10-Q, 8-K filings support
- Key metrics extraction (revenue, net income, assets, cash, debt)
- Real-time search and updates
- **Portfolio Analytics**
- Stock holdings tracking
- Real-time price updates (Yahoo Finance API)
- Automatic P&L calculations
- Performance charts (pie chart allocation, line chart performance)
- **Watchlist Management**
- Add/remove stocks to watchlist
- Track company and sector information
- Quick access to filings and portfolio
- **Authentication**
- NextAuth.js with multiple providers
- GitHub, Google OAuth, Email/Password
- JWT-based session management with 30-day expiration
- **OpenClaw Integration**
- AI portfolio insights
- AI filing analysis
- Discord notification endpoints
- Authentication (email/password via Better Auth)
- Watchlist management
- SEC filings ingestion (10-K, 10-Q, 8-K)
- Filing analysis jobs (async AI pipeline)
- Portfolio holdings and summary analytics
- Price refresh jobs (async)
- AI portfolio insight jobs (async)
- Task tracking endpoint and UI polling
## Tech Stack
## Architecture
- **Backend**: Elysia.js (Bun runtime)
- **Frontend**: Next.js 14 + TailwindCSS + Recharts
- **Database**: PostgreSQL with automatic P&L calculations
- **Data Sources**: SEC EDGAR API, Yahoo Finance API
- **Authentication**: NextAuth.js (GitHub, Google, Credentials)
- **Deployment**: Coolify (Docker Compose)
- `frontend/`: Next.js App Router UI
- `backend/`: Elysia API + Better Auth + domain routes
- `backend/src/worker.ts`: durable queue worker
- `docs/REBUILD_DECISIONS.md`: one-by-one architecture decisions
## Getting Started
Runtime topology:
1. Frontend web app
2. Backend API
3. Worker process for long tasks
4. PostgreSQL
### Prerequisites
- Node.js 20+
- Bun 1.0+
- PostgreSQL 16
- GitHub account
- Coolify instance with API access
### Installation
## Local Setup
```bash
# Clone repository
git clone https://git.b11studio.xyz/francy51/fiscal-clone.git
cd fiscal-clone
cp .env.example .env
```
# Install backend dependencies
### 1) Backend
```bash
cd backend
bun install
# Install frontend dependencies
cd frontend
npm install
# Copy environment variables
cp .env.example .env
# Edit .env with your configuration
nano .env
bun run db:migrate
bun run dev
```
### Environment Variables
```env
# Database
DATABASE_URL=postgres://postgres:your_password@localhost:5432/fiscal
POSTGRES_USER=postgres
POSTGRES_PASSWORD=your_password
POSTGRES_DB=fiscal
# Backend
PORT=3001
NODE_ENV=production
JWT_SECRET=your-jwt-secret-key-min-32-characters
GITHUB_ID=your_github_oauth_client_id
GITHUB_SECRET=your_github_oauth_client_secret
GOOGLE_ID=your_google_oauth_client_id
GOOGLE_SECRET=your_google_oauth_client_secret
# Frontend
NEXT_PUBLIC_API_URL=http://localhost:3001
```
### Running Locally
### 2) Worker (new terminal)
```bash
# Run database migrations
cd backend
bun run db:migrate
bun run dev:worker
```
# Start backend
cd backend
bun run dev
### 3) Frontend (new terminal)
# Start frontend (new terminal)
```bash
cd frontend
npm install
npm run dev
```
## Deployment via Gitea to Coolify
Frontend: `http://localhost:3000`
Backend: `http://localhost:3001`
Swagger: `http://localhost:3001/swagger`
### 1. Push to Gitea
## Docker Compose
```bash
# Initialize git
cd /data/workspace/fiscal-clone
git init
git add .
git commit -m "feat: Initial Fiscal Clone release"
# Add remote
git remote add gitea https://git.b11studio.xyz/francy51/fiscal-clone.git
# Push to Gitea
git push -u gitea:your-gitea-username main
docker compose up --build
```
### 2. Deploy to Coolify
This starts: `postgres`, `backend`, `worker`, `frontend`.
In Coolify dashboard:
## Coolify
1. **Create Application**
- Type: Docker Compose
- Name: `fiscal-clone`
- Source: Git Repository
- Repository: `git@git.b11studio.xyz:francy51/fiscal-clone.git`
- Branch: `main`
- Build Context: `/`
- Docker Compose File: `docker-compose.yml`
Deploy using the root compose file and configure separate public domains for:
- `frontend` on port `3000`
- `backend` on port `3001`
2. **Configure Domains**
- Frontend: `fiscal.b11studio.xyz`
- Backend API: `api.fiscal.b11studio.xyz`
Use the full guide in `COOLIFY.md`.
3. **Add Environment Variables**
```
DATABASE_URL=postgres://postgres:password@postgres:5432/fiscal
POSTGRES_USER=postgres
POSTGRES_PASSWORD=your-password
POSTGRES_DB=fiscal
PORT=3001
JWT_SECRET=your-jwt-secret
NEXT_PUBLIC_API_URL=http://backend:3000
GITHUB_ID=your-github-oauth-id
GITHUB_SECRET=your-github-oauth-secret
GOOGLE_ID=your-google-oauth-id
GOOGLE_SECRET=your-google-oauth-secret
```
Critical variables for Coolify:
- `FRONTEND_URL` = frontend public URL
- `BETTER_AUTH_BASE_URL` = backend public URL
- `NEXT_PUBLIC_API_URL` = backend public URL (build-time in frontend)
4. **Deploy**
## Core API Surface
## API Endpoints
Auth:
- `ALL /api/auth/*` (Better Auth handler)
- `GET /api/me`
### Authentication
- `POST /api/auth/register` - Register new user
- `POST /api/auth/login` - Login
- `POST /api/auth/verify` - Verify JWT token
Watchlist:
- `GET /api/watchlist`
- `POST /api/watchlist`
- `DELETE /api/watchlist/:id`
### SEC Filings
- `GET /api/filings` - Get all filings
- `GET /api/filings/:ticker` - Get filings by ticker
- `POST /api/filings/refresh/:ticker` - Refresh filings
Portfolio:
- `GET /api/portfolio/holdings`
- `POST /api/portfolio/holdings`
- `PATCH /api/portfolio/holdings/:id`
- `DELETE /api/portfolio/holdings/:id`
- `GET /api/portfolio/summary`
- `POST /api/portfolio/refresh-prices` (queues task)
- `POST /api/portfolio/insights/generate` (queues task)
- `GET /api/portfolio/insights/latest`
### Portfolio
- `GET /api/portfolio/:userId` - Get portfolio
- `GET /api/portfolio/:userId/summary` - Get summary
- `POST /api/portfolio` - Add holding
- `PUT /api/portfolio/:id` - Update holding
- `DELETE /api/portfolio/:id` - Delete holding
Filings:
- `GET /api/filings?ticker=&limit=`
- `GET /api/filings/:accessionNumber`
- `POST /api/filings/sync` (queues task)
- `POST /api/filings/:accessionNumber/analyze` (queues task)
### Watchlist
- `GET /api/watchlist/:userId` - Get watchlist
- `POST /api/watchlist` - Add stock
- `DELETE /api/watchlist/:id` - Remove stock
Task tracking:
- `GET /api/tasks`
- `GET /api/tasks/:taskId`
### OpenClaw Integration
- `POST /api/openclaw/notify/filing` - Discord notification
- `POST /api/openclaw/insights/portfolio` - Portfolio analysis
- `POST /api/openclaw/insights/filing` - Filing analysis
## OpenClaw / ZeroClaw Integration
## Database Schema
Set these in `.env`:
### Users
```sql
CREATE TABLE users (
id SERIAL PRIMARY KEY,
email VARCHAR(255) UNIQUE NOT NULL,
password TEXT NOT NULL,
name VARCHAR(255),
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
```env
OPENCLAW_BASE_URL=http://localhost:4000
OPENCLAW_API_KEY=...
OPENCLAW_MODEL=zeroclaw
```
### Filings
```sql
CREATE TABLE filings (
id SERIAL PRIMARY KEY,
ticker VARCHAR(10) NOT NULL,
filing_type VARCHAR(20) NOT NULL,
filing_date DATE NOT NULL,
accession_number VARCHAR(40) UNIQUE NOT NULL,
cik VARCHAR(20) NOT NULL,
company_name TEXT NOT NULL,
key_metrics JSONB,
insights TEXT,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
```
The backend expects an OpenAI-compatible `/v1/chat/completions` endpoint.
### Portfolio (with auto-calculations)
```sql
CREATE TABLE portfolio (
id SERIAL PRIMARY KEY,
user_id INTEGER REFERENCES users(id) ON DELETE CASCADE,
ticker VARCHAR(10) NOT NULL,
shares NUMERIC(20, 4) NOT NULL,
avg_cost NUMERIC(10, 4) NOT NULL,
current_price NUMERIC(10, 4),
current_value NUMERIC(20, 4),
gain_loss NUMERIC(20, 4),
gain_loss_pct NUMERIC(10, 4),
last_updated TIMESTAMP,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
UNIQUE(user_id, ticker)
);
```
## Decision Log
### Watchlist
```sql
CREATE TABLE watchlist (
id SERIAL PRIMARY KEY,
user_id INTEGER REFERENCES users(id) ON DELETE CASCADE,
ticker VARCHAR(10) NOT NULL,
company_name TEXT NOT NULL,
sector VARCHAR(100),
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
UNIQUE(user_id, ticker)
);
```
## Project Structure
```
fiscal-clone/
├── backend/
│ ├── src/
│ │ ├── index.ts # Main server
│ │ ├── db/
│ │ │ ├── index.ts # Database connection
│ │ │ └── migrate.ts # Database migrations
│ │ ├── routes/
│ │ │ ├── auth.ts # Authentication
│ │ │ ├── filings.ts # SEC filings API
│ │ │ ├── portfolio.ts # Portfolio management
│ │ │ ├── watchlist.ts # Watchlist management
│ │ │ └── openclaw.ts # AI integration
│ │ └── services/
│ │ ├── sec.ts # SEC EDGAR scraper
│ │ └── prices.ts # Yahoo Finance service
│ ├── Dockerfile
│ ├── docker-compose.yml
│ └── package.json
├── frontend/
│ ├── app/
│ │ ├── layout.tsx # Root layout
│ │ ├── page.tsx # Dashboard
│ │ ├── auth/
│ │ │ ├── signin/page.tsx # Login
│ │ │ └── signup/page.tsx # Registration
│ │ ├── portfolio/page.tsx # Portfolio management
│ │ ├── filings/page.tsx # SEC filings
│ │ └── watchlist/page.tsx # Watchlist
│ ├── lib/
│ │ ├── auth.ts # Auth helpers
│ │ └── utils.ts # Utility functions
│ ├── globals.css
│ ├── tailwind.config.js
│ ├── next.config.js
│ ├── tsconfig.json
│ └── package.json
├── docker-compose.yml # Full stack deployment
└── .env.example # Environment variables template
```
## Features Status
### ✅ Implemented
- [x] User authentication (GitHub, Google, Email/Password)
- [x] SEC EDGAR data scraping
- [x] Key metrics extraction from filings
- [x] Stock holdings tracking
- [x] Real-time price updates (Yahoo Finance)
- [x] Automatic P&L calculations
- [x] Portfolio value summary
- [x] Gain/loss tracking with percentages
- [x] Portfolio allocation pie chart
- [x] Performance line chart
- [x] Watchlist management
- [x] Add/delete holdings
- [x] Add/remove stocks from watchlist
- [x] OpenClaw AI integration endpoints
- [x] Database migrations with triggers
- [x] Full CRUD operations
- [x] Responsive design
- [x] Loading states
- [x] User feedback
- [x] Production-ready Docker configs
### 🚀 Future Enhancements
- [ ] WebSocket for real-time stock prices
- [ ] Two-factor authentication
- [ ] More filing types (S-1, 13D, DEF 14A, etc.)
- [ ] PDF parsing for full filing documents
- [ ] Export functionality (CSV, PDF)
- [ ] Mobile app
- [ ] Advanced analytics and reports
- [ ] Social features (follow portfolios, share holdings)
- [ ] Custom alerts and notifications
- [ ] Tax reporting features
## Security
- Passwords hashed with bcryptjs
- JWT tokens with 30-day expiration
- Protected routes with session checks
- CORS configured for allowed origins
- SQL injection prevention with parameterized queries
- XSS prevention with proper input sanitization
- HTTPS support (via Coolify proxy)
- Environment variables for sensitive data
## Contributing
1. Fork the repository
2. Create a feature branch
3. Commit your changes
4. Push to the branch
5. Create a Pull Request
## License
MIT License - see LICENSE file for details
## Support
For issues or questions:
- Open an issue on Gitea
- Check the documentation
- Contact the maintainers
---
**Status:** ✅ Production Ready
**Version:** 1.0.0
**Last Updated:** 2026-02-15
See `docs/REBUILD_DECISIONS.md` for the detailed rationale and tradeoffs behind each major design choice.