Commit Graph

56 Commits

Author SHA1 Message Date
4313058d65 Fix P0 issues in financial ingestion architecture
Some checks failed
PR Checks / typecheck-and-build (pull_request) Has been cancelled
Taxonomy Sidecar / taxonomy-sidecar (pull_request) Has been cancelled
- Wrap snapshot updates in transactions with error context for each child table
- Add sidecar retry with exponential backoff (3 attempts, 2s base, 10s max, 30% jitter)
- Add HTTP timeout (30s per request) and SEC rate limiting (10 req/s) in Rust
- Add XBRL validation with status reporting (checks root element, tag balance)
2026-03-15 16:51:32 -04:00
24aa8e33d4 Consolidate metric definitions with Rust JSON as single source of truth
- Add core.computed.json with 32 ratio definitions (filing + market derived)
- Add Rust types for ComputedDefinition and ComputationSpec
- Create generate-taxonomy.ts to generate TypeScript from Rust JSON
- Generate lib/generated/ (gitignored) with surfaces, computed, kpis
- Update financial-metrics.ts to use generated definitions
- Add build-time generation via 'bun run generate'
- Add taxonomy architecture documentation

Two-phase ratio computation:
- Filing-derived: margins, returns, per-share, growth (Rust computes)
- Market-derived: valuation ratios (TypeScript computes with price data)

All 32 ratios defined in core.computed.json:
- Margins: gross, operating, ebitda, net, fcf
- Returns: roa, roe, roic, roce
- Financial health: debt_to_equity, net_debt_to_ebitda, cash_to_debt, current_ratio
- Per-share: revenue, fcf, book_value
- Growth: yoy metrics + 3y/5y cagr
- Valuation: market_cap, ev, p/e, p/fcf, p/b, ev/sales, ev/ebitda, ev/fcf
2026-03-15 15:22:51 -04:00
ed4420b8db Add atomic task deduplication with partial unique index
- Add partial unique index for active resource-scoped tasks
- Implement createTaskRunRecordAtomic for race-free task creation
- Update findOrEnqueueTask to use atomic insert first
- Add tests for concurrent task creation deduplication
2026-03-15 14:40:38 -04:00
a7f7be50b4 Remove legacy TypeScript financial surface mapping, make Rust JSON single source of truth
- Delete standard-template.ts, surface.ts, materialize.ts (dead code)
- Delete financial-taxonomy.test.ts (relied on removed code)
- Add missing income statement surfaces to core.surface.json
- Add cost_of_revenue mapping to core.income-bridge.json
- Refactor standardize.ts to remove template dependency
- Simplify financial-taxonomy.ts to use only DB snapshots
- Add architecture documentation
2026-03-15 14:38:48 -04:00
7a42d73a48 Fix filing taxonomy schema mismatch by adding explicit column verification
The filing_taxonomy_snapshot table was missing parser_engine and related columns
on databases created before the taxonomy surface sidecar migration. This caused
filing sync workflows to fail with 'table has no column named parser_engine'.

Changes:
- Add TAXONOMY_SNAPSHOT_REQUIRED_COLUMNS constant for required columns
- Add verifyCriticalSchema() to fail fast at startup if schema is incompatible
- Reorder ensureTaxonomySnapshotCompat to check table existence before column ops
- Add explicit column verification after ALTER TABLE attempts
- Add regression tests for missing column detection

Fixes #2
2026-03-15 13:15:01 -04:00
529437c760 Stop substituting synthetic market data when providers fail
- Replace synthetic fallback in getQuote()/getPriceHistory() with null returns
- Add QuoteResult/PriceHistoryResult types with { value, stale } structure
- Implement stale-while-revalidate: return cached value with stale=true on live fetch failure
- Cache failures for 30s to avoid hammering provider
- Update CompanyAnalysis type to use PriceData<T> wrapper
- Update task-processors to track failed/stale tickers explicitly
- Update price-history-card UI to show unavailable state and stale indicator
- Add comprehensive tests for failure cases
- Add e2e tests for null data, stale data, and live data scenarios

Resolves #14
2026-03-14 23:37:12 -04:00
69b45f35e3 Make coverage filing sync explicit 2026-03-14 19:54:59 -04:00
0d6c684227 Collapse filing sync notifications into one batch surface 2026-03-14 19:32:09 -04:00
61b072d31f Fix filings ticker scope consistency 2026-03-14 19:16:04 -04:00
b735b864d2 Fix SQLite taxonomy schema bootstrap drift 2026-03-14 19:00:29 -04:00
72bf64aeec Merge branch 't3code/company-overview-loading-cache' 2026-03-13 19:05:25 -04:00
0394f4e795 Add company overview skeleton and cache 2026-03-13 19:05:17 -04:00
30977dc15f Fix financial taxonomy snapshot normalization 2026-03-13 19:01:56 -04:00
b1c9c0ef08 Merge branch 't3code/ui-yahoo-finance-description' 2026-03-13 00:28:41 -04:00
61282ec380 Use Yahoo Finance company descriptions 2026-03-13 00:28:24 -04:00
1052bdfa85 Merge branch 't3code/investigate-unmapped-details' 2026-03-13 00:20:36 -04:00
e5141238fb WIP main worktree changes before merge 2026-03-13 00:20:22 -04:00
5998066524 Fix residual detail row aggregation 2026-03-13 00:14:30 -04:00
01199d489a Add untracked chart and schema files 2026-03-13 00:11:59 -04:00
ba385586bc Rebuild company overview analysis page 2026-03-12 20:39:30 -04:00
c274f4d55b refactor(taxonomy): remove legacy parser and add rollout checks 2026-03-12 15:25:06 -04:00
58061af006 feat(taxonomy): add rust sidecar compact surface pipeline 2026-03-12 15:23:10 -04:00
f2c25fb9c6 Improve workflow error messaging 2026-03-09 23:51:37 -04:00
fae8c54121 Merge branch 't3code/improve-job-status-notification-details' 2026-03-09 18:54:02 -04:00
12a9741eca Improve job status notifications 2026-03-09 18:53:41 -04:00
9f972305e6 Fix annual financial selector and QCOM standardization 2026-03-09 18:50:59 -04:00
7a70545f09 Merge branch 't3code/expand-research-management-plan'
# Conflicts:
#	app/analysis/page.tsx
#	app/watchlist/page.tsx
#	components/shell/app-shell.tsx
#	lib/api.ts
#	lib/query/options.ts
#	lib/server/api/app.ts
#	lib/server/db/index.test.ts
#	lib/server/db/index.ts
#	lib/server/db/schema.ts
#	lib/server/repos/research-journal.ts
#	lib/types.ts
2026-03-07 20:39:49 -05:00
e20aba998b Add search and RAG workspace flows 2026-03-07 20:34:00 -05:00
62bacdf104 Add research workspace and graphing flows 2026-03-07 16:52:35 -05:00
db01f207a5 Expand financials surfaces with ratios, KPIs, and cadence support
- Add bundled financial modeling pipeline (ratios, KPI dimensions/notes, trend series, standardization)
- Introduce company financial bundles storage (Drizzle migration + repo wiring)
- Refactor financials page/API/query flow to use surfaceKind + cadence and new response shapes
2026-03-07 15:16:35 -05:00
52136271d3 Implement fiscal-style research MVP flows
Some checks failed
PR Checks / typecheck-and-build (push) Has been cancelled
2026-03-07 09:51:18 -05:00
8b1fff4130 Implement dual-surface financials and db bootstrap 2026-03-06 16:24:56 -05:00
8e62c66677 Run playwright UI tests 2026-03-06 14:40:43 -05:00
610fce8db3 Add category and tags granularity to company sync flows 2026-03-03 23:10:08 -05:00
da2ce23bab refactor: make AI runtime z.ai-only and default to glm-5 2026-03-02 22:27:39 -05:00
d81a681905 feat: migrate task jobs to workflow notifications + timeline 2026-03-02 14:29:31 -05:00
cf6f26fa15 test(financials-v2): cover statement parsing and aggregation internals 2026-03-02 09:34:51 -05:00
9b20448d93 feat(financials-v2): add financial statements endpoint and query plumbing 2026-03-02 09:34:05 -05:00
3f3182310b feat(financials-v2): hydrate filing statements and aggregate history 2026-03-02 09:33:58 -05:00
bcf4c69c92 feat(financials-v2): add statement snapshot schema and shared types 2026-03-02 09:33:44 -05:00
94a962bf8e Transpose financials tables and expose company KPI metrics 2026-03-01 19:27:06 -05:00
b55fbf0942 Auto-queue filings sync on new ticker inserts 2026-03-01 18:55:59 -05:00
2a5b548d89 Prioritize SEC financials for 10-K/10-Q and keep other filings qualitative 2026-03-01 00:37:47 -05:00
953d7c0099 Fix AI workflow retry loops and improve fallback handling 2026-02-28 22:01:07 -05:00
a09001501e Implement dual-model filing pipeline with Ollama extraction 2026-02-28 16:31:25 -05:00
0615534f4b chore: commit all current changes 2026-02-28 15:13:21 -05:00
a53a5222c1 Remove OpenClaw fallback compatibility 2026-02-28 15:05:46 -05:00
b9f3b7f9d0 Migrate AI runtime to SDK and hardcode Zhipu coding endpoint 2026-02-28 13:59:00 -05:00
73d9ccafd6 Support Nginx-authenticated OpenClaw and fix workflow retry start context 2026-02-27 11:42:50 -05:00
7c3836068f Add company analysis view with financials, price history, filings, and AI reports 2026-02-27 09:57:44 -05:00