Remove legacy TypeScript financial surface mapping, make Rust JSON single source of truth
- Delete standard-template.ts, surface.ts, materialize.ts (dead code) - Delete financial-taxonomy.test.ts (relied on removed code) - Add missing income statement surfaces to core.surface.json - Add cost_of_revenue mapping to core.income-bridge.json - Refactor standardize.ts to remove template dependency - Simplify financial-taxonomy.ts to use only DB snapshots - Add architecture documentation
This commit is contained in:
145
docs/architecture/financial-surfaces.md
Normal file
145
docs/architecture/financial-surfaces.md
Normal file
@@ -0,0 +1,145 @@
|
||||
# Financial Surface Definitions Architecture
|
||||
|
||||
## Overview
|
||||
|
||||
As of Issue #26, the financial statement mapping architecture follows a **Rust-first approach** where the Rust sidecar is the authoritative source for surface definitions.
|
||||
|
||||
**All legacy TypeScript template code has been removed.**
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ SEC EDGAR Filing │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ Rust Sidecar (fiscal-xbrl) │
|
||||
│ ┌─────────────────────────────────────────────────────────┐ │
|
||||
│ │ rust/taxonomy/fiscal/v1/core.surface.json │ │
|
||||
│ │ rust/taxonomy/fiscal/v1/core.income-bridge.json │ │
|
||||
│ └─────────────────────────────────────────────────────────┘ │
|
||||
│ ┌─────────────────────────────────────────────────────────┐ │
|
||||
│ │ surface_mapper.rs - builds surface_rows │ │
|
||||
│ │ kpi_mapper.rs - builds kpi_rows │ │
|
||||
│ └─────────────────────────────────────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ SQLite Database │
|
||||
│ filing_taxonomy_snapshot.surface_rows │
|
||||
│ filing_taxonomy_snapshot.detail_rows │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ TypeScript Layer │
|
||||
│ financial-taxonomy.ts:aggregateSurfaceRows() │
|
||||
│ - Reads surface_rows from DB snapshots │
|
||||
│ - Aggregates across selected periods │
|
||||
│ - Returns to frontend for display │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Source of Truth
|
||||
|
||||
### Authoritative Sources (Edit These)
|
||||
1. **`rust/taxonomy/fiscal/v1/core.surface.json`**
|
||||
- Defines all surface keys, labels, categories, orders, and formulas
|
||||
- Example: `revenue`, `cost_of_revenue`, `gross_profit`, `net_income`
|
||||
|
||||
2. **`rust/taxonomy/fiscal/v1/core.income-bridge.json`**
|
||||
- Maps XBRL concepts to income statement surfaces
|
||||
- Defines component surfaces for formula derivation
|
||||
|
||||
### Removed Files (Do NOT Recreate)
|
||||
The following files have been **permanently removed**:
|
||||
1. ~~`lib/server/financials/standard-template.ts`~~ - Template definitions (now in Rust JSON)
|
||||
2. ~~`lib/server/financials/surface.ts`~~ - Fallback surface builder (no longer needed)
|
||||
3. ~~`lib/server/financials/standardize.ts`~~ - Template-based row builder (replaced by Rust)
|
||||
|
||||
### Remaining TypeScript Helpers
|
||||
`lib/server/financials/standardize.ts` (simplified version) contains only:
|
||||
- `buildLtmStandardizedRows` - Computes LTM values from quarterly data
|
||||
- `buildDimensionBreakdown` - Builds dimension breakdowns from facts
|
||||
|
||||
These operate on already-mapped surface data from the Rust sidecar.
|
||||
|
||||
2. **`lib/server/financials/standardize.ts`**
|
||||
- Contains `buildStandardizedRows` - kept for fallback/testing only
|
||||
- Marked as `@deprecated`
|
||||
|
||||
## How to Add a New Surface
|
||||
|
||||
1. **Add to `rust/taxonomy/fiscal/v1/core.surface.json`**:
|
||||
```json
|
||||
{
|
||||
"surface_key": "new_metric",
|
||||
"statement": "income",
|
||||
"label": "New Metric",
|
||||
"category": "surface",
|
||||
"order": 100,
|
||||
"unit": "currency",
|
||||
"rollup_policy": "direct_only",
|
||||
"allowed_source_concepts": ["us-gaap:NewMetricConcept"],
|
||||
"allowed_authoritative_concepts": ["us-gaap:NewMetricConcept"],
|
||||
"formula_fallback": null,
|
||||
"detail_grouping_policy": "top_level_only",
|
||||
"materiality_policy": "income_default"
|
||||
}
|
||||
```
|
||||
|
||||
2. **Add concept mapping to `core.income-bridge.json`** (if needed):
|
||||
```json
|
||||
"new_metric": {
|
||||
"direct_authoritative_concepts": ["us-gaap:NewMetricConcept"],
|
||||
"direct_source_concepts": ["NewMetricConcept"],
|
||||
"component_surfaces": { "positive": [], "negative": [] },
|
||||
"component_concept_groups": { "positive": [], "negative": [] },
|
||||
"formula": "direct",
|
||||
"not_meaningful_for_pack": false,
|
||||
"warning_codes_when_used": []
|
||||
}
|
||||
```
|
||||
|
||||
3. **Rebuild the Rust sidecar**:
|
||||
```bash
|
||||
cd rust && cargo build --release
|
||||
```
|
||||
|
||||
4. **Re-ingest filings** to populate the new surface
|
||||
|
||||
## Key Surfaces
|
||||
### Income Statement
|
||||
| Key | Order | Description |
|
||||
|-----|------|-------------|
|
||||
| `revenue` | 10 | Top-line revenue |
|
||||
| `cost_of_revenue` | 20 | Cost of revenue/COGS |
|
||||
| `gross_profit` | 30 | Revenue - Cost of Revenue |
|
||||
| `gross_margin` | 35 | Gross Profit / Revenue (percent) |
|
||||
| `operating_expenses` | 40 | Total operating expenses |
|
||||
| `operating_income` | 60 | Gross Profit - Operating Expenses |
|
||||
| `operating_margin` | 65 | Operating Income / Revenue (percent) |
|
||||
| `pretax_income` | 80 | Income before taxes |
|
||||
| `income_tax_expense` | 85 | Income tax provision |
|
||||
| `effective_tax_rate` | 87 | Tax Expense / Pretax Income (percent) |
|
||||
| `ebitda` | 88 | Operating Income + D&A |
|
||||
| `net_income` | 90 | Bottom-line net income |
|
||||
| `diluted_eps` | 100 | Diluted earnings per share |
|
||||
| `basic_eps` | 105 | Basic earnings per share |
|
||||
| `diluted_shares` | 110 | Weighted avg diluted shares |
|
||||
| `basic_shares` | 115 | Weighted avg basic shares |
|
||||
|
||||
### Balance Sheet
|
||||
See `rust/taxonomy/fiscal/v1/core.surface.json` for complete list.
|
||||
|
||||
### Cash Flow Statement
|
||||
See `rust/taxonomy/fiscal/v1/core.surface.json` for complete list.
|
||||
|
||||
## Related Files
|
||||
- `rust/fiscal-xbrl-core/src/surface_mapper.rs` - Surface resolution logic
|
||||
- `rust/fiscal-xbrl-core/src/taxonomy_loader.rs` - JSON loading
|
||||
- `lib/server/repos/filing-taxonomy.ts` - DB operations
|
||||
- `lib/server/financial-taxonomy.ts` - Main entry point
|
||||
Reference in New Issue
Block a user