Francy51/Neon-Desk
2
0
Fork 0
Code Issues 22 Pull Requests Actions Packages Projects Releases Wiki Activity

[ARCH] Financial statement mapping has two parallel implementations (Rust + TypeScript) #26

New Issue
Open
opened 2026-03-15 17:06:12 +00:00 by Francy51 · 1 comment
Francy51 commented 2026-03-15 17:06:12 +00:00
Owner
Copy Link

Description

The financial statement mapping logic exists in two places:

  1. Rust (authoritative): rust/taxonomy/fiscal/v1/*.json files
  2. TypeScript (possibly unused): lib/server/financials/standard-template.ts

Evidence

The Rust sidecar CLI (rust/fiscal-xbrl-core/src/lib.rs) parses XBRL and builds surfaces using:

  • load_surface_pack() - loads JSON surface definitions
  • load_crosswalk() - loads XBRL concept mappings
  • surface_mapper.rs - performs the actual mapping

The TypeScript standard-template.ts defines similar mappings but appears to NOT be used by the actual XBRL parsing pipeline.

Impact

  1. Confusion: Developers may edit TypeScript files expecting changes, but the Rust JSON files are the actual source of truth
  2. Duplication: ~1500 lines of TypeScript code that may be dead code
  3. Maintenance burden: Changes must be made in two places
  4. Missing features: Some items defined in TypeScript (like `cost_of_revenue` surface) are NOT in the Rust JSON files, causing mapping gaps

Missing Surface Definitions

The `cost_of_revenue` surface is:

  • ✅ Defined in TypeScript `standard-template.ts` (order 20)
  • ❌ NOT in Rust `core.surface.json`
  • ✅ Referenced in Rust `core.income-bridge.json` as a component for gross_profit

This mismatch causes the cost_of_revenue row to not appear in the UI!

Proposed Fix

  1. Decide on single source of truth: Either use Rust JSON files OR TypeScript files
  2. If Rust is authoritative: Delete/deprecate TypeScript standard-template.ts and ensure all surfaces are defined in JSON
  3. If TypeScript is authoritative: Fix the Rust sidecar to use TypeScript definitions or generate JSON from TypeScript
  4. Add `cost_of_revenue` to `core.surface.json` immediately as a quick fix

Files Affected

  • `rust/taxonomy/fiscal/v1/core.surface.json`
  • `lib/server/financials/standard-template.ts`
  • Documentation
## Description The financial statement mapping logic exists in two places: 1. **Rust (authoritative)**: `rust/taxonomy/fiscal/v1/*.json` files 2. **TypeScript (possibly unused)**: `lib/server/financials/standard-template.ts` ## Evidence The Rust sidecar CLI (`rust/fiscal-xbrl-core/src/lib.rs`) parses XBRL and builds surfaces using: - `load_surface_pack()` - loads JSON surface definitions - `load_crosswalk()` - loads XBRL concept mappings - `surface_mapper.rs` - performs the actual mapping The TypeScript `standard-template.ts` defines similar mappings but appears to NOT be used by the actual XBRL parsing pipeline. ## Impact 1. **Confusion**: Developers may edit TypeScript files expecting changes, but the Rust JSON files are the actual source of truth 2. **Duplication**: ~1500 lines of TypeScript code that may be dead code 3. **Maintenance burden**: Changes must be made in two places 4. **Missing features**: Some items defined in TypeScript (like \`cost_of_revenue\` surface) are NOT in the Rust JSON files, causing mapping gaps ## Missing Surface Definitions The \`cost_of_revenue\` surface is: - ✅ Defined in TypeScript \`standard-template.ts\` (order 20) - ❌ NOT in Rust \`core.surface.json\` - ✅ Referenced in Rust \`core.income-bridge.json\` as a component for gross_profit This mismatch causes the cost_of_revenue row to not appear in the UI! ## Proposed Fix 1. **Decide on single source of truth**: Either use Rust JSON files OR TypeScript files 2. **If Rust is authoritative**: Delete/deprecate TypeScript standard-template.ts and ensure all surfaces are defined in JSON 3. **If TypeScript is authoritative**: Fix the Rust sidecar to use TypeScript definitions or generate JSON from TypeScript 4. **Add \`cost_of_revenue\` to \`core.surface.json\`** immediately as a quick fix ## Files Affected - \`rust/taxonomy/fiscal/v1/core.surface.json\` - \`lib/server/financials/standard-template.ts\` - Documentation
Francy51 commented 2026-03-15 17:19:24 +00:00
Author
Owner
Copy Link

CONFIRMED: TypeScript Code is Dead/Legacy Code

Data Flow Verified:

Rust fiscal-xbrl CLI (uses rust/taxonomy/*.json)
    ↓ (spawns via Bun.spawn in parser-client.ts)
lib/server/taxonomy/parser-client.ts
    ↓ (JSON.parse stdout, stores to DB)
Database: filing_taxonomy_snapshot table
    ↓ (SELECT surface_rows, detail_rows)
lib/server/financial-taxonomy.ts
    ↓ (aggregateSurfaceRows, aggregateDetailRows)
API Response → UI

Evidence That TypeScript Mapping Code is NOT Used:

  1. buildStandardizedRows in standardize.ts:

    • Imports from standard-template.ts
    • grep for buildStandardizedRows shows it's only called from surface.ts line 205

  2. buildCompactHydrationModel in surface.ts:

    • grep for buildCompactHydrationModel shows ZERO production callers
    • Only exists in the file where it's defined

  3. standard-template.ts (1471 lines):

    • Defines STANDARD_FINANCIAL_TEMPLATES
    • Only imported by standardize.ts
    • Which is only used by the dead surface.ts code

  4. Production Path:

    • task-processors.ts:760 calls upsertFilingTaxonomySnapshot(normalizedSnapshot)
    • The snapshot comes DIRECTLY from Rust sidecar via hydrateFilingTaxonomySnapshot()
    • TypeScript mapping code is NEVER in the pipeline

The Real Source of Truth:

File Purpose Status
rust/taxonomy/fiscal/v1/core.surface.json Surface definitions ✅ ACTIVE
rust/taxonomy/fiscal/v1/core.income-bridge.json Income statement bridge ✅ ACTIVE
rust/taxonomy/crosswalk/us-gaap.json XBRL concept mapping ✅ ACTIVE
rust/fiscal-xbrl-core/src/surface_mapper.rs Mapping logic ✅ ACTIVE
lib/server/financials/standard-template.ts TypeScript templates ❌ DEAD
lib/server/financials/standardize.ts TypeScript standardization ❌ DEAD
lib/server/financials/surface.ts TypeScript surface builder ❌ DEAD

Root Cause of Bugs:

The bugs reported in issues #20-25 are ALL in the Rust JSON files, NOT TypeScript:

  • #21 Missing COGS: cost_of_revenue missing from core.surface.json
  • #20 Duplicate rows: Concepts not in allowed_source_concepts arrays
  • #25 Cash flow unmapped: Missing concept names in core.surface.json

Recommended Actions:

  1. Immediate: Fix bugs by editing rust/taxonomy/fiscal/v1/core.surface.json

  2. Short-term:

    • Add deprecation notices to TypeScript files
    • Update documentation to point to Rust JSON as source of truth

  3. Long-term:

    • Delete dead TypeScript code (standard-template.ts, standardize.ts, surface.ts)
    • Or repurpose surface.ts if it provides value for UI-specific transformations

Test Coverage Note:

The tests in lib/server/financial-taxonomy.test.ts use __financialTaxonomyInternals.buildStandardizedRows which tests the DEAD code path. These tests should be updated to test the Rust JSON configurations instead.

## CONFIRMED: TypeScript Code is Dead/Legacy Code ### Data Flow Verified: ``` Rust fiscal-xbrl CLI (uses rust/taxonomy/*.json) ↓ (spawns via Bun.spawn in parser-client.ts) lib/server/taxonomy/parser-client.ts ↓ (JSON.parse stdout, stores to DB) Database: filing_taxonomy_snapshot table ↓ (SELECT surface_rows, detail_rows) lib/server/financial-taxonomy.ts ↓ (aggregateSurfaceRows, aggregateDetailRows) API Response → UI ``` ### Evidence That TypeScript Mapping Code is NOT Used: 1. **`buildStandardizedRows`** in `standardize.ts`: - Imports from `standard-template.ts` - `grep` for `buildStandardizedRows` shows it's only called from `surface.ts` line 205 2. **`buildCompactHydrationModel`** in `surface.ts`: - `grep` for `buildCompactHydrationModel` shows **ZERO production callers** - Only exists in the file where it's defined 3. **`standard-template.ts`** (1471 lines): - Defines `STANDARD_FINANCIAL_TEMPLATES` - Only imported by `standardize.ts` - Which is only used by the dead `surface.ts` code 4. **Production Path**: - `task-processors.ts:760` calls `upsertFilingTaxonomySnapshot(normalizedSnapshot)` - The snapshot comes DIRECTLY from Rust sidecar via `hydrateFilingTaxonomySnapshot()` - TypeScript mapping code is NEVER in the pipeline ### The Real Source of Truth: | File | Purpose | Status | |------|---------|--------| | `rust/taxonomy/fiscal/v1/core.surface.json` | Surface definitions | ✅ ACTIVE | | `rust/taxonomy/fiscal/v1/core.income-bridge.json` | Income statement bridge | ✅ ACTIVE | | `rust/taxonomy/crosswalk/us-gaap.json` | XBRL concept mapping | ✅ ACTIVE | | `rust/fiscal-xbrl-core/src/surface_mapper.rs` | Mapping logic | ✅ ACTIVE | | `lib/server/financials/standard-template.ts` | TypeScript templates | ❌ DEAD | | `lib/server/financials/standardize.ts` | TypeScript standardization | ❌ DEAD | | `lib/server/financials/surface.ts` | TypeScript surface builder | ❌ DEAD | ### Root Cause of Bugs: The bugs reported in issues #20-25 are ALL in the Rust JSON files, NOT TypeScript: - **#21 Missing COGS**: `cost_of_revenue` missing from `core.surface.json` - **#20 Duplicate rows**: Concepts not in `allowed_source_concepts` arrays - **#25 Cash flow unmapped**: Missing concept names in `core.surface.json` ### Recommended Actions: 1. **Immediate**: Fix bugs by editing `rust/taxonomy/fiscal/v1/core.surface.json` 2. **Short-term**: - Add deprecation notices to TypeScript files - Update documentation to point to Rust JSON as source of truth 3. **Long-term**: - Delete dead TypeScript code (`standard-template.ts`, `standardize.ts`, `surface.ts`) - Or repurpose `surface.ts` if it provides value for UI-specific transformations ### Test Coverage Note: The tests in `lib/server/financial-taxonomy.test.ts` use `__financialTaxonomyInternals.buildStandardizedRows` which tests the DEAD code path. These tests should be updated to test the Rust JSON configurations instead.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
P0
Critical priority
 P1
High priority
 P2
Medium priority
 P3
Low priority
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
coolify_deployment Francy51
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Francy51/Neon-Desk#26
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?