diff --git a/docs/architecture/financial-surfaces.md b/docs/architecture/financial-surfaces.md new file mode 100644 index 0000000..80704d1 --- /dev/null +++ b/docs/architecture/financial-surfaces.md @@ -0,0 +1,145 @@ +# Financial Surface Definitions Architecture + +## Overview + +As of Issue #26, the financial statement mapping architecture follows a **Rust-first approach** where the Rust sidecar is the authoritative source for surface definitions. + +**All legacy TypeScript template code has been removed.** + +## Architecture + +``` +┌─────────────────────────────────────────────────────────────────┐ +│ SEC EDGAR Filing │ +└─────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────┐ +│ Rust Sidecar (fiscal-xbrl) │ +│ ┌─────────────────────────────────────────────────────────┐ │ +│ │ rust/taxonomy/fiscal/v1/core.surface.json │ │ +│ │ rust/taxonomy/fiscal/v1/core.income-bridge.json │ │ +│ └─────────────────────────────────────────────────────────┘ │ +│ ┌─────────────────────────────────────────────────────────┐ │ +│ │ surface_mapper.rs - builds surface_rows │ │ +│ │ kpi_mapper.rs - builds kpi_rows │ │ +│ └─────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────┐ +│ SQLite Database │ +│ filing_taxonomy_snapshot.surface_rows │ +│ filing_taxonomy_snapshot.detail_rows │ +└─────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────┐ +│ TypeScript Layer │ +│ financial-taxonomy.ts:aggregateSurfaceRows() │ +│ - Reads surface_rows from DB snapshots │ +│ - Aggregates across selected periods │ +│ - Returns to frontend for display │ +└─────────────────────────────────────────────────────────────────┘ +``` + +## Source of Truth + +### Authoritative Sources (Edit These) +1. **`rust/taxonomy/fiscal/v1/core.surface.json`** + - Defines all surface keys, labels, categories, orders, and formulas + - Example: `revenue`, `cost_of_revenue`, `gross_profit`, `net_income` + +2. **`rust/taxonomy/fiscal/v1/core.income-bridge.json`** + - Maps XBRL concepts to income statement surfaces + - Defines component surfaces for formula derivation + +### Removed Files (Do NOT Recreate) +The following files have been **permanently removed**: +1. ~~`lib/server/financials/standard-template.ts`~~ - Template definitions (now in Rust JSON) +2. ~~`lib/server/financials/surface.ts`~~ - Fallback surface builder (no longer needed) +3. ~~`lib/server/financials/standardize.ts`~~ - Template-based row builder (replaced by Rust) + +### Remaining TypeScript Helpers +`lib/server/financials/standardize.ts` (simplified version) contains only: +- `buildLtmStandardizedRows` - Computes LTM values from quarterly data +- `buildDimensionBreakdown` - Builds dimension breakdowns from facts + +These operate on already-mapped surface data from the Rust sidecar. + +2. **`lib/server/financials/standardize.ts`** + - Contains `buildStandardizedRows` - kept for fallback/testing only + - Marked as `@deprecated` + +## How to Add a New Surface + +1. **Add to `rust/taxonomy/fiscal/v1/core.surface.json`**: + ```json + { + "surface_key": "new_metric", + "statement": "income", + "label": "New Metric", + "category": "surface", + "order": 100, + "unit": "currency", + "rollup_policy": "direct_only", + "allowed_source_concepts": ["us-gaap:NewMetricConcept"], + "allowed_authoritative_concepts": ["us-gaap:NewMetricConcept"], + "formula_fallback": null, + "detail_grouping_policy": "top_level_only", + "materiality_policy": "income_default" + } + ``` + +2. **Add concept mapping to `core.income-bridge.json`** (if needed): + ```json + "new_metric": { + "direct_authoritative_concepts": ["us-gaap:NewMetricConcept"], + "direct_source_concepts": ["NewMetricConcept"], + "component_surfaces": { "positive": [], "negative": [] }, + "component_concept_groups": { "positive": [], "negative": [] }, + "formula": "direct", + "not_meaningful_for_pack": false, + "warning_codes_when_used": [] + } + ``` + +3. **Rebuild the Rust sidecar**: + ```bash + cd rust && cargo build --release + ``` + +4. **Re-ingest filings** to populate the new surface + +## Key Surfaces +### Income Statement +| Key | Order | Description | +|-----|------|-------------| +| `revenue` | 10 | Top-line revenue | +| `cost_of_revenue` | 20 | Cost of revenue/COGS | +| `gross_profit` | 30 | Revenue - Cost of Revenue | +| `gross_margin` | 35 | Gross Profit / Revenue (percent) | +| `operating_expenses` | 40 | Total operating expenses | +| `operating_income` | 60 | Gross Profit - Operating Expenses | +| `operating_margin` | 65 | Operating Income / Revenue (percent) | +| `pretax_income` | 80 | Income before taxes | +| `income_tax_expense` | 85 | Income tax provision | +| `effective_tax_rate` | 87 | Tax Expense / Pretax Income (percent) | +| `ebitda` | 88 | Operating Income + D&A | +| `net_income` | 90 | Bottom-line net income | +| `diluted_eps` | 100 | Diluted earnings per share | +| `basic_eps` | 105 | Basic earnings per share | +| `diluted_shares` | 110 | Weighted avg diluted shares | +| `basic_shares` | 115 | Weighted avg basic shares | + +### Balance Sheet +See `rust/taxonomy/fiscal/v1/core.surface.json` for complete list. + +### Cash Flow Statement +See `rust/taxonomy/fiscal/v1/core.surface.json` for complete list. + +## Related Files +- `rust/fiscal-xbrl-core/src/surface_mapper.rs` - Surface resolution logic +- `rust/fiscal-xbrl-core/src/taxonomy_loader.rs` - JSON loading +- `lib/server/repos/filing-taxonomy.ts` - DB operations +- `lib/server/financial-taxonomy.ts` - Main entry point diff --git a/lib/server/financial-taxonomy.test.ts b/lib/server/financial-taxonomy.test.ts deleted file mode 100644 index 994ec17..0000000 --- a/lib/server/financial-taxonomy.test.ts +++ /dev/null @@ -1,2065 +0,0 @@ -import { describe, expect, it } from 'bun:test'; -import { - __financialTaxonomyInternals, - getCompanyFinancialTaxonomy -} from './financial-taxonomy'; -import type { FilingTaxonomySnapshotRecord } from './repos/filing-taxonomy'; -import type { - FinancialStatementKind, - FinancialStatementPeriod, - StructuredKpiRow, - TaxonomyFactRow, - TaxonomyStatementRow -} from '@/lib/types'; - -function createRow(input: { - key?: string; - label?: string; - conceptKey?: string; - qname?: string; - localName?: string; - statement?: FinancialStatementKind; - roleUri?: string | null; - order?: number; - depth?: number; - hasDimensions?: boolean; - values: Record; - sourceFactIds?: number[]; - unit?: string | null; -}): TaxonomyStatementRow { - const localName = input.localName ?? 'RevenueFromContractWithCustomerExcludingAssessedTax'; - const conceptKey = input.conceptKey ?? `http://fasb.org/us-gaap/2024#${localName}`; - const qname = input.qname ?? `us-gaap:${localName}`; - - return { - key: input.key ?? conceptKey, - label: input.label ?? localName, - conceptKey, - qname, - namespaceUri: 'http://fasb.org/us-gaap/2024', - localName, - isExtension: false, - statement: input.statement ?? 'income', - roleUri: input.roleUri ?? input.statement ?? 'income', - order: input.order ?? 1, - depth: input.depth ?? 0, - parentKey: null, - values: input.values, - units: Object.fromEntries(Object.keys(input.values).map((periodId) => [periodId, input.unit ?? 'iso4217:USD'])), - hasDimensions: input.hasDimensions ?? false, - sourceFactIds: input.sourceFactIds ?? [1] - }; -} - -function createSnapshot(input: { - filingId: number; - filingType: '10-K' | '10-Q'; - filingDate: string; - periods: Array<{ - id: string; - periodStart: string | null; - periodEnd: string; - periodLabel: string; - }>; - statement: FinancialStatementKind; - rows?: TaxonomyStatementRow[]; -}) { - const defaultRow = createRow({ - statement: input.statement, - values: Object.fromEntries(input.periods.map((period, index) => [period.id, 100 + index])) - }); - const faithfulRows = { - income: input.statement === 'income' ? (input.rows ?? [defaultRow]) : [], - balance: input.statement === 'balance' ? (input.rows ?? [{ ...defaultRow, statement: 'balance' }]) : [], - cash_flow: input.statement === 'cash_flow' ? (input.rows ?? [{ ...defaultRow, statement: 'cash_flow' }]) : [], - equity: [], - comprehensive_income: [] - } satisfies FilingTaxonomySnapshotRecord['faithful_rows']; - - return { - id: input.filingId, - filing_id: input.filingId, - ticker: 'MSFT', - filing_date: input.filingDate, - filing_type: input.filingType, - parse_status: 'ready', - parse_error: null, - source: 'xbrl_instance', - parser_engine: 'fiscal-xbrl', - parser_version: '0.1.0', - taxonomy_regime: 'us-gaap', - fiscal_pack: 'core', - periods: input.periods.map((period) => ({ - id: period.id, - filingId: input.filingId, - accessionNumber: `0000-${input.filingId}`, - filingDate: input.filingDate, - periodStart: period.periodStart, - periodEnd: period.periodEnd, - filingType: input.filingType, - periodLabel: period.periodLabel - })), - faithful_rows: faithfulRows, - statement_rows: faithfulRows, - surface_rows: { - income: [], - balance: [], - cash_flow: [], - equity: [], - comprehensive_income: [] - }, - detail_rows: { - income: {}, - balance: {}, - cash_flow: {}, - equity: {}, - comprehensive_income: {} - }, - kpi_rows: [], - derived_metrics: null, - validation_result: null, - normalization_summary: null, - facts_count: 0, - concepts_count: 0, - dimensions_count: 0, - created_at: input.filingDate, - updated_at: input.filingDate - } satisfies FilingTaxonomySnapshotRecord; -} - -function createPeriod(input: { - id: string; - filingId: number; - filingDate: string; - periodEnd: string; - periodStart?: string | null; - filingType?: '10-K' | '10-Q'; -}): FinancialStatementPeriod { - return { - id: input.id, - filingId: input.filingId, - accessionNumber: `0000-${input.filingId}`, - filingDate: input.filingDate, - periodStart: input.periodStart ?? null, - periodEnd: input.periodEnd, - filingType: input.filingType ?? '10-Q', - periodLabel: 'Test period' - }; -} - -function createDimensionFact(input: { - filingId: number; - filingDate: string; - conceptKey: string; - qname: string; - localName: string; - periodEnd: string; - value: number; - axis?: string; - member?: string; -}): TaxonomyFactRow { - return { - id: input.filingId, - snapshotId: input.filingId, - filingId: input.filingId, - filingDate: input.filingDate, - statement: 'income', - roleUri: 'income', - conceptKey: input.conceptKey, - qname: input.qname, - namespaceUri: 'http://fasb.org/us-gaap/2024', - localName: input.localName, - value: input.value, - contextId: `ctx-${input.filingId}`, - unit: 'iso4217:USD', - decimals: null, - periodStart: '2025-01-01', - periodEnd: input.periodEnd, - periodInstant: null, - dimensions: [{ - axis: input.axis ?? 'srt:ProductOrServiceAxis', - member: input.member ?? 'msft:CloudMember' - }], - isDimensionless: false, - sourceFile: null - }; -} - -function createFact(input: { - id?: number; - filingId: number; - filingDate: string; - statement?: FinancialStatementKind; - conceptKey?: string; - qname?: string; - localName: string; - periodEnd: string; - periodStart?: string | null; - periodInstant?: string | null; - value: number; - unit?: string | null; -}): TaxonomyFactRow { - const conceptKey = input.conceptKey ?? `http://fasb.org/us-gaap/2024#${input.localName}`; - const qname = input.qname ?? `us-gaap:${input.localName}`; - - return { - id: input.id ?? input.filingId, - snapshotId: input.filingId, - filingId: input.filingId, - filingDate: input.filingDate, - statement: input.statement ?? 'income', - roleUri: input.statement ?? 'income', - conceptKey, - qname, - namespaceUri: 'http://fasb.org/us-gaap/2024', - localName: input.localName, - value: input.value, - contextId: `ctx-${input.filingId}-${input.localName}`, - unit: input.unit ?? 'iso4217:USD', - decimals: null, - periodStart: input.periodStart ?? null, - periodEnd: input.periodEnd, - periodInstant: input.periodInstant ?? null, - dimensions: [], - isDimensionless: true, - sourceFile: null - }; -} - -function createKpiRow(input: { - key: string; - values: Record; - provenanceType?: StructuredKpiRow['provenanceType']; - sourceConcepts?: string[]; - sourceFactIds?: number[]; -}): StructuredKpiRow { - return { - key: input.key, - label: input.key, - category: 'operating_kpi', - unit: 'percent', - order: 10, - segment: null, - axis: null, - member: null, - values: input.values, - sourceConcepts: input.sourceConcepts ?? [], - sourceFactIds: input.sourceFactIds ?? [], - provenanceType: input.provenanceType ?? 'taxonomy', - hasDimensions: false - }; -} - -function findRow(rows: ReturnType, key: string) { - const row = rows.find((entry) => entry.key === key); - expect(row).toBeDefined(); - return row!; -} - -function findPeriodId(periods: FinancialStatementPeriod[], periodEnd: string) { - const period = periods.find((entry) => entry.periodEnd === periodEnd); - expect(period).toBeDefined(); - return period!.id; -} - -function findStandardizedResponseRow( - response: Awaited>, - key: string -) { - const row = response.statementRows?.standardized.find((entry) => entry.key === key); - expect(row).toBeDefined(); - return row!; -} - -describe('financial taxonomy internals', () => { - it('selects the primary quarter duration for 10-Q income statements', () => { - const snapshot = createSnapshot({ - filingId: 1, - filingType: '10-Q', - filingDate: '2026-01-28', - statement: 'income', - periods: [ - { id: 'instant', periodStart: null, periodEnd: '2025-12-31', periodLabel: 'Instant' }, - { id: 'quarter', periodStart: '2025-10-01', periodEnd: '2025-12-31', periodLabel: '2025-10-01 to 2025-12-31' }, - { id: 'ytd', periodStart: '2025-07-01', periodEnd: '2025-12-31', periodLabel: '2025-07-01 to 2025-12-31' } - ] - }); - - const selection = __financialTaxonomyInternals.selectPrimaryPeriodsByCadence([snapshot], 'income', 'quarterly'); - - expect(selection.periods).toHaveLength(1); - expect(selection.periods[0]?.id).toBe('quarter'); - }); - - it('selects the latest instant for balance sheets', () => { - const snapshot = createSnapshot({ - filingId: 2, - filingType: '10-K', - filingDate: '2025-07-30', - statement: 'balance', - periods: [ - { id: 'prior', periodStart: null, periodEnd: '2024-06-30', periodLabel: 'Instant' }, - { id: 'current', periodStart: null, periodEnd: '2025-06-30', periodLabel: 'Instant' } - ] - }); - - const selection = __financialTaxonomyInternals.selectPrimaryPeriodsByCadence([snapshot], 'balance', 'annual'); - - expect(selection.periods).toHaveLength(1); - expect(selection.periods[0]?.id).toBe('current'); - }); - - it('builds one reporting period per filing for the selected statement', () => { - const annual = createSnapshot({ - filingId: 10, - filingType: '10-K', - filingDate: '2025-07-30', - statement: 'income', - periods: [ - { id: 'annual', periodStart: '2024-07-01', periodEnd: '2025-06-30', periodLabel: '2024-07-01 to 2025-06-30' }, - { id: 'quarter', periodStart: '2025-04-01', periodEnd: '2025-06-30', periodLabel: '2025-04-01 to 2025-06-30' } - ] - }); - const quarterly = createSnapshot({ - filingId: 11, - filingType: '10-Q', - filingDate: '2025-10-29', - statement: 'income', - periods: [ - { id: 'instant', periodStart: null, periodEnd: '2025-09-30', periodLabel: 'Instant' }, - { id: 'quarter', periodStart: '2025-07-01', periodEnd: '2025-09-30', periodLabel: '2025-07-01 to 2025-09-30' }, - { id: 'ytd', periodStart: '2025-01-01', periodEnd: '2025-09-30', periodLabel: '2025-01-01 to 2025-09-30' } - ] - }); - - const annualPeriods = __financialTaxonomyInternals.selectPrimaryPeriodsByCadence([annual, quarterly], 'income', 'annual').periods; - const quarterlyPeriods = __financialTaxonomyInternals.selectPrimaryPeriodsByCadence([annual, quarterly], 'income', 'quarterly').periods; - - expect(annualPeriods.map((period) => period.id)).toEqual(['annual']); - expect(quarterlyPeriods.map((period) => period.id)).toEqual(['quarter']); - }); - - it('ignores future-dated fallback note periods for annual income selection', () => { - const snapshot = createSnapshot({ - filingId: 12, - filingType: '10-K', - filingDate: '2025-11-05', - statement: 'income', - periods: [ - { id: 'annual', periodStart: '2024-09-30', periodEnd: '2025-09-28', periodLabel: '2024-09-30 to 2025-09-28' }, - { id: 'future-note', periodStart: '2026-09-28', periodEnd: '2027-09-26', periodLabel: '2026-09-28 to 2027-09-26' } - ], - rows: [ - createRow({ - localName: 'Revenues', - label: 'Revenues', - values: { annual: 44_284_000_000 } - }), - createRow({ - localName: 'CostOfSales', - label: 'Cost of Sales', - values: { annual: 19_738_000_000 }, - order: 2 - }), - createRow({ - localName: 'EffectiveIncomeTaxRateContinuingOperations', - label: 'Effective Income Tax Rate Continuing Operations', - roleUri: null, - order: Number.MAX_SAFE_INTEGER, - values: { 'future-note': 0.16 }, - unit: 'pure' - }) - ] - }); - - const selection = __financialTaxonomyInternals.selectPrimaryPeriodsByCadence([snapshot], 'income', 'annual'); - - expect(selection.periods).toHaveLength(1); - expect(selection.periods[0]?.id).toBe('annual'); - }); - - it('prefers broader presented-row coverage over sparser later annual periods', () => { - const snapshot = createSnapshot({ - filingId: 13, - filingType: '10-K', - filingDate: '2025-07-30', - statement: 'income', - periods: [ - { id: 'primary', periodStart: '2024-07-01', periodEnd: '2025-06-30', periodLabel: '2024-07-01 to 2025-06-30' }, - { id: 'sparse-later', periodStart: '2024-07-16', periodEnd: '2025-07-15', periodLabel: '2024-07-16 to 2025-07-15' } - ], - rows: [ - createRow({ - localName: 'Revenues', - label: 'Revenues', - values: { primary: 245_122_000_000, 'sparse-later': 246_000_000_000 } - }), - createRow({ - localName: 'OperatingIncomeLoss', - label: 'Operating Income', - values: { primary: 109_433_000_000 }, - order: 2 - }), - createRow({ - localName: 'NetIncomeLoss', - label: 'Net Income', - values: { primary: 88_136_000_000 }, - order: 3 - }) - ] - }); - - const selection = __financialTaxonomyInternals.selectPrimaryPeriodsByCadence([snapshot], 'income', 'annual'); - - expect(selection.periods).toHaveLength(1); - expect(selection.periods[0]?.id).toBe('primary'); - }); - - it('falls back to plausible non-presented periods when no presented rows exist', () => { - const snapshot = createSnapshot({ - filingId: 14, - filingType: '10-K', - filingDate: '2025-11-05', - statement: 'income', - periods: [ - { id: 'annual', periodStart: '2024-09-30', periodEnd: '2025-09-28', periodLabel: '2024-09-30 to 2025-09-28' }, - { id: 'future-note', periodStart: '2026-09-28', periodEnd: '2027-09-26', periodLabel: '2026-09-28 to 2027-09-26' } - ], - rows: [ - createRow({ - localName: 'Revenues', - label: 'Revenues', - roleUri: null, - order: Number.MAX_SAFE_INTEGER, - values: { annual: 44_284_000_000 } - }), - createRow({ - localName: 'EffectiveIncomeTaxRateContinuingOperations', - label: 'Effective Income Tax Rate Continuing Operations', - roleUri: null, - order: Number.MAX_SAFE_INTEGER, - values: { 'future-note': 0.16 }, - unit: 'pure' - }) - ] - }); - - const selection = __financialTaxonomyInternals.selectPrimaryPeriodsByCadence([snapshot], 'income', 'annual'); - - expect(selection.periods).toHaveLength(1); - expect(selection.periods[0]?.id).toBe('annual'); - }); - - it('maps overlapping GAAP aliases into one standardized COGS row while preserving faithful rows', () => { - const period2024 = createPeriod({ - id: '2024-q4', - filingId: 30, - filingDate: '2025-01-29', - periodEnd: '2024-12-31' - }); - const period2025 = createPeriod({ - id: '2025-q4', - filingId: 31, - filingDate: '2026-01-28', - periodEnd: '2025-12-31' - }); - - const faithfulRows = __financialTaxonomyInternals.buildRows([ - createSnapshot({ - filingId: 30, - filingType: '10-Q', - filingDate: '2025-01-29', - statement: 'income', - periods: [{ - id: '2024-q4', - periodStart: '2024-10-01', - periodEnd: '2024-12-31', - periodLabel: '2024-10-01 to 2024-12-31' - }], - rows: [ - createRow({ - localName: 'CostOfRevenue', - label: 'Cost of Revenue', - values: { '2024-q4': 45_000 }, - sourceFactIds: [101] - }) - ] - }), - createSnapshot({ - filingId: 31, - filingType: '10-Q', - filingDate: '2026-01-28', - statement: 'income', - periods: [{ - id: '2025-q4', - periodStart: '2025-10-01', - periodEnd: '2025-12-31', - periodLabel: '2025-10-01 to 2025-12-31' - }], - rows: [ - createRow({ - localName: 'CostOfGoodsSold', - label: 'Cost of Goods Sold', - values: { '2025-q4': 48_000 }, - sourceFactIds: [202] - }) - ] - }) - ], 'income', new Set(['2024-q4', '2025-q4'])); - - const standardizedRows = __financialTaxonomyInternals.buildStandardizedRows( - { - rows: faithfulRows, - statement: 'income', - periods: [period2024, period2025], - facts: [] - } - ); - - expect(faithfulRows).toHaveLength(2); - - const cogs = standardizedRows.find((row) => row.key === 'cost_of_revenue'); - expect(cogs).toBeDefined(); - expect(cogs?.values['2024-q4']).toBe(45_000); - expect(cogs?.values['2025-q4']).toBe(48_000); - expect(cogs?.sourceConcepts).toEqual([ - 'us-gaap:CostOfGoodsSold', - 'us-gaap:CostOfRevenue' - ]); - expect(cogs?.sourceRowKeys).toHaveLength(2); - }); - - it('aggregates standardized dimension drill-down across mapped source concepts', () => { - const period2024 = createPeriod({ - id: '2024-q4', - filingId: 40, - filingDate: '2025-01-29', - periodEnd: '2024-12-31' - }); - const period2025 = createPeriod({ - id: '2025-q4', - filingId: 41, - filingDate: '2026-01-28', - periodEnd: '2025-12-31' - }); - const faithfulRows = [ - createRow({ - localName: 'CostOfRevenue', - label: 'Cost of Revenue', - values: { '2024-q4': 45_000 } - }), - createRow({ - localName: 'CostOfGoodsSold', - label: 'Cost of Goods Sold', - values: { '2025-q4': 48_000 } - }) - ]; - const standardizedRows = __financialTaxonomyInternals.buildStandardizedRows( - { - rows: faithfulRows, - statement: 'income', - periods: [period2024, period2025], - facts: [] - } - ); - - const breakdown = __financialTaxonomyInternals.buildDimensionBreakdown([ - createDimensionFact({ - filingId: 40, - filingDate: '2025-01-29', - conceptKey: faithfulRows[0].key, - qname: faithfulRows[0].qname, - localName: faithfulRows[0].localName, - periodEnd: '2024-12-31', - value: 20_000, - member: 'msft:ProductivityMember' - }), - createDimensionFact({ - filingId: 41, - filingDate: '2026-01-28', - conceptKey: faithfulRows[1].key, - qname: faithfulRows[1].qname, - localName: faithfulRows[1].localName, - periodEnd: '2025-12-31', - value: 28_000, - member: 'msft:IntelligentCloudMember' - }) - ], [period2024, period2025], faithfulRows, standardizedRows); - - const cogs = breakdown?.['cost_of_revenue'] ?? []; - expect(cogs).toHaveLength(2); - expect(cogs.map((row) => row.sourceLabel)).toEqual([ - 'Cost of Revenue', - 'Cost of Goods Sold' - ]); - }); - - it('prefers exact Microsoft income aliases over pro forma and reconciliation rows', () => { - const period = createPeriod({ - id: '2024-fy', - filingId: 80, - filingDate: '2024-07-30', - periodStart: '2023-07-01', - periodEnd: '2024-06-30', - filingType: '10-K' - }); - - const standardizedRows = __financialTaxonomyInternals.buildStandardizedRows({ - statement: 'income', - periods: [period], - facts: [], - rows: [ - createRow({ - localName: 'BusinessAcquisitionsProFormaRevenue', - label: 'Business Acquisitions Pro Forma Revenue', - values: { '2024-fy': 247_442_000_000 } - }), - createRow({ - localName: 'RevenueFromContractWithCustomerExcludingAssessedTax', - label: 'Revenue From Contract With Customer Excluding Assessed Tax', - values: { '2024-fy': 245_122_000_000 } - }), - createRow({ - localName: 'NonoperatingIncomeExpense', - label: 'Nonoperating Income Expense', - values: { '2024-fy': -1_646_000_000 } - }), - createRow({ - localName: 'SellingAndMarketingExpense', - label: 'Selling And Marketing Expense', - values: { '2024-fy': 24_456_000_000 } - }), - createRow({ - localName: 'GeneralAndAdministrativeExpense', - label: 'General And Administrative Expense', - values: { '2024-fy': 7_609_000_000 } - }), - createRow({ - localName: 'OperatingIncomeLoss', - label: 'Operating Income Loss', - values: { '2024-fy': 109_433_000_000 } - }), - createRow({ - localName: 'BusinessAcquisitionsProFormaNetIncomeLoss', - label: 'Business Acquisitions Pro Forma Net Income Loss', - values: { '2024-fy': 88_308_000_000 }, - hasDimensions: true - }), - createRow({ - localName: 'NetIncomeLoss', - label: 'Net Income Loss', - values: { '2024-fy': 88_136_000_000 } - }), - createRow({ - localName: 'CurrentIncomeTaxExpenseBenefit', - label: 'Current Income Tax Expense Benefit', - values: { '2024-fy': 24_389_000_000 } - }), - createRow({ - localName: 'IncomeTaxExpenseBenefit', - label: 'Income Tax Expense Benefit', - values: { '2024-fy': 19_651_000_000 } - }), - createRow({ - localName: 'IncomeLossFromContinuingOperationsBeforeIncomeTaxesExtraordinaryItemsNoncontrollingInterest', - label: 'Income Loss From Continuing Operations Before Income Taxes Extraordinary Items Noncontrolling Interest', - values: { '2024-fy': 107_787_000_000 } - }), - createRow({ - localName: 'EffectiveIncomeTaxRateReconciliationInterestIncomeExpense', - label: 'Effective Income Tax Rate Reconciliation Interest Income Expense', - values: { '2024-fy': 0.011 }, - unit: 'pure' - }), - createRow({ - localName: 'InvestmentIncomeNet', - label: 'Investment Income Net', - values: { '2024-fy': 3_157_000_000 } - }), - createRow({ - localName: 'CostOfGoodsAndServicesSold', - label: 'Cost Of Goods And Services Sold', - values: { '2024-fy': 74_114_000_000 } - }) - ] - }); - - const revenue = findRow(standardizedRows, 'revenue'); - expect(revenue.values['2024-fy']).toBe(245_122_000_000); - expect(revenue.resolvedSourceRowKeys['2024-fy']).toContain('RevenueFromContractWithCustomerExcludingAssessedTax'); - - const operatingIncome = findRow(standardizedRows, 'operating_income'); - expect(operatingIncome.values['2024-fy']).toBe(109_433_000_000); - expect(operatingIncome.resolvedSourceRowKeys['2024-fy']).toContain('OperatingIncomeLoss'); - - const sga = findRow(standardizedRows, 'selling_general_and_administrative'); - expect(sga.values['2024-fy']).toBe(32_065_000_000); - expect(sga.formulaKey).toBe('selling_general_and_administrative'); - - const netIncome = findRow(standardizedRows, 'net_income'); - expect(netIncome.values['2024-fy']).toBe(88_136_000_000); - expect(netIncome.resolvedSourceRowKeys['2024-fy']).toContain('NetIncomeLoss'); - - const taxExpense = findRow(standardizedRows, 'income_tax_expense'); - expect(taxExpense.values['2024-fy']).toBe(19_651_000_000); - expect(taxExpense.resolvedSourceRowKeys['2024-fy']).toContain('IncomeTaxExpenseBenefit'); - - const pretaxIncome = findRow(standardizedRows, 'pretax_income'); - expect(pretaxIncome.values['2024-fy']).toBe(107_787_000_000); - - const interestIncome = findRow(standardizedRows, 'interest_income'); - expect(interestIncome.values['2024-fy']).toBe(3_157_000_000); - expect(interestIncome.resolvedSourceRowKeys['2024-fy']).toContain('InvestmentIncomeNet'); - - const cogs = findRow(standardizedRows, 'cost_of_revenue'); - expect(cogs.label).toBe('Cost of Sales'); - expect(cogs.values['2024-fy']).toBe(74_114_000_000); - - expect(standardizedRows.some((row) => row.key.includes('BusinessAcquisitionsProFormaRevenue'))).toBe(true); - expect(standardizedRows.some((row) => row.key.includes('BusinessAcquisitionsProFormaNetIncomeLoss'))).toBe(true); - }); - - it('merges detail rows across taxonomy-version concept drift for Microsoft residuals', () => { - const aggregated = __financialTaxonomyInternals.aggregateDetailRows({ - statement: 'income', - selectedPeriodIds: new Set(['2024-fy', '2025-fy']), - snapshots: [ - { - ...createSnapshot({ - filingId: 90, - filingType: '10-K', - filingDate: '2024-07-30', - statement: 'income', - periods: [{ - id: '2024-fy', - periodStart: '2023-07-01', - periodEnd: '2024-06-30', - periodLabel: 'FY24' - }] - }), - detail_rows: { - income: { - unmapped: [ - { - key: 'http://fasb.org/us-gaap/2024#AdvertisingExpense', - parentSurfaceKey: 'unmapped', - label: 'Advertising Expense', - conceptKey: 'http://fasb.org/us-gaap/2024#AdvertisingExpense', - qname: 'us-gaap:AdvertisingExpense', - namespaceUri: 'http://fasb.org/us-gaap/2024', - localName: 'AdvertisingExpense', - unit: 'iso4217:USD', - values: { '2024-fy': 1_500_000_000 }, - sourceFactIds: [101], - isExtension: false, - dimensionsSummary: [], - residualFlag: true - }, - { - key: 'https://xbrl.microsoft.com/2024#BusinessAcquisitionsProFormaRevenue', - parentSurfaceKey: 'unmapped', - label: 'Business Acquisitions Pro Forma Revenue', - conceptKey: 'https://xbrl.microsoft.com/2024#BusinessAcquisitionsProFormaRevenue', - qname: 'msft:BusinessAcquisitionsProFormaRevenue', - namespaceUri: 'https://xbrl.microsoft.com/2024', - localName: 'BusinessAcquisitionsProFormaRevenue', - unit: 'iso4217:USD', - values: { '2024-fy': 247_442_000_000 }, - sourceFactIds: [102], - isExtension: true, - dimensionsSummary: [], - residualFlag: true - } - ] - }, - balance: {}, - cash_flow: {}, - equity: {}, - comprehensive_income: {} - } - } satisfies FilingTaxonomySnapshotRecord, - { - ...createSnapshot({ - filingId: 91, - filingType: '10-K', - filingDate: '2025-07-30', - statement: 'income', - periods: [{ - id: '2025-fy', - periodStart: '2024-07-01', - periodEnd: '2025-06-30', - periodLabel: 'FY25' - }] - }), - detail_rows: { - income: { - unmapped: [ - { - key: 'http://fasb.org/us-gaap/2025#AdvertisingExpense', - parentSurfaceKey: 'unmapped', - label: 'Advertising Expense', - conceptKey: 'http://fasb.org/us-gaap/2025#AdvertisingExpense', - qname: 'us-gaap:AdvertisingExpense', - namespaceUri: 'http://fasb.org/us-gaap/2025', - localName: 'AdvertisingExpense', - unit: 'iso4217:USD', - values: { '2025-fy': 1_700_000_000 }, - sourceFactIds: [201], - isExtension: false, - dimensionsSummary: [], - residualFlag: true - }, - { - key: 'https://xbrl.microsoft.com/2025#BusinessAcquisitionsProFormaRevenue', - parentSurfaceKey: 'unmapped', - label: 'Business Acquisitions Pro Forma Revenue', - conceptKey: 'https://xbrl.microsoft.com/2025#BusinessAcquisitionsProFormaRevenue', - qname: 'msft:BusinessAcquisitionsProFormaRevenue', - namespaceUri: 'https://xbrl.microsoft.com/2025', - localName: 'BusinessAcquisitionsProFormaRevenue', - unit: 'iso4217:USD', - values: { '2025-fy': 219_790_000_000 }, - sourceFactIds: [202], - isExtension: true, - dimensionsSummary: [], - residualFlag: true - } - ] - }, - balance: {}, - cash_flow: {}, - equity: {}, - comprehensive_income: {} - } - } satisfies FilingTaxonomySnapshotRecord - ] - }); - - const unmapped = aggregated.unmapped ?? []; - expect(unmapped).toHaveLength(2); - - const advertising = unmapped.find((row) => row.label === 'Advertising Expense'); - expect(advertising?.values).toEqual({ - '2024-fy': 1_500_000_000, - '2025-fy': 1_700_000_000 - }); - - const proFormaRevenue = unmapped.find((row) => row.label === 'Business Acquisitions Pro Forma Revenue'); - expect(proFormaRevenue?.values).toEqual({ - '2024-fy': 247_442_000_000, - '2025-fy': 219_790_000_000 - }); - }); - - it('resolves CASY-style income gaps with ordered fallbacks and tighter interest income exclusions', () => { - const period = createPeriod({ - id: '2025-fy', - filingId: 81, - filingDate: '2025-06-23', - periodStart: '2024-05-01', - periodEnd: '2025-04-30', - filingType: '10-K' - }); - - const standardizedRows = __financialTaxonomyInternals.buildStandardizedRows({ - statement: 'income', - periods: [period], - facts: [], - rows: [ - createRow({ - localName: 'Revenues', - label: 'Revenues', - values: { '2025-fy': 15_940_899_000 } - }), - createRow({ - localName: 'CostOfGoodsAndServiceExcludingDepreciationDepletionAndAmortization', - label: 'Cost Of Goods And Service Excluding Depreciation Depletion And Amortization', - values: { '2025-fy': 12_188_496_000 } - }), - createRow({ - localName: 'OperatingExpenses', - label: 'Operating Expenses', - values: { '2025-fy': 2_552_356_000 } - }), - createRow({ - localName: 'CostOfGoodsAndServicesSoldDepreciationAndAmortization', - label: 'Cost Of Goods And Services Sold Depreciation And Amortization', - values: { '2025-fy': 403_647_000 } - }), - createRow({ - localName: 'NetIncomeLoss', - label: 'Net Income Loss', - values: { '2025-fy': 546_520_000 } - }), - createRow({ - localName: 'IncomeTaxExpenseBenefit', - label: 'Income Tax Expense Benefit', - values: { '2025-fy': 165_929_000 } - }), - createRow({ - localName: 'InterestExpense', - label: 'Interest Expense', - values: { '2025-fy': 83_951_000 } - }), - createRow({ - localName: 'InterestIncomeExpenseNet', - label: 'Interest Income Expense Net', - values: { '2025-fy': 13_102_000 } - }), - createRow({ - localName: 'EffectiveIncomeTaxRateContinuingOperations', - label: 'Effective Income Tax Rate Continuing Operations', - values: { '2025-fy': 0.233 }, - unit: 'pure' - }) - ] - }); - - expect(findRow(standardizedRows, 'cost_of_revenue').values['2025-fy']).toBe(12_188_496_000); - expect(findRow(standardizedRows, 'gross_profit').values['2025-fy']).toBe(3_752_403_000); - expect(findRow(standardizedRows, 'selling_general_and_administrative').values['2025-fy']).toBe(2_552_356_000); - expect(findRow(standardizedRows, 'pretax_income').values['2025-fy']).toBe(712_449_000); - expect(findRow(standardizedRows, 'operating_income').values['2025-fy']).toBe(796_400_000); - expect(findRow(standardizedRows, 'depreciation_and_amortization').values['2025-fy']).toBe(403_647_000); - expect(findRow(standardizedRows, 'depreciation_and_amortization_expenses').values['2025-fy']).toBe(403_647_000); - expect(findRow(standardizedRows, 'ebitda').values['2025-fy']).toBe(1_200_047_000); - expect(findRow(standardizedRows, 'effective_tax_rate').values['2025-fy']).toBe(0.233); - expect(standardizedRows.some((row) => row.key === 'interest_income')).toBe(false); - }); - - it('keeps EBITDA formula-only instead of mapping it directly from operating income', () => { - const period = createPeriod({ - id: '2024-fy', - filingId: 90, - filingDate: '2024-07-30', - periodStart: '2023-07-01', - periodEnd: '2024-06-30', - filingType: '10-K' - }); - - const standardizedRows = __financialTaxonomyInternals.buildStandardizedRows({ - statement: 'income', - periods: [period], - facts: [], - rows: [ - createRow({ - localName: 'OperatingIncomeLoss', - label: 'Operating Income Loss', - values: { '2024-fy': 109_433_000_000 } - }), - createRow({ - localName: 'DepreciationDepletionAndAmortization', - label: 'Depreciation Depletion And Amortization', - values: { '2024-fy': 22_287_000_000 } - }) - ] - }); - - const ebitda = findRow(standardizedRows, 'ebitda'); - expect(ebitda.values['2024-fy']).toBe(131_720_000_000); - expect(ebitda.formulaKey).toBe('ebitda'); - expect(ebitda.resolvedSourceRowKeys['2024-fy']).toBeNull(); - expect(ebitda.sourceRowKeys).toHaveLength(0); - }); - - it('uses balance template ordering and sections, with equity falling back to assets minus liabilities', () => { - const period = createPeriod({ - id: '2025-balance', - filingId: 95, - filingDate: '2025-07-30', - periodEnd: '2025-06-30', - filingType: '10-K' - }); - - const standardizedRows = __financialTaxonomyInternals.buildStandardizedRows({ - statement: 'balance', - periods: [period], - facts: [], - rows: [ - createRow({ - statement: 'balance', - localName: 'Assets', - label: 'Assets', - values: { '2025-balance': 619_003_000_000 } - }), - createRow({ - statement: 'balance', - localName: 'Liabilities', - label: 'Liabilities', - values: { '2025-balance': 275_524_000_000 } - }), - createRow({ - statement: 'balance', - localName: 'AssetsCurrent', - label: 'Assets Current', - values: { '2025-balance': 191_131_000_000 } - }) - ] - }); - - const currentAssets = findRow(standardizedRows, 'current_assets'); - const totalAssets = findRow(standardizedRows, 'total_assets'); - const totalLiabilities = findRow(standardizedRows, 'total_liabilities'); - const totalEquity = findRow(standardizedRows, 'total_equity'); - - expect(currentAssets.category).toBe('assets'); - expect(totalLiabilities.category).toBe('liabilities'); - expect(totalEquity.category).toBe('equity'); - expect(totalEquity.values['2025-balance']).toBe(343_479_000_000); - expect(totalEquity.formulaKey).toBe('total_equity'); - expect(currentAssets.order).toBeLessThan(totalAssets.order); - expect(totalAssets.order).toBeLessThan(totalLiabilities.order); - expect(totalLiabilities.order).toBeLessThan(totalEquity.order); - }); - - it('maps cash flow capex and computes free cash flow from the template formula', () => { - const period = createPeriod({ - id: '2024-cf', - filingId: 100, - filingDate: '2024-07-30', - periodStart: '2023-07-01', - periodEnd: '2024-06-30', - filingType: '10-K' - }); - - const standardizedRows = __financialTaxonomyInternals.buildStandardizedRows({ - statement: 'cash_flow', - periods: [period], - facts: [], - rows: [ - createRow({ - statement: 'cash_flow', - localName: 'NetCashProvidedByUsedInOperatingActivities', - label: 'Net Cash Provided By Used In Operating Activities', - values: { '2024-cf': 118_548_000_000 } - }), - createRow({ - statement: 'cash_flow', - localName: 'PaymentsToAcquirePropertyPlantAndEquipment', - label: 'Payments To Acquire Property Plant And Equipment', - values: { '2024-cf': 46_937_000_000 } - }) - ] - }); - - const capex = findRow(standardizedRows, 'capital_expenditures'); - expect(capex.values['2024-cf']).toBe(-46_937_000_000); - - const freeCashFlow = findRow(standardizedRows, 'free_cash_flow'); - expect(freeCashFlow.values['2024-cf']).toBe(71_611_000_000); - expect(freeCashFlow.formulaKey).toBe('free_cash_flow'); - }); - - it('prefers dimensionless facts over dimensioned or label-only rows for canonical balance rows', () => { - const period = createPeriod({ - id: '2025-balance', - filingId: 110, - filingDate: '2025-02-05', - periodEnd: '2024-12-31', - filingType: '10-K' - }); - - const standardizedRows = __financialTaxonomyInternals.buildStandardizedRows({ - statement: 'balance', - periods: [period], - rows: [ - createRow({ - statement: 'balance', - localName: 'Assets', - label: 'Assets', - values: { '2025-balance': 8_700_000_000 }, - hasDimensions: true - }), - createRow({ - statement: 'balance', - localName: 'IntangibleAssetsGrossExcludingGoodwill', - label: 'Intangible Assets Gross Excluding Goodwill', - values: { '2025-balance': 2_793_000_000 } - }) - ], - facts: [ - createFact({ - filingId: 110, - filingDate: '2025-02-05', - statement: 'balance', - localName: 'Assets', - periodEnd: '2024-12-31', - periodInstant: '2024-12-31', - value: 450_256_000_000 - }), - createFact({ - id: 111, - filingId: 110, - filingDate: '2025-02-05', - statement: 'balance', - localName: 'Goodwill', - periodEnd: '2024-12-31', - periodInstant: '2024-12-31', - value: 31_885_000_000 - }) - ] - }); - - const totalAssets = findRow(standardizedRows, 'total_assets'); - expect(totalAssets.values['2025-balance']).toBe(450_256_000_000); - expect(totalAssets.resolvedSourceRowKeys['2025-balance']).toContain('Assets'); - expect(standardizedRows.some((row) => row.key === 'other:http://fasb.org/us-gaap/2024#Assets')).toBe(false); - - const goodwill = findRow(standardizedRows, 'goodwill'); - expect(goodwill.values['2025-balance']).toBe(31_885_000_000); - expect(goodwill.resolvedSourceRowKeys['2025-balance']).toContain('Goodwill'); - }); - - it('uses alias priority for Apple-style long-term investments and unearned revenue rows', () => { - const period = createPeriod({ - id: '2025-balance', - filingId: 120, - filingDate: '2025-10-31', - periodEnd: '2025-09-27', - filingType: '10-K' - }); - - const standardizedRows = __financialTaxonomyInternals.buildStandardizedRows({ - statement: 'balance', - periods: [period], - rows: [ - createRow({ - statement: 'balance', - localName: 'AvailableForSaleSecuritiesDebtSecurities', - label: 'Available For Sale Securities Debt Securities', - values: { '2025-balance': 98_027_000_000 } - }), - createRow({ - statement: 'balance', - localName: 'AvailableForSaleSecuritiesDebtMaturitiesSingleMaturityDate', - label: 'Available For Sale Securities Debt Maturities Single Maturity Date', - values: { '2025-balance': 77_723_000_000 } - }), - createRow({ - statement: 'balance', - localName: 'ContractWithCustomerLiability', - label: 'Contract With Customer Liability', - values: { '2025-balance': 13_700_000_000 } - }), - createRow({ - statement: 'balance', - localName: 'ContractWithCustomerLiabilityCurrent', - label: 'Contract With Customer Liability Current', - values: { '2025-balance': 9_055_000_000 } - }) - ], - facts: [] - }); - - const longTermInvestments = findRow(standardizedRows, 'long_term_investments'); - expect(longTermInvestments.values['2025-balance']).toBe(77_723_000_000); - expect(longTermInvestments.resolvedSourceRowKeys['2025-balance']).toContain('AvailableForSaleSecuritiesDebtMaturitiesSingleMaturityDate'); - - const unearnedRevenue = findRow(standardizedRows, 'unearned_revenue'); - expect(unearnedRevenue.values['2025-balance']).toBe(9_055_000_000); - expect(unearnedRevenue.resolvedSourceRowKeys['2025-balance']).toContain('ContractWithCustomerLiabilityCurrent'); - }); - - it('maps WMS extension aliases and direct effective tax rate values', () => { - const period = createPeriod({ - id: '2025-fy', - filingId: 130, - filingDate: '2025-05-15', - periodStart: '2024-04-01', - periodEnd: '2025-03-31', - filingType: '10-K' - }); - - const standardizedRows = __financialTaxonomyInternals.buildStandardizedRows({ - statement: 'income', - periods: [period], - rows: [ - createRow({ - localName: 'CostOfGoodsAndServicesSold', - label: 'Cost Of Goods And Services Sold', - values: { '2025-fy': 340_800_000 } - }), - createRow({ - localName: 'SellingGeneralAndAdministrativeExpenseEmployeeStockOptionPlanSpecialDividendCompensation', - label: 'Selling General And Administrative Expense Employee Stock Option Plan Special Dividend Compensation', - values: { '2025-fy': 0 } - }) - ], - facts: [ - createFact({ - filingId: 130, - filingDate: '2025-05-15', - localName: 'CostOfGoodsSoldExcludingEmployeeStockOptionPlanSpecialDividendCompensation', - periodStart: '2024-04-01', - periodEnd: '2025-03-31', - value: 1_810_004_000 - }), - createFact({ - id: 131, - filingId: 130, - filingDate: '2025-05-15', - localName: 'SellingGeneralAndAdministrativeExpenseExcludingEmployeeStockOptionPlanSpecialDividendCompensation', - periodStart: '2024-04-01', - periodEnd: '2025-03-31', - value: 380_378_000 - }), - createFact({ - id: 132, - filingId: 130, - filingDate: '2025-05-15', - localName: 'EffectiveIncomeTaxRateContinuingOperations', - periodStart: '2024-04-01', - periodEnd: '2025-03-31', - value: 0.239, - unit: 'pure' - }), - createFact({ - id: 133, - filingId: 130, - filingDate: '2025-05-15', - localName: 'InterestIncomeExpenseNonoperatingNet', - periodStart: '2024-04-01', - periodEnd: '2025-03-31', - value: -91_803_000 - }) - ] - }); - - expect(findRow(standardizedRows, 'cost_of_revenue').values['2025-fy']).toBe(1_810_004_000); - expect(findRow(standardizedRows, 'selling_general_and_administrative').values['2025-fy']).toBe(380_378_000); - expect(findRow(standardizedRows, 'effective_tax_rate').values['2025-fy']).toBe(0.239); - expect(findRow(standardizedRows, 'interest_expense').values['2025-fy']).toBe(91_803_000); - }); - - it('inverts Fiscal.ai cash flow outflows and working-capital changes', () => { - const period = createPeriod({ - id: '2025-cf', - filingId: 140, - filingDate: '2025-05-15', - periodStart: '2024-04-01', - periodEnd: '2025-03-31', - filingType: '10-K' - }); - - const standardizedRows = __financialTaxonomyInternals.buildStandardizedRows({ - statement: 'cash_flow', - periods: [period], - facts: [], - rows: [ - createRow({ - statement: 'cash_flow', - localName: 'NetCashProvidedByUsedInOperatingActivities', - label: 'Net Cash Provided By Used In Operating Activities', - values: { '2025-cf': 1_000_000_000 } - }), - createRow({ - statement: 'cash_flow', - localName: 'PaymentsToAcquirePropertyPlantAndEquipment', - label: 'Payments To Acquire Property Plant And Equipment', - values: { '2025-cf': 250_000_000 } - }), - createRow({ - statement: 'cash_flow', - localName: 'IncreaseDecreaseInReceivables', - label: 'Increase Decrease In Receivables', - values: { '2025-cf': -37_487_000 } - }), - createRow({ - statement: 'cash_flow', - localName: 'IncreaseDecreaseInInventories', - label: 'Increase Decrease In Inventories', - values: { '2025-cf': 15_749_000 } - }), - createRow({ - statement: 'cash_flow', - localName: 'PaymentsForRepurchaseOfCommonStock', - label: 'Payments For Repurchase Of Common Stock', - values: { '2025-cf': 100_000_000 } - }), - createRow({ - statement: 'cash_flow', - localName: 'PaymentsOfDividends', - label: 'Payments Of Dividends', - values: { '2025-cf': 40_000_000 } - }), - createRow({ - statement: 'cash_flow', - localName: 'OtherInvestingActivitiesNet', - label: 'Other Investing Activities Net', - values: { '2025-cf': 12_000_000 } - }) - ] - }); - - expect(findRow(standardizedRows, 'changes_trade_receivables').values['2025-cf']).toBe(37_487_000); - expect(findRow(standardizedRows, 'changes_inventories').values['2025-cf']).toBe(-15_749_000); - expect(findRow(standardizedRows, 'capital_expenditures').values['2025-cf']).toBe(-250_000_000); - expect(findRow(standardizedRows, 'share_repurchases').values['2025-cf']).toBe(-100_000_000); - expect(findRow(standardizedRows, 'dividends_paid').values['2025-cf']).toBe(-40_000_000); - expect(findRow(standardizedRows, 'other_investing_activities').values['2025-cf']).toBe(-12_000_000); - expect(findRow(standardizedRows, 'free_cash_flow').values['2025-cf']).toBe(750_000_000); - }); - - it('aggregates multiple matching component rows for template rows that require it', () => { - const period = createPeriod({ - id: '2025-cf', - filingId: 150, - filingDate: '2025-05-15', - periodStart: '2024-04-01', - periodEnd: '2025-03-31', - filingType: '10-K' - }); - - const standardizedRows = __financialTaxonomyInternals.buildStandardizedRows({ - statement: 'cash_flow', - periods: [period], - facts: [], - rows: [ - createRow({ - statement: 'cash_flow', - localName: 'IncreaseDecreaseInOtherOperatingAssets', - label: 'Increase Decrease In Other Operating Assets', - values: { '2025-cf': 25_000_000 } - }), - createRow({ - statement: 'cash_flow', - localName: 'IncreaseDecreaseInOtherOperatingLiabilities', - label: 'Increase Decrease In Other Operating Liabilities', - values: { '2025-cf': -40_000_000 } - }) - ] - }); - - const otherOperating = findRow(standardizedRows, 'changes_other_operating_activities'); - expect(otherOperating.values['2025-cf']).toBe(15_000_000); - expect(otherOperating.resolvedSourceRowKeys['2025-cf']).toBeNull(); - expect(otherOperating.sourceRowKeys).toHaveLength(2); - }); - - it('aggregates CASY balance and cash-flow component rows from dimensionless facts', () => { - const balancePeriod = createPeriod({ - id: '2025-balance', - filingId: 151, - filingDate: '2025-06-23', - periodEnd: '2025-04-30', - filingType: '10-K' - }); - const cashFlowPeriod = createPeriod({ - id: '2025-cf', - filingId: 152, - filingDate: '2025-06-23', - periodStart: '2024-05-01', - periodEnd: '2025-04-30', - filingType: '10-K' - }); - - const balanceRows = __financialTaxonomyInternals.buildStandardizedRows({ - statement: 'balance', - periods: [balancePeriod], - facts: [], - rows: [ - createRow({ - statement: 'balance', - localName: 'PropertyPlantAndEquipmentAndFinanceLeaseRightOfUseAssetAfterAccumulatedDepreciationAndAmortization', - label: 'Property Plant And Equipment And Finance Lease Right Of Use Asset After Accumulated Depreciation And Amortization', - values: { '2025-balance': 5_413_244_000 } - }), - createRow({ - statement: 'balance', - localName: 'EmployeeRelatedLiabilitiesCurrent', - label: 'Employee Related Liabilities Current', - values: { '2025-balance': 80_633_000 } - }), - createRow({ - statement: 'balance', - localName: 'OtherLiabilitiesCurrent', - label: 'Other Liabilities Current', - values: { '2025-balance': 189_870_000 } - }), - createRow({ - statement: 'balance', - localName: 'AccruedPropertyTaxes', - qname: 'caseys:AccruedPropertyTaxes', - conceptKey: 'http://www.caseys.com/20250430#AccruedPropertyTaxes', - label: 'Accrued Property Taxes', - values: { '2025-balance': 59_843_000 } - }), - createRow({ - statement: 'balance', - localName: 'DeferredIncomeTaxLiabilitiesNet', - label: 'Deferred Income Tax Liabilities Net', - values: { '2025-balance': 646_905_000 } - }), - createRow({ - statement: 'balance', - localName: 'OtherLiabilitiesNoncurrent', - label: 'Other Liabilities Noncurrent', - values: { '2025-balance': 69_380_000 } - }), - createRow({ - statement: 'balance', - localName: 'LiabilitiesCurrent', - label: 'Liabilities Current', - values: { '2025-balance': 1_101_693_000 } - }), - createRow({ - statement: 'balance', - localName: 'OperatingLeaseLiability', - label: 'Operating Lease Liability', - values: { '2025-balance': 449_354_000 } - }), - createRow({ - statement: 'balance', - localName: 'FinanceLeaseLiability', - label: 'Finance Lease Liability', - values: { '2025-balance': 108_920_000 } - }) - ] - }); - - expect(findRow(balanceRows, 'property_plant_equipment').values['2025-balance']).toBe(5_413_244_000); - expect(findRow(balanceRows, 'accrued_liabilities').values['2025-balance']).toBe(330_346_000); - expect(findRow(balanceRows, 'other_long_term_liabilities').values['2025-balance']).toBe(69_380_000); - expect(findRow(balanceRows, 'total_current_liabilities').values['2025-balance']).toBe(1_101_693_000); - expect(findRow(balanceRows, 'leases').values['2025-balance']).toBe(558_274_000); - - const cashFlowRows = __financialTaxonomyInternals.buildStandardizedRows({ - statement: 'cash_flow', - periods: [cashFlowPeriod], - facts: [ - createFact({ - id: 2001, - filingId: 152, - filingDate: '2025-06-23', - statement: 'income', - localName: 'IncreaseDecreaseInDeferredIncomeTaxes', - periodStart: '2024-05-01', - periodEnd: '2025-04-30', - value: -59_958_000 - }), - createFact({ - id: 2002, - filingId: 152, - filingDate: '2025-06-23', - statement: 'cash_flow', - localName: 'OtherNoncashIncomeExpense', - periodStart: '2024-05-01', - periodEnd: '2025-04-30', - value: 4_054_000 - }), - createFact({ - id: 2003, - filingId: 152, - filingDate: '2025-06-23', - statement: 'income', - localName: 'IncreaseDecreaseInIncomeTaxes', - periodStart: '2024-05-01', - periodEnd: '2025-04-30', - value: -84_000 - }), - createFact({ - id: 2004, - filingId: 152, - filingDate: '2025-06-23', - statement: 'income', - localName: 'IncreaseDecreaseInIncomeTaxesReceivable', - periodStart: '2024-05-01', - periodEnd: '2025-04-30', - value: -15_460_000 - }), - createFact({ - id: 2005, - filingId: 152, - filingDate: '2025-06-23', - statement: 'balance', - localName: 'IncreaseDecreaseInAccruedLiabilities', - periodStart: '2024-05-01', - periodEnd: '2025-04-30', - value: 21_525_000 - }), - createFact({ - id: 2006, - filingId: 152, - filingDate: '2025-06-23', - statement: 'income', - localName: 'IncreaseDecreaseInPrepaidExpense', - periodStart: '2024-05-01', - periodEnd: '2025-04-30', - value: -3_658_000 - }), - createFact({ - id: 2007, - filingId: 152, - filingDate: '2025-06-23', - statement: 'cash_flow', - localName: 'ProceedsFromSaleOfPropertyPlantAndEquipment', - periodStart: '2024-05-01', - periodEnd: '2025-04-30', - value: 18_805_000 - }) - ], - rows: [] - }); - - expect(findRow(cashFlowRows, 'changes_income_taxes_payable').values['2025-cf']).toBe(-84_000); - expect(findRow(cashFlowRows, 'changes_accrued_expenses').values['2025-cf']).toBe(21_525_000); - expect(findRow(cashFlowRows, 'other_adjustments').values['2025-cf']).toBe(55_904_000); - expect(findRow(cashFlowRows, 'changes_other_operating_activities').values['2025-cf']).toBe(63_616_000); - expect(findRow(cashFlowRows, 'proceeds_from_sale_of_property_plant_and_equipment').values['2025-cf']).toBe(18_805_000); - }); - - it('matches local MSFT annual income regression on exact period-end dates', async () => { - const response = await getCompanyFinancialTaxonomy({ - ticker: 'MSFT', - surfaceKind: 'income_statement', - cadence: 'annual', - includeDimensions: false, - includeFacts: false, - queuedSync: false, - v3Enabled: true - }); - - const period2024 = findPeriodId(response.periods, '2024-06-30'); - - expect(response.periods.map((period) => period.periodEnd)).toEqual([ - '2022-06-30', - '2023-06-30', - '2024-06-30', - '2025-06-30' - ]); - expect(findStandardizedResponseRow(response, 'revenue').values[period2024]).toBe(245_122_000_000); - expect(findStandardizedResponseRow(response, 'operating_income').values[period2024]).toBe(109_433_000_000); - expect(findStandardizedResponseRow(response, 'net_income').values[period2024]).toBe(88_136_000_000); - expect(findStandardizedResponseRow(response, 'pretax_income').values[period2024]).toBe(107_787_000_000); - expect(findStandardizedResponseRow(response, 'income_tax_expense').values[period2024]).toBe(19_651_000_000); - expect(findStandardizedResponseRow(response, 'effective_tax_rate').values[period2024]).toBe(0.182); - expect(findStandardizedResponseRow(response, 'revenue').templateSection).toBe('statement'); - }); - - it('matches local QCOM annual income regression on exact period-end dates', async () => { - const response = await getCompanyFinancialTaxonomy({ - ticker: 'QCOM', - surfaceKind: 'income_statement', - cadence: 'annual', - includeDimensions: false, - includeFacts: false, - queuedSync: false, - v3Enabled: true - }); - - const period2024 = findPeriodId(response.periods, '2024-09-29'); - const period2025 = findPeriodId(response.periods, '2025-09-28'); - - expect(response.periods.map((period) => period.periodEnd)).toEqual([ - '2022-09-25', - '2023-09-24', - '2024-09-29', - '2025-09-28' - ]); - expect(response.periods.some((period) => period.periodEnd === '2026-09-28')).toBe(false); - expect(response.periods.some((period) => period.periodEnd === '2027-09-26')).toBe(false); - expect(findStandardizedResponseRow(response, 'revenue').values[period2024]).toBe(38_962_000_000); - expect(findStandardizedResponseRow(response, 'cost_of_revenue').values[period2024]).toBe(17_060_000_000); - expect(findStandardizedResponseRow(response, 'gross_profit').values[period2024]).toBe(21_902_000_000); - expect(findStandardizedResponseRow(response, 'selling_general_and_administrative').values[period2024]).toBe(2_759_000_000); - expect(findStandardizedResponseRow(response, 'research_and_development').values[period2024]).toBe(8_893_000_000); - expect(findStandardizedResponseRow(response, 'pretax_income').values[period2024]).toBe(10_336_000_000); - expect(findStandardizedResponseRow(response, 'ebitda').values[period2024]).toBe(11_777_000_000); - expect(findStandardizedResponseRow(response, 'effective_tax_rate').values[period2024]).toBe(0.02); - expect(findStandardizedResponseRow(response, 'revenue').values[period2025]).toBe(44_284_000_000); - expect(findStandardizedResponseRow(response, 'income_tax_expense').values[period2025]).toBe(7_122_000_000); - expect(findStandardizedResponseRow(response, 'net_income').values[period2025]).toBe(5_541_000_000); - }); - - it('matches local MSFT annual balance regression on the June 30, 2025 balance sheet', async () => { - const response = await getCompanyFinancialTaxonomy({ - ticker: 'MSFT', - surfaceKind: 'balance_sheet', - cadence: 'annual', - includeDimensions: false, - includeFacts: false, - queuedSync: false, - v3Enabled: true - }); - - const period2025 = findPeriodId(response.periods, '2025-06-30'); - - expect(findStandardizedResponseRow(response, 'total_assets').values[period2025]).toBe(619_003_000_000); - expect(findStandardizedResponseRow(response, 'total_liabilities').values[period2025]).toBe(275_524_000_000); - expect(findStandardizedResponseRow(response, 'total_equity').values[period2025]).toBe(343_479_000_000); - expect(findStandardizedResponseRow(response, 'total_assets').templateSection).toBe('assets'); - expect(findStandardizedResponseRow(response, 'total_liabilities').templateSection).toBe('liabilities'); - expect(findStandardizedResponseRow(response, 'total_equity').templateSection).toBe('equity'); - }); - - it('matches local MSFT annual cash flow regression on the June 30, 2025 filing period', async () => { - const response = await getCompanyFinancialTaxonomy({ - ticker: 'MSFT', - surfaceKind: 'cash_flow_statement', - cadence: 'annual', - includeDimensions: false, - includeFacts: false, - queuedSync: false, - v3Enabled: true - }); - - const period2025 = findPeriodId(response.periods, '2025-06-30'); - - expect(findStandardizedResponseRow(response, 'depreciation_and_amortization').values[period2025]).toBe(34_153_000_000); - expect(findStandardizedResponseRow(response, 'stock_based_compensation').values[period2025]).toBe(11_974_000_000); - expect(findStandardizedResponseRow(response, 'free_cash_flow').values[period2025]).toBe(71_611_000_000); - expect(findStandardizedResponseRow(response, 'operating_cash_flow').templateSection).toBe('operating'); - expect(findStandardizedResponseRow(response, 'free_cash_flow').templateSection).toBe('free_cash_flow'); - }); - - it('matches local CASY annual income regression on the April 30, 2025 period', async () => { - const response = await getCompanyFinancialTaxonomy({ - ticker: 'CASY', - surfaceKind: 'income_statement', - cadence: 'annual', - includeDimensions: false, - includeFacts: false, - queuedSync: false, - v3Enabled: true - }); - - const period2025 = findPeriodId(response.periods, '2025-04-30'); - - expect(findStandardizedResponseRow(response, 'revenue').values[period2025]).toBe(15_940_899_000); - expect(findStandardizedResponseRow(response, 'cost_of_revenue').values[period2025]).toBe(12_188_496_000); - expect(findStandardizedResponseRow(response, 'gross_profit').values[period2025]).toBe(3_752_403_000); - expect(findStandardizedResponseRow(response, 'selling_general_and_administrative').values[period2025]).toBe(2_552_356_000); - expect(findStandardizedResponseRow(response, 'operating_income').values[period2025]).toBe(796_400_000); - expect(findStandardizedResponseRow(response, 'pretax_income').values[period2025]).toBe(712_449_000); - expect(findStandardizedResponseRow(response, 'income_tax_expense').values[period2025]).toBe(165_929_000); - expect(findStandardizedResponseRow(response, 'net_income').values[period2025]).toBe(546_520_000); - expect(findStandardizedResponseRow(response, 'ebitda').values[period2025]).toBe(1_200_047_000); - expect(findStandardizedResponseRow(response, 'effective_tax_rate').values[period2025]).toBe(0.233); - }); - - it('matches local CASY annual balance and cash-flow regression on the April 30, 2025 period', async () => { - const balance = await getCompanyFinancialTaxonomy({ - ticker: 'CASY', - surfaceKind: 'balance_sheet', - cadence: 'annual', - includeDimensions: false, - includeFacts: false, - queuedSync: false, - v3Enabled: true - }); - const cash = await getCompanyFinancialTaxonomy({ - ticker: 'CASY', - surfaceKind: 'cash_flow_statement', - cadence: 'annual', - includeDimensions: false, - includeFacts: false, - queuedSync: false, - v3Enabled: true - }); - - const balancePeriod2025 = findPeriodId(balance.periods, '2025-04-30'); - const cashPeriod2025 = findPeriodId(cash.periods, '2025-04-30'); - - expect(findStandardizedResponseRow(balance, 'total_assets').values[balancePeriod2025]).toBe(8_208_118_000); - expect(findStandardizedResponseRow(balance, 'total_liabilities').values[balancePeriod2025]).toBe(4_699_448_000); - expect(findStandardizedResponseRow(balance, 'total_equity').values[balancePeriod2025]).toBe(3_508_670_000); - expect(findStandardizedResponseRow(balance, 'cash_and_equivalents').values[balancePeriod2025]).toBe(326_662_000); - expect(findStandardizedResponseRow(balance, 'accrued_liabilities').values[balancePeriod2025]).toBe(330_731_000); - expect(findStandardizedResponseRow(balance, 'other_long_term_liabilities').values[balancePeriod2025]).toBe(121_485_000); - - expect(findStandardizedResponseRow(cash, 'operating_cash_flow').values[cashPeriod2025]).toBe(1_090_854_000); - expect(findStandardizedResponseRow(cash, 'capital_expenditures').values[cashPeriod2025]).toBe(-506_224_000); - expect(findStandardizedResponseRow(cash, 'free_cash_flow').values[cashPeriod2025]).toBe(584_630_000); - expect(findStandardizedResponseRow(cash, 'acquisitions').values[cashPeriod2025]).toBe(-1_239_249_000); - expect(findStandardizedResponseRow(cash, 'long_term_debt_issued').values[cashPeriod2025]).toBe(1_100_000_000); - expect(findStandardizedResponseRow(cash, 'debt_repaid').values[cashPeriod2025]).toBe(-239_492_000); - expect(findStandardizedResponseRow(cash, 'dividends_paid').values[cashPeriod2025]).toBe(-72_309_000); - expect(findStandardizedResponseRow(cash, 'financing_cash_flow').values[cashPeriod2025]).toBe(755_994_000); - expect(findStandardizedResponseRow(cash, 'changes_income_taxes_payable').values[cashPeriod2025]).toBe(-84_000); - expect(findStandardizedResponseRow(cash, 'changes_accrued_expenses').values[cashPeriod2025]).toBe(21_525_000); - expect(findStandardizedResponseRow(cash, 'other_adjustments').values[cashPeriod2025]).toBe(55_904_000); - }); - - it('merges KPI rows by priority without overwriting higher-priority periods', () => { - const merged = __financialTaxonomyInternals.mergeStructuredKpiRowsByPriority([ - [ - createKpiRow({ - key: 'loan_growth', - values: { p1: 0.12 }, - sourceConcepts: ['us-gaap:LoansReceivableNetReportedAmount'], - sourceFactIds: [1] - }) - ], - [ - createKpiRow({ - key: 'loan_growth', - values: { p1: 0.11, p2: 0.09 }, - sourceConcepts: ['us-gaap:FinancingReceivableRecordedInvestment'], - sourceFactIds: [2] - }) - ], - [ - createKpiRow({ - key: 'loan_growth', - values: { p2: 0.08, p3: 0.07 }, - provenanceType: 'structured_note', - sourceFactIds: [3] - }) - ] - ]); - - expect(merged).toHaveLength(1); - expect(merged[0]?.values).toEqual({ p1: 0.12, p2: 0.09, p3: 0.07 }); - expect(merged[0]?.sourceConcepts).toEqual([ - 'us-gaap:FinancingReceivableRecordedInvestment', - 'us-gaap:LoansReceivableNetReportedAmount' - ]); - expect(merged[0]?.sourceFactIds).toEqual([1, 2, 3]); - expect(merged[0]?.provenanceType).toBe('taxonomy'); - }); - - it('builds faithful rows when persisted statement rows are missing sourceFactIds', () => { - const malformedSnapshot = { - ...createSnapshot({ - filingId: 19, - filingType: '10-K', - filingDate: '2026-02-20', - statement: 'income', - periods: [ - { id: '2025-fy', periodStart: '2025-01-01', periodEnd: '2025-12-31', periodLabel: '2025 FY' } - ] - }), - statement_rows: { - income: [{ - ...createRow({ - key: 'revenue', - label: 'Revenue', - statement: 'income', - values: { '2025-fy': 123_000_000 } - }), - sourceFactIds: undefined - } as unknown as TaxonomyStatementRow], - balance: [], - cash_flow: [], - equity: [], - comprehensive_income: [] - } - } satisfies FilingTaxonomySnapshotRecord; - - const rows = __financialTaxonomyInternals.buildRows( - [malformedSnapshot], - 'income', - new Set(['2025-fy']) - ); - - expect(rows).toHaveLength(1); - expect(rows[0]?.key).toBe('revenue'); - expect(rows[0]?.sourceFactIds).toEqual([]); - }); - - it('aggregates persisted surface rows when legacy snapshots are missing source arrays', () => { - const snapshot = { - ...createSnapshot({ - filingId: 20, - filingType: '10-K', - filingDate: '2026-02-21', - statement: 'income', - periods: [ - { id: '2025-fy', periodStart: '2025-01-01', periodEnd: '2025-12-31', periodLabel: '2025 FY' } - ] - }), - surface_rows: { - income: [{ - key: 'revenue', - label: 'Revenue', - category: 'revenue', - templateSection: 'statement', - order: 10, - unit: 'currency', - values: { '2025-fy': 123_000_000 }, - sourceConcepts: undefined, - sourceRowKeys: undefined, - sourceFactIds: undefined, - formulaKey: null, - hasDimensions: false, - resolvedSourceRowKeys: {}, - statement: 'income', - detailCount: 0, - resolutionMethod: 'direct', - confidence: 'high', - warningCodes: [] - } as unknown as FilingTaxonomySnapshotRecord['surface_rows']['income'][number]], - balance: [], - cash_flow: [], - equity: [], - comprehensive_income: [] - } - } satisfies FilingTaxonomySnapshotRecord; - - const rows = __financialTaxonomyInternals.aggregateSurfaceRows({ - snapshots: [snapshot], - statement: 'income', - selectedPeriodIds: new Set(['2025-fy']) - }); - - expect(rows).toHaveLength(1); - expect(rows[0]).toMatchObject({ - key: 'revenue', - sourceConcepts: [], - sourceRowKeys: [], - sourceFactIds: [] - }); - }); - - it('aggregates persisted detail rows when legacy snapshots are missing dimension arrays', () => { - const snapshot = { - ...createSnapshot({ - filingId: 21, - filingType: '10-K', - filingDate: '2026-02-22', - statement: 'income', - periods: [ - { id: '2025-fy', periodStart: '2025-01-01', periodEnd: '2025-12-31', periodLabel: '2025 FY' } - ] - }), - detail_rows: { - income: { - revenue: [{ - key: 'revenue_detail', - parentSurfaceKey: 'revenue', - label: 'Revenue Detail', - conceptKey: 'us-gaap:RevenueDetail', - qname: 'us-gaap:RevenueDetail', - namespaceUri: 'http://fasb.org/us-gaap/2024', - localName: 'RevenueDetail', - unit: 'iso4217:USD', - values: { '2025-fy': 123_000_000 }, - sourceFactIds: undefined, - isExtension: false, - dimensionsSummary: undefined, - residualFlag: false - } as unknown as FilingTaxonomySnapshotRecord['detail_rows']['income'][string][number]] - }, - balance: {}, - cash_flow: {}, - equity: {}, - comprehensive_income: {} - } - } satisfies FilingTaxonomySnapshotRecord; - - const rows = __financialTaxonomyInternals.aggregateDetailRows({ - snapshots: [snapshot], - statement: 'income', - selectedPeriodIds: new Set(['2025-fy']) - }); - - expect(rows.revenue).toHaveLength(1); - expect(rows.revenue?.[0]).toMatchObject({ - key: 'revenue_detail', - sourceFactIds: [], - dimensionsSummary: [] - }); - }); - - it('builds normalization metadata from snapshot fiscal pack and counts', () => { - const snapshot = { - ...createSnapshot({ - filingId: 15, - filingType: '10-Q', - filingDate: '2026-01-28', - statement: 'income', - periods: [ - { id: 'quarter', periodStart: '2025-10-01', periodEnd: '2025-12-31', periodLabel: '2025-10-01 to 2025-12-31' } - ] - }), - parser_version: '0.1.0', - fiscal_pack: 'bank_lender', - normalization_summary: { - surfaceRowCount: 5, - detailRowCount: 3, - kpiRowCount: 2, - unmappedRowCount: 4, - materialUnmappedRowCount: 1, - warnings: [] - } - } satisfies FilingTaxonomySnapshotRecord; - - expect(__financialTaxonomyInternals.buildNormalizationMetadata([snapshot])).toEqual({ - parserEngine: 'fiscal-xbrl', - regime: 'us-gaap', - fiscalPack: 'bank_lender', - parserVersion: '0.1.0', - surfaceRowCount: 5, - detailRowCount: 3, - kpiRowCount: 2, - unmappedRowCount: 4, - materialUnmappedRowCount: 1, - warnings: [] - }); - }); - - it('aggregates normalization counts and warning codes across snapshots while using the latest parser identity', () => { - const olderSnapshot = { - ...createSnapshot({ - filingId: 17, - filingType: '10-K', - filingDate: '2025-02-13', - statement: 'income', - periods: [ - { id: '2024-fy', periodStart: '2024-01-01', periodEnd: '2024-12-31', periodLabel: '2024 FY' } - ] - }), - parser_engine: 'fiscal-xbrl', - parser_version: '0.9.0', - fiscal_pack: 'core', - normalization_summary: { - surfaceRowCount: 4, - detailRowCount: 2, - kpiRowCount: 1, - unmappedRowCount: 3, - materialUnmappedRowCount: 1, - warnings: ['balance_residual_detected', 'income_sparse_mapping'] - } - } satisfies FilingTaxonomySnapshotRecord; - - const latestSnapshot = { - ...createSnapshot({ - filingId: 18, - filingType: '10-Q', - filingDate: '2026-02-13', - statement: 'income', - periods: [ - { id: '2025-q4', periodStart: '2025-10-01', periodEnd: '2025-12-31', periodLabel: '2025 Q4' } - ] - }), - parser_engine: 'fiscal-xbrl', - parser_version: '1.1.0', - fiscal_pack: 'bank_lender', - normalization_summary: { - surfaceRowCount: 6, - detailRowCount: 5, - kpiRowCount: 4, - unmappedRowCount: 2, - materialUnmappedRowCount: 0, - warnings: ['income_sparse_mapping', 'unmapped_cash_flow_bridge'] - } - } satisfies FilingTaxonomySnapshotRecord; - - expect(__financialTaxonomyInternals.buildNormalizationMetadata([olderSnapshot, latestSnapshot])).toEqual({ - parserEngine: 'fiscal-xbrl', - regime: 'us-gaap', - fiscalPack: 'bank_lender', - parserVersion: '1.1.0', - surfaceRowCount: 10, - detailRowCount: 7, - kpiRowCount: 5, - unmappedRowCount: 5, - materialUnmappedRowCount: 1, - warnings: [ - 'balance_residual_detected', - 'income_sparse_mapping', - 'unmapped_cash_flow_bridge' - ] - }); - }); - - it('retains pinned income surface rows even when they are intentionally null', () => { - const snapshot = { - ...createSnapshot({ - filingId: 16, - filingType: '10-K', - filingDate: '2026-02-13', - statement: 'income', - periods: [ - { id: '2025-fy', periodStart: '2025-01-01', periodEnd: '2025-12-31', periodLabel: '2025 FY' } - ] - }), - fiscal_pack: 'bank_lender', - surface_rows: { - income: [ - { - key: 'revenue', - label: 'Revenue', - category: 'surface', - templateSection: 'surface', - order: 10, - unit: 'currency', - values: { '2025-fy': 100_000_000 }, - sourceConcepts: ['us-gaap:TotalNetRevenues'], - sourceRowKeys: ['revenue'], - sourceFactIds: [1], - formulaKey: null, - hasDimensions: false, - resolvedSourceRowKeys: { '2025-fy': 'revenue' }, - statement: 'income', - detailCount: 0, - resolutionMethod: 'direct', - confidence: 'high', - warningCodes: [] - }, - { - key: 'gross_profit', - label: 'Gross Profit', - category: 'surface', - templateSection: 'surface', - order: 20, - unit: 'currency', - values: { '2025-fy': null }, - sourceConcepts: [], - sourceRowKeys: [], - sourceFactIds: [], - formulaKey: null, - hasDimensions: false, - resolvedSourceRowKeys: { '2025-fy': null }, - statement: 'income', - detailCount: 0, - resolutionMethod: 'not_meaningful', - confidence: 'low', - warningCodes: ['gross_profit_not_meaningful_bank_pack'] - }, - { - key: 'selling_general_and_administrative', - label: 'SG&A', - category: 'surface', - templateSection: 'surface', - order: 31, - unit: 'currency', - values: { '2025-fy': null }, - sourceConcepts: [], - sourceRowKeys: [], - sourceFactIds: [], - formulaKey: null, - hasDimensions: false, - resolvedSourceRowKeys: { '2025-fy': null }, - statement: 'income', - detailCount: 0, - resolutionMethod: 'not_meaningful', - confidence: 'low', - warningCodes: ['selling_general_and_administrative_not_meaningful_bank_pack'] - } - ], - balance: [], - cash_flow: [], - equity: [], - comprehensive_income: [] - } - } satisfies FilingTaxonomySnapshotRecord; - - const rows = __financialTaxonomyInternals.aggregateSurfaceRows({ - snapshots: [snapshot], - statement: 'income', - selectedPeriodIds: new Set(['2025-fy']) - }); - - const grossProfit = rows.find((row) => row.key === 'gross_profit'); - const sga = rows.find((row) => row.key === 'selling_general_and_administrative'); - expect(grossProfit).toBeDefined(); - expect(grossProfit?.values['2025-fy']).toBeNull(); - expect(grossProfit?.resolutionMethod).toBe('not_meaningful'); - expect(grossProfit?.warningCodes).toEqual(['gross_profit_not_meaningful_bank_pack']); - expect(sga).toBeDefined(); - expect(sga?.values['2025-fy']).toBeNull(); - expect(sga?.resolutionMethod).toBe('not_meaningful'); - expect(sga?.warningCodes).toEqual(['selling_general_and_administrative_not_meaningful_bank_pack']); - }); -}); diff --git a/lib/server/financial-taxonomy.ts b/lib/server/financial-taxonomy.ts index 2b15b0c..385d1d1 100644 --- a/lib/server/financial-taxonomy.ts +++ b/lib/server/financial-taxonomy.ts @@ -36,8 +36,7 @@ import { } from '@/lib/server/financials/bundles'; import { buildDimensionBreakdown, - buildLtmStandardizedRows, - buildStandardizedRows + buildLtmStandardizedRows } from '@/lib/server/financials/standardize'; import { buildRatioRows } from '@/lib/server/financials/ratios'; import { buildFinancialCategories, buildTrendSeries } from '@/lib/server/financials/trend-series'; @@ -620,16 +619,7 @@ function buildQuarterlyStatementSurfaceRows(input: { selectedPeriodIds: input.selectedPeriodIds }); - if (aggregatedRows.length > 0) { - return aggregatedRows; - } - - return buildStandardizedRows({ - rows: input.faithfulRows, - statement: input.statement, - periods: input.sourcePeriods, - facts: input.facts - }) as SurfaceFinancialRow[]; + return aggregatedRows; } function aggregatePersistedKpiRows(input: { @@ -1303,7 +1293,6 @@ export async function getCompanyFinancialTaxonomy(input: GetCompanyFinancialsInp export const __financialTaxonomyInternals = { buildRows, - buildStandardizedRows, buildDimensionBreakdown, buildNormalizationMetadata, aggregateSurfaceRows, diff --git a/lib/server/financials/standard-template.ts b/lib/server/financials/standard-template.ts deleted file mode 100644 index e522c3c..0000000 --- a/lib/server/financials/standard-template.ts +++ /dev/null @@ -1,1471 +0,0 @@ -import type { - FinancialStatementKind, - FinancialUnit -} from '@/lib/types'; - -export type TemplateSelectionPolicy = - | 'single_best_period_match' - | 'prefer_exact_local_name' - | 'prefer_primary_statement_concept' - | 'formula_only' - | 'aggregate_multiple_components' - | 'direct_or_formula_fallback'; - -export type TemplateFormula = - | { - kind: 'sum'; - sourceKeys: readonly string[]; - treatNullAsZero?: boolean; - } - | { - kind: 'subtract'; - left: string; - right: string; - } - | { - kind: 'divide'; - numerator: string; - denominator: string; - }; - -export type StandardTemplateRowDefinition = { - key: string; - label: string; - category: string; - includeInOutput?: boolean; - order: number; - statement: Extract; - unit: FinancialUnit; - selectionPolicy: TemplateSelectionPolicy; - matchers: { - exactLocalNames?: readonly string[]; - secondaryLocalNames?: readonly string[]; - allowedLabelPhrases?: readonly string[]; - excludeLabelPhrases?: readonly string[]; - excludeLocalNames?: readonly string[]; - }; - fallbackFormula?: TemplateFormula; - signTransform?: 'invert' | 'absolute'; -}; - -const income = ( - definition: Omit & { category?: string } -) => ({ - statement: 'income' as const, - category: definition.category ?? 'statement', - ...definition -}); - -const balance = ( - definition: Omit -) => ({ - statement: 'balance' as const, - ...definition -}); - -const cashFlow = ( - definition: Omit -) => ({ - statement: 'cash_flow' as const, - ...definition -}); - -export const STANDARD_FINANCIAL_TEMPLATES: Record< - Extract, - StandardTemplateRowDefinition[] -> = { - income: [ - income({ - key: 'revenue', - label: 'Total Revenues', - order: 10, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: [ - 'Revenues', - 'RevenueFromContractWithCustomerExcludingAssessedTax', - 'SalesRevenueNet', - 'TotalRevenuesAndOtherIncome' - ], - secondaryLocalNames: ['RevenueFromContractWithCustomerIncludingAssessedTax'], - allowedLabelPhrases: ['total revenues', 'total revenue', 'revenue', 'net sales'], - excludeLabelPhrases: ['pro forma', 'acquiree'], - excludeLocalNames: [ - 'BusinessAcquisitionsProFormaRevenue', - 'BusinessCombinationProFormaInformationRevenueOfAcquireeSinceAcquisitionDateActual' - ] - } - }), - income({ - key: 'cost_of_revenue', - label: 'Cost of Sales', - order: 20, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: [ - 'CostOfGoodsSoldExcludingEmployeeStockOptionPlanSpecialDividendCompensation', - 'CostOfRevenue', - 'CostOfGoodsSold', - 'CostOfSales', - 'CostOfGoodsAndServicesSold', - 'CostOfGoodsAndServiceExcludingDepreciationDepletionAndAmortization', - 'CostOfProductsSold', - 'CostOfServices' - ], - allowedLabelPhrases: ['cost of sales', 'cost of revenue', 'cost of goods sold'], - excludeLocalNames: ['CostOfGoodsAndServicesSoldDepreciationAndAmortization'] - } - }), - income({ - key: 'gross_profit', - label: 'Gross Profit', - order: 30, - unit: 'currency', - selectionPolicy: 'direct_or_formula_fallback', - matchers: { - exactLocalNames: ['GrossProfit'], - allowedLabelPhrases: ['gross profit'] - }, - fallbackFormula: { - kind: 'subtract', - left: 'revenue', - right: 'cost_of_revenue' - } - }), - income({ - key: 'gross_margin', - label: 'Gross Profit Margin', - order: 40, - unit: 'percent', - selectionPolicy: 'formula_only', - matchers: {}, - fallbackFormula: { - kind: 'divide', - numerator: 'gross_profit', - denominator: 'revenue' - } - }), - income({ - key: 'selling_general_and_administrative', - label: 'Selling, General & Administrative Expenses', - order: 50, - unit: 'currency', - selectionPolicy: 'direct_or_formula_fallback', - matchers: { - exactLocalNames: [ - 'SellingGeneralAndAdministrativeExpenseExcludingEmployeeStockOptionPlanSpecialDividendCompensation', - 'SellingGeneralAndAdministrativeExpense' - ], - secondaryLocalNames: ['OperatingExpenses'], - allowedLabelPhrases: ['selling general administrative', 'selling general and administrative'], - excludeLocalNames: ['SellingGeneralAndAdministrativeExpenseEmployeeStockOptionPlanSpecialDividendCompensation'] - }, - fallbackFormula: { - kind: 'sum', - sourceKeys: ['sales_and_marketing', 'general_and_administrative'] - } - }), - income({ - key: 'research_and_development', - label: 'Research & Development Expenses', - order: 60, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['ResearchAndDevelopmentExpense'], - allowedLabelPhrases: ['research development', 'research and development'] - } - }), - income({ - key: 'sales_and_marketing', - label: 'Sales & Marketing', - includeInOutput: false, - order: 61, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['SalesAndMarketingExpense', 'SellingAndMarketingExpense'], - allowedLabelPhrases: ['sales and marketing', 'selling and marketing'] - } - }), - income({ - key: 'general_and_administrative', - label: 'General & Administrative', - includeInOutput: false, - order: 62, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['GeneralAndAdministrativeExpense'], - allowedLabelPhrases: ['general and administrative'] - } - }), - income({ - key: 'operating_income', - label: 'Operating Profit', - order: 70, - unit: 'currency', - selectionPolicy: 'direct_or_formula_fallback', - matchers: { - exactLocalNames: ['OperatingIncomeLoss', 'IncomeLossFromOperations'], - allowedLabelPhrases: ['operating profit', 'operating income', 'income from operations'], - excludeLocalNames: ['NonoperatingIncomeExpense', 'OtherNonoperatingIncomeExpense'] - } - }), - income({ - key: 'operating_margin', - label: 'Operating Margin', - order: 80, - unit: 'percent', - selectionPolicy: 'formula_only', - matchers: {}, - fallbackFormula: { - kind: 'divide', - numerator: 'operating_income', - denominator: 'revenue' - } - }), - income({ - key: 'pretax_income', - label: 'Pretax Income', - order: 90, - unit: 'currency', - selectionPolicy: 'direct_or_formula_fallback', - matchers: { - exactLocalNames: [ - 'IncomeLossFromContinuingOperationsBeforeIncomeTaxesExtraordinaryItemsNoncontrollingInterest', - 'IncomeBeforeTaxExpenseBenefit', - 'PretaxIncome' - ], - allowedLabelPhrases: ['pretax income', 'income before taxes', 'income before tax'] - }, - fallbackFormula: { - kind: 'sum', - sourceKeys: ['net_income', 'income_tax_expense'] - } - }), - income({ - key: 'income_before_provision_for_income_taxes', - label: 'Income Before Provision for Income Taxes', - order: 95, - unit: 'currency', - selectionPolicy: 'direct_or_formula_fallback', - matchers: { - exactLocalNames: [ - 'IncomeLossFromContinuingOperationsBeforeIncomeTaxesExtraordinaryItemsNoncontrollingInterest', - 'IncomeBeforeTaxExpenseBenefit', - 'PretaxIncome' - ], - allowedLabelPhrases: ['income before provision for income taxes', 'income before taxes', 'pretax income'] - }, - fallbackFormula: { - kind: 'sum', - sourceKeys: ['net_income', 'income_tax_expense'] - } - }), - income({ - key: 'income_tax_expense', - label: 'Income Tax Expense', - order: 100, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['IncomeTaxExpenseBenefit'], - secondaryLocalNames: ['IncomeTaxes'], - allowedLabelPhrases: ['income tax expense', 'income tax provision'], - excludeLocalNames: [ - 'CurrentIncomeTaxExpenseBenefit', - 'DeferredIncomeTaxExpenseBenefit', - 'DeferredFederalIncomeTaxExpenseBenefit', - 'DeferredForeignIncomeTaxExpenseBenefit', - 'DeferredStateAndLocalIncomeTaxExpenseBenefit' - ], - excludeLabelPhrases: ['current income tax', 'deferred income tax'] - } - }), - income({ - key: 'provision_for_income_taxes', - label: 'Provision for Income Taxes', - order: 105, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['IncomeTaxExpenseBenefit'], - secondaryLocalNames: ['IncomeTaxes'], - allowedLabelPhrases: ['provision for income taxes', 'income tax provision'] - } - }), - income({ - key: 'net_income', - label: 'Net Income', - order: 110, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['NetIncomeLossAvailableToCommonStockholdersBasic', 'NetIncomeLoss', 'ProfitLoss'], - allowedLabelPhrases: ['net income', 'net earnings'], - excludeLocalNames: ['BusinessAcquisitionsProFormaNetIncomeLoss'], - excludeLabelPhrases: ['pro forma'] - } - }), - income({ - key: 'consolidated_net_income', - label: 'Consolidated Net Income', - order: 112, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['NetIncomeLoss', 'ProfitLoss'], - allowedLabelPhrases: ['consolidated net income', 'net income'] - } - }), - income({ - key: 'net_income_attributable_to_common_shareholders', - label: 'Net Income Attributable to Common Shareholders', - order: 114, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['NetIncomeLossAvailableToCommonStockholdersBasic'], - allowedLabelPhrases: ['net income attributable to common shareholders', 'net income available to common stockholders'] - } - }), - income({ - key: 'depreciation_and_amortization', - label: 'Depreciation & Amortization', - order: 200, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: [ - 'DepreciationDepletionAndAmortization', - 'DepreciationAmortizationAndAccretionNet', - 'DepreciationAndAmortization', - 'DepreciationAmortizationAndOther', - 'CostOfGoodsAndServicesSoldDepreciationAndAmortization' - ], - secondaryLocalNames: ['AmortizationOfIntangibleAssets'], - allowedLabelPhrases: ['depreciation amortization', 'depreciation and amortization'] - } - }), - income({ - key: 'depreciation_and_amortization_expenses', - label: 'Depreciation & Amortization Expense', - order: 205, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['CostOfGoodsAndServicesSoldDepreciationAndAmortization'], - allowedLabelPhrases: ['depreciation amortization expense', 'depreciation and amortization expense'] - } - }), - income({ - key: 'interest_income', - label: 'Interest Income', - order: 210, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['InterestIncomeOther', 'InvestmentIncomeInterest'], - secondaryLocalNames: ['InvestmentIncomeNet'], - allowedLabelPhrases: ['interest income'], - excludeLabelPhrases: ['effective income tax rate reconciliation', 'reconciliation'], - excludeLocalNames: ['InterestIncomeExpenseNet'] - } - }), - income({ - key: 'interest_expense', - label: 'Interest Expense', - order: 220, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['InterestIncomeExpenseNonoperatingNet', 'InterestExpense', 'InterestAndDebtExpense'], - allowedLabelPhrases: ['interest expense'] - }, - signTransform: 'absolute' - }), - income({ - key: 'other_non_operating_income', - label: 'Other Non-Operating Income', - order: 230, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['OtherNonoperatingIncomeExpense', 'NonoperatingIncomeExpense'], - allowedLabelPhrases: ['other non operating', 'other non-operating'] - } - }), - income({ - key: 'stock_based_compensation', - label: 'Share-Based Compensation Expense', - order: 240, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['ShareBasedCompensation', 'AllocatedShareBasedCompensationExpense'], - allowedLabelPhrases: ['share based compensation', 'stock based compensation'] - } - }), - income({ - key: 'ebitda', - label: 'EBITDA', - order: 250, - unit: 'currency', - selectionPolicy: 'formula_only', - matchers: {}, - fallbackFormula: { - kind: 'sum', - sourceKeys: ['operating_income', 'depreciation_and_amortization'] - } - }), - income({ - key: 'effective_tax_rate', - label: 'Effective Tax Rate', - order: 260, - unit: 'percent', - selectionPolicy: 'direct_or_formula_fallback', - matchers: { - exactLocalNames: ['EffectiveIncomeTaxRateContinuingOperations'], - allowedLabelPhrases: ['effective tax rate'] - }, - fallbackFormula: { - kind: 'divide', - numerator: 'income_tax_expense', - denominator: 'pretax_income' - } - }), - income({ - key: 'diluted_eps', - label: 'Diluted EPS', - order: 300, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['EarningsPerShareDiluted', 'DilutedEarningsPerShare'], - allowedLabelPhrases: ['diluted eps', 'diluted earnings per share'] - } - }), - income({ - key: 'basic_eps', - label: 'Basic EPS', - order: 310, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['EarningsPerShareBasic', 'BasicEarningsPerShare'], - allowedLabelPhrases: ['basic eps', 'basic earnings per share'] - } - }), - income({ - key: 'diluted_shares', - label: 'Diluted Shares', - order: 320, - unit: 'shares', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: [ - 'WeightedAverageNumberOfDilutedSharesOutstanding', - 'WeightedAverageNumberOfShareOutstandingDiluted' - ], - allowedLabelPhrases: ['diluted shares', 'weighted average diluted'] - } - }), - income({ - key: 'basic_shares', - label: 'Basic Shares', - order: 330, - unit: 'shares', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: [ - 'WeightedAverageNumberOfSharesOutstandingBasic', - 'WeightedAverageNumberOfShareOutstandingBasicAndDiluted' - ], - allowedLabelPhrases: ['basic shares', 'weighted average basic'] - } - }) - ], - balance: [ - balance({ - key: 'cash_and_equivalents', - label: 'Cash and Cash Equivalents', - category: 'assets', - order: 10, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['CashAndCashEquivalentsAtCarryingValue'], - secondaryLocalNames: ['Cash', 'CashCashEquivalentsAndFederalFundsSold'], - allowedLabelPhrases: ['cash and cash equivalents'] - } - }), - balance({ - key: 'short_term_investments', - label: 'Short-Term Investments', - category: 'assets', - order: 20, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['AvailableForSaleSecuritiesCurrent', 'ShortTermInvestments', 'MarketableSecuritiesCurrent'], - allowedLabelPhrases: ['short term investments', 'marketable securities'] - } - }), - balance({ - key: 'total_cash_and_equivalents', - label: 'Total Cash and Cash Equivalents', - category: 'assets', - order: 30, - unit: 'currency', - selectionPolicy: 'aggregate_multiple_components', - matchers: { - exactLocalNames: ['CashCashEquivalentsAndShortTermInvestments', 'CashAndShortTermInvestments'], - allowedLabelPhrases: ['total cash and cash equivalents'] - }, - fallbackFormula: { - kind: 'sum', - sourceKeys: ['cash_and_equivalents', 'short_term_investments'], - treatNullAsZero: true - } - }), - balance({ - key: 'accounts_receivable', - label: 'Accounts Receivable', - category: 'assets', - order: 40, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['AccountsReceivableNetCurrent', 'ReceivablesNetCurrent'], - allowedLabelPhrases: ['accounts receivable'] - } - }), - balance({ - key: 'inventory', - label: 'Inventories', - category: 'assets', - order: 50, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['InventoryNet'], - allowedLabelPhrases: ['inventories', 'inventory'] - } - }), - balance({ - key: 'other_current_assets', - label: 'Other Current Assets', - category: 'assets', - order: 60, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['OtherAssetsCurrent'], - allowedLabelPhrases: ['other current assets'] - } - }), - balance({ - key: 'deferred_income_taxes_asset', - label: 'Deferred Income Taxes', - category: 'assets', - order: 70, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['DeferredIncomeTaxAssetsNet', 'DeferredTaxAssetsNet'], - allowedLabelPhrases: ['deferred income taxes', 'deferred tax assets'], - excludeLocalNames: ['DeferredIncomeTaxLiabilities', 'DeferredIncomeTaxLiabilitiesNet'] - } - }), - balance({ - key: 'current_assets', - label: 'Total Current Assets', - category: 'assets', - order: 80, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['AssetsCurrent'], - allowedLabelPhrases: ['total current assets', 'current assets'] - } - }), - balance({ - key: 'property_plant_equipment', - label: 'Net Property, Plant & Equipment', - category: 'assets', - order: 90, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['PropertyPlantAndEquipmentNet'], - secondaryLocalNames: ['PropertyPlantAndEquipmentAndFinanceLeaseRightOfUseAssetAfterAccumulatedDepreciationAndAmortization'], - allowedLabelPhrases: ['property plant equipment', 'property and equipment net'] - } - }), - balance({ - key: 'operating_lease_right_of_use_assets', - label: 'Operating Lease Right-of-Use Assets', - category: 'assets', - order: 100, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['OperatingLeaseRightOfUseAsset'], - allowedLabelPhrases: ['operating lease right of use assets', 'operating lease right of use asset'] - } - }), - balance({ - key: 'intangible_assets', - label: 'Net Intangible Assets', - category: 'assets', - order: 110, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: [ - 'FiniteLivedIntangibleAssetsNet', - 'IntangibleAssetsNetExcludingGoodwill', - 'FiniteLivedIntangibleAssetsNetExcludingGoodwill' - ], - allowedLabelPhrases: ['net intangible assets', 'intangible assets net'] - } - }), - balance({ - key: 'goodwill', - label: 'Goodwill', - category: 'assets', - order: 120, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['Goodwill'], - allowedLabelPhrases: ['goodwill'], - excludeLabelPhrases: ['excluding goodwill'] - } - }), - balance({ - key: 'long_term_investments', - label: 'Long-Term Investments', - category: 'assets', - order: 130, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: [ - 'AvailableForSaleSecuritiesDebtMaturitiesSingleMaturityDate', - 'AvailableForSaleSecuritiesNoncurrent', - 'LongTermInvestments' - ], - secondaryLocalNames: ['AvailableForSaleSecuritiesDebtSecurities'], - allowedLabelPhrases: ['long term investments'] - } - }), - balance({ - key: 'other_long_term_assets', - label: 'Other Long-Term Assets', - category: 'assets', - order: 140, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['OtherAssetsNoncurrent'], - allowedLabelPhrases: ['other long term assets', 'other noncurrent assets'], - excludeLocalNames: ['NoncurrentAssets'] - } - }), - balance({ - key: 'total_assets', - label: 'Total Assets', - category: 'assets', - order: 150, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['Assets'], - allowedLabelPhrases: ['total assets'] - } - }), - balance({ - key: 'accounts_payable', - label: 'Accounts Payable', - category: 'liabilities', - order: 160, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['AccountsPayableCurrent'], - allowedLabelPhrases: ['accounts payable'], - excludeLabelPhrases: ['business combination', 'liabilities assumed'] - } - }), - balance({ - key: 'accrued_liabilities', - label: 'Accrued Expenses', - category: 'liabilities', - order: 170, - unit: 'currency', - selectionPolicy: 'aggregate_multiple_components', - matchers: { - exactLocalNames: [ - 'AccruedLiabilitiesCurrent', - 'OtherAccruedLiabilitiesCurrent', - 'AccruedCompensationCurrent', - 'EmployeeRelatedLiabilitiesCurrent', - 'OtherLiabilitiesCurrent', - 'AccruedPropertyTaxes' - ], - allowedLabelPhrases: ['accrued expenses', 'accrued liabilities', 'accrued compensation', 'accrued property taxes'] - } - }), - balance({ - key: 'short_term_debt', - label: 'Short-Term Debt', - category: 'liabilities', - order: 180, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['ShortTermBorrowings', 'DebtCurrent', 'CommercialPaper'], - allowedLabelPhrases: ['short term debt', 'short-term debt'] - } - }), - balance({ - key: 'current_debt', - label: 'Current Portion of Long-Term Debt', - category: 'liabilities', - order: 190, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['LongTermDebtCurrent', 'CurrentPortionOfLongTermDebt', 'LongTermDebtAndCapitalLeaseObligationsCurrent'], - allowedLabelPhrases: ['current portion of long term debt', 'current portion of long-term debt'] - } - }), - balance({ - key: 'long_term_debt', - label: 'Long-Term Debt', - category: 'liabilities', - order: 200, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['LongTermDebtNoncurrent', 'DebtNoncurrent', 'LongTermDebtAndCapitalLeaseObligations'], - secondaryLocalNames: ['LongTermDebt', 'DebtInstrumentCarryingAmount'], - allowedLabelPhrases: ['long term debt', 'long-term debt'] - } - }), - balance({ - key: 'unearned_revenue', - label: 'Unearned Revenue', - category: 'liabilities', - order: 210, - unit: 'currency', - selectionPolicy: 'direct_or_formula_fallback', - matchers: { - exactLocalNames: ['ContractWithCustomerLiabilityCurrent', 'DeferredRevenueCurrent'], - secondaryLocalNames: ['ContractWithCustomerLiability', 'DeferredRevenue'], - allowedLabelPhrases: ['unearned revenue', 'deferred revenue'] - }, - fallbackFormula: { - kind: 'sum', - sourceKeys: ['deferred_revenue_current', 'deferred_revenue_noncurrent'], - treatNullAsZero: true - } - }), - balance({ - key: 'deferred_revenue_current', - label: 'Deferred Revenue, Current', - includeInOutput: false, - category: 'liabilities', - order: 211, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['ContractWithCustomerLiabilityCurrent', 'DeferredRevenueCurrent'], - allowedLabelPhrases: ['deferred revenue current', 'current deferred revenue'] - } - }), - balance({ - key: 'deferred_revenue_noncurrent', - label: 'Deferred Revenue, Noncurrent', - includeInOutput: false, - category: 'liabilities', - order: 212, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['ContractWithCustomerLiabilityNoncurrent', 'DeferredRevenueNoncurrent'], - allowedLabelPhrases: ['deferred revenue noncurrent', 'noncurrent deferred revenue'] - } - }), - balance({ - key: 'deferred_income_taxes_liability', - label: 'Deferred Income Taxes', - category: 'liabilities', - order: 220, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['DeferredIncomeTaxLiabilitiesNet', 'DeferredIncomeTaxLiabilities'], - allowedLabelPhrases: ['deferred income taxes', 'deferred tax liabilities'], - excludeLocalNames: ['DeferredIncomeTaxAssetsNet', 'DeferredTaxAssetsNet'] - } - }), - balance({ - key: 'other_long_term_liabilities', - label: 'Other Long-Term Liabilities', - category: 'liabilities', - order: 230, - unit: 'currency', - selectionPolicy: 'aggregate_multiple_components', - matchers: { - exactLocalNames: [ - 'DeferredIncomeTaxLiabilitiesNet', - 'DeferredIncomeTaxLiabilities', - 'AssetRetirementObligationsNoncurrent', - 'OtherLiabilitiesNoncurrent' - ], - allowedLabelPhrases: ['other long term liabilities', 'other noncurrent liabilities'], - excludeLocalNames: [ - 'DeferredIncomeTaxLiabilitiesNet', - 'DeferredIncomeTaxLiabilities', - 'OperatingLeaseLiability', - 'OperatingLeaseLiabilityCurrent', - 'OperatingLeaseLiabilityNoncurrent', - 'FinanceLeaseLiability', - 'FinanceLeaseLiabilityCurrent', - 'FinanceLeaseLiabilityNoncurrent', - 'LesseeOperatingLeaseLiability' - ] - } - }), - balance({ - key: 'current_liabilities', - label: 'Current Liabilities', - category: 'liabilities', - order: 240, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['LiabilitiesCurrent'], - allowedLabelPhrases: ['current liabilities'] - } - }), - balance({ - key: 'total_current_liabilities', - label: 'Total Current Liabilities', - category: 'liabilities', - order: 241, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['LiabilitiesCurrent'], - allowedLabelPhrases: ['total current liabilities'] - } - }), - balance({ - key: 'lease_liabilities', - label: 'Lease Liabilities', - category: 'liabilities', - order: 250, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: [ - 'OperatingLeaseLiabilityNoncurrent', - 'OperatingLeaseLiability', - 'FinanceLeaseLiability', - 'FinanceLeaseLiabilityNoncurrent', - 'LesseeOperatingLeaseLiability' - ], - allowedLabelPhrases: ['lease liabilities'] - } - }), - balance({ - key: 'leases', - label: 'Leases', - category: 'liabilities', - order: 255, - unit: 'currency', - selectionPolicy: 'aggregate_multiple_components', - matchers: { - exactLocalNames: [ - 'OperatingLeaseLiability', - 'OperatingLeaseLiabilityCurrent', - 'OperatingLeaseLiabilityNoncurrent', - 'FinanceLeaseLiability', - 'FinanceLeaseLiabilityCurrent', - 'FinanceLeaseLiabilityNoncurrent', - 'LesseeOperatingLeaseLiability' - ], - allowedLabelPhrases: ['leases'] - } - }), - balance({ - key: 'total_liabilities', - label: 'Total Liabilities', - category: 'liabilities', - order: 260, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['Liabilities'], - allowedLabelPhrases: ['total liabilities'] - } - }), - balance({ - key: 'common_stock', - label: 'Common Stock', - category: 'equity', - order: 270, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['CommonStocksIncludingAdditionalPaidInCapital', 'CommonStockValue'], - secondaryLocalNames: ['AdditionalPaidInCapitalCommonStock', 'AdditionalPaidInCapital'], - allowedLabelPhrases: ['common stock'] - } - }), - balance({ - key: 'accumulated_other_comprehensive_income', - label: 'Accumulated Other Comprehensive Income', - category: 'equity', - order: 280, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['AccumulatedOtherComprehensiveIncomeLossNetOfTax'], - allowedLabelPhrases: ['accumulated other comprehensive income'] - } - }), - balance({ - key: 'retained_earnings', - label: 'Retained Earnings', - category: 'equity', - order: 290, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['RetainedEarningsAccumulatedDeficit'], - allowedLabelPhrases: ['retained earnings', 'accumulated deficit'] - } - }), - balance({ - key: 'total_equity', - label: "Total Shareholders' Equity", - category: 'equity', - order: 300, - unit: 'currency', - selectionPolicy: 'direct_or_formula_fallback', - matchers: { - exactLocalNames: ['StockholdersEquity', 'StockholdersEquityIncludingPortionAttributableToNoncontrollingInterest', 'PartnersCapital'], - allowedLabelPhrases: ['total shareholders equity', 'total stockholders equity', 'total equity'] - }, - fallbackFormula: { - kind: 'subtract', - left: 'total_assets', - right: 'total_liabilities' - } - }), - balance({ - key: 'total_common_shareholders_equity', - label: "Total Common Shareholders' Equity", - category: 'equity', - order: 301, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['StockholdersEquity'], - allowedLabelPhrases: ['total common shareholders equity', 'total stockholders equity'] - } - }), - balance({ - key: 'total_liabilities_and_equity', - label: "Total Liabilities and Shareholders' Equity", - category: 'equity', - order: 310, - unit: 'currency', - selectionPolicy: 'direct_or_formula_fallback', - matchers: { - exactLocalNames: ['LiabilitiesAndStockholdersEquity'], - allowedLabelPhrases: ['total liabilities and shareholders equity', 'total liabilities and stockholders equity'] - }, - fallbackFormula: { - kind: 'sum', - sourceKeys: ['total_liabilities', 'total_equity'] - } - }), - balance({ - key: 'total_debt', - label: 'Total Debt', - category: 'liabilities', - order: 320, - unit: 'currency', - selectionPolicy: 'formula_only', - matchers: {}, - fallbackFormula: { - kind: 'sum', - sourceKeys: ['short_term_debt', 'current_debt', 'long_term_debt', 'lease_liabilities'], - treatNullAsZero: true - } - }), - balance({ - key: 'net_cash_position', - label: 'Net Cash Position', - category: 'assets', - order: 330, - unit: 'currency', - selectionPolicy: 'formula_only', - matchers: {}, - fallbackFormula: { - kind: 'subtract', - left: 'total_cash_and_equivalents', - right: 'total_debt' - } - }) - ], - cash_flow: [ - cashFlow({ - key: 'net_income', - label: 'Net Income', - category: 'operating', - order: 10, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['NetIncomeLossAvailableToCommonStockholdersBasic', 'NetIncomeLoss', 'ProfitLoss'], - allowedLabelPhrases: ['net income'] - } - }), - cashFlow({ - key: 'depreciation_and_amortization', - label: 'Depreciation & Amortization', - category: 'operating', - order: 20, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: [ - 'DepreciationDepletionAndAmortization', - 'DepreciationAmortizationAndAccretionNet', - 'DepreciationAndAmortization', - 'DepreciationAmortizationAndOther' - ], - secondaryLocalNames: ['AmortizationOfIntangibleAssets'], - allowedLabelPhrases: ['depreciation amortization', 'depreciation and amortization'] - } - }), - cashFlow({ - key: 'stock_based_compensation', - label: 'Share-Based Compensation Expense', - category: 'operating', - order: 30, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['ShareBasedCompensation', 'AllocatedShareBasedCompensationExpense'], - allowedLabelPhrases: ['share based compensation', 'stock based compensation'] - } - }), - cashFlow({ - key: 'other_adjustments', - label: 'Other Adjustments', - category: 'operating', - order: 40, - unit: 'currency', - selectionPolicy: 'aggregate_multiple_components', - matchers: { - exactLocalNames: [ - 'OtherAdjustmentsToReconcileNetIncomeLossToCashProvidedByUsedInOperatingActivities', - 'IncreaseDecreaseInDeferredIncomeTaxes', - 'OtherNoncashIncomeExpense' - ], - allowedLabelPhrases: ['other adjustments'] - }, - signTransform: 'invert' - }), - cashFlow({ - key: 'changes_trade_receivables', - label: 'Changes in Trade Receivables', - category: 'operating', - order: 50, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['IncreaseDecreaseInAccountsReceivable', 'IncreaseDecreaseInReceivables'], - allowedLabelPhrases: ['changes in trade receivables', 'accounts receivable'] - }, - signTransform: 'invert' - }), - cashFlow({ - key: 'changes_inventories', - label: 'Changes in Inventories', - category: 'operating', - order: 60, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['IncreaseDecreaseInInventories'], - allowedLabelPhrases: ['changes in inventories', 'inventories'] - }, - signTransform: 'invert' - }), - cashFlow({ - key: 'changes_accounts_payable', - label: 'Changes in Accounts Payable', - category: 'operating', - order: 70, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['IncreaseDecreaseInAccountsPayable'], - allowedLabelPhrases: ['changes in accounts payable', 'accounts payable'] - } - }), - cashFlow({ - key: 'changes_accrued_expenses', - label: 'Changes in Accrued Expenses', - category: 'operating', - order: 75, - unit: 'currency', - selectionPolicy: 'aggregate_multiple_components', - matchers: { - exactLocalNames: [ - 'IncreaseDecreaseInAccruedLiabilities', - 'IncreaseDecreaseInEmployeeRelatedLiabilitiesCurrent', - 'IncreaseDecreaseInOtherLiabilitiesCurrent' - ], - allowedLabelPhrases: ['changes in accrued expenses', 'increase decrease in accrued liabilities'] - } - }), - cashFlow({ - key: 'changes_income_taxes_payable', - label: 'Changes in Income Taxes Payable', - category: 'operating', - order: 80, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['IncreaseDecreaseInAccruedIncomeTaxesPayable', 'IncreaseDecreaseInIncomeTaxes'], - allowedLabelPhrases: ['changes in income taxes payable', 'income taxes'], - excludeLocalNames: ['IncreaseDecreaseInIncomeTaxesReceivable'] - } - }), - cashFlow({ - key: 'changes_unearned_revenue', - label: 'Changes in Unearned Revenue', - category: 'operating', - order: 90, - unit: 'currency', - selectionPolicy: 'direct_or_formula_fallback', - matchers: { - exactLocalNames: ['IncreaseDecreaseInDeferredRevenue'], - allowedLabelPhrases: ['changes in unearned revenue', 'deferred revenue'] - }, - fallbackFormula: { - kind: 'subtract', - left: 'contract_liability_incurred', - right: 'contract_liability_recognized' - } - }), - cashFlow({ - key: 'contract_liability_incurred', - label: 'Unearned Revenue Incurred', - includeInOutput: false, - category: 'operating', - order: 91, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['ContractWithCustomerLiabilityIncurred'], - allowedLabelPhrases: ['deferral of unearned revenue'] - } - }), - cashFlow({ - key: 'contract_liability_recognized', - label: 'Unearned Revenue Recognized', - includeInOutput: false, - category: 'operating', - order: 92, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['ContractWithCustomerLiabilityRevenueRecognized'], - allowedLabelPhrases: ['recognition of unearned revenue'] - } - }), - cashFlow({ - key: 'changes_other_operating_activities', - label: 'Changes in Other Operating Activities', - category: 'operating', - order: 100, - unit: 'currency', - selectionPolicy: 'aggregate_multiple_components', - matchers: { - exactLocalNames: [ - 'IncreaseDecreaseInOtherOperatingAssets', - 'IncreaseDecreaseInOtherOperatingLiabilities', - 'IncreaseDecreaseInDeferredIncomeTaxes', - 'IncreaseDecreaseInPrepaidExpense' - ], - allowedLabelPhrases: ['changes in other operating activities'] - }, - fallbackFormula: { - kind: 'sum', - sourceKeys: [ - 'changes_other_current_assets', - 'changes_other_current_liabilities', - 'changes_other_noncurrent_assets', - 'changes_other_noncurrent_liabilities' - ], - treatNullAsZero: true - }, - signTransform: 'invert' - }), - cashFlow({ - key: 'changes_other_current_assets', - label: 'Other Current Assets', - includeInOutput: false, - category: 'operating', - order: 101, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['IncreaseDecreaseInOtherCurrentAssets'] - } - }), - cashFlow({ - key: 'changes_other_current_liabilities', - label: 'Other Current Liabilities', - includeInOutput: false, - category: 'operating', - order: 102, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['IncreaseDecreaseInOtherCurrentLiabilities'] - } - }), - cashFlow({ - key: 'changes_other_noncurrent_assets', - label: 'Other Long-Term Assets', - includeInOutput: false, - category: 'operating', - order: 103, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['IncreaseDecreaseInOtherNoncurrentAssets'] - } - }), - cashFlow({ - key: 'changes_other_noncurrent_liabilities', - label: 'Other Long-Term Liabilities', - includeInOutput: false, - category: 'operating', - order: 104, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['IncreaseDecreaseInOtherNoncurrentLiabilities'] - } - }), - cashFlow({ - key: 'operating_cash_flow', - label: 'Cash from Operating Activities', - category: 'operating', - order: 110, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: [ - 'NetCashProvidedByUsedInOperatingActivities', - 'NetCashProvidedByUsedInOperatingActivitiesContinuingOperations' - ], - allowedLabelPhrases: ['cash from operating activities', 'net cash from operations'] - } - }), - cashFlow({ - key: 'capital_expenditures', - label: 'Capital Expenditures', - category: 'investing', - order: 120, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['PaymentsToAcquirePropertyPlantAndEquipment'], - secondaryLocalNames: ['CapitalExpendituresIncurredButNotYetPaid'], - allowedLabelPhrases: ['capital expenditures', 'capital expenditure'] - }, - signTransform: 'invert' - }), - cashFlow({ - key: 'acquisitions', - label: 'Acquisitions', - category: 'investing', - order: 130, - unit: 'currency', - selectionPolicy: 'prefer_primary_statement_concept', - matchers: { - exactLocalNames: [ - 'PaymentsToAcquireBusinessesNetOfCashAcquired', - 'AcquisitionsNetOfCashAcquiredAndPurchasesOfIntangibleAndOtherAssets' - ], - allowedLabelPhrases: ['acquisitions'] - }, - signTransform: 'invert' - }), - cashFlow({ - key: 'investments', - label: 'Investments', - category: 'investing', - order: 140, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['PaymentsForProceedsFromOtherInvestingActivities'], - allowedLabelPhrases: ['investments'] - } - }), - cashFlow({ - key: 'proceeds_from_sale_of_property_plant_and_equipment', - label: 'Proceeds from Sale of Property, Plant & Equipment', - category: 'investing', - order: 145, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['ProceedsFromSaleOfPropertyPlantAndEquipment'], - allowedLabelPhrases: ['proceeds from sale of property plant equipment'] - } - }), - cashFlow({ - key: 'other_investing_activities', - label: 'Other Investing Activities', - category: 'investing', - order: 150, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['OtherInvestingActivitiesNet', 'OtherCashFlowFromInvestingActivities'], - allowedLabelPhrases: ['other investing activities'] - }, - signTransform: 'invert' - }), - cashFlow({ - key: 'investing_cash_flow', - label: 'Cash from Investing Activities', - category: 'investing', - order: 160, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['NetCashProvidedByUsedInInvestingActivities'], - allowedLabelPhrases: ['cash from investing activities'] - } - }), - cashFlow({ - key: 'short_term_debt_issued', - label: 'Short-Term Debt Issued', - category: 'financing', - order: 170, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['ProceedsFromShortTermDebt'], - allowedLabelPhrases: ['short term debt issued'] - } - }), - cashFlow({ - key: 'long_term_debt_issued', - label: 'Long-Term Debt Issued', - category: 'financing', - order: 180, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['ProceedsFromIssuanceOfLongTermDebt'], - allowedLabelPhrases: ['long term debt issued'] - } - }), - cashFlow({ - key: 'debt_repaid', - label: 'Debt Repaid', - category: 'financing', - order: 190, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['RepaymentsOfDebt', 'RepaymentsOfLongTermDebt'], - allowedLabelPhrases: ['debt repaid'] - }, - signTransform: 'invert' - }), - cashFlow({ - key: 'share_repurchases', - label: 'Repurchases of Common Shares', - category: 'financing', - order: 200, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['PaymentsForRepurchaseOfCommonStock', 'PaymentsForRepurchaseOfEquity'], - allowedLabelPhrases: ['repurchases of common shares', 'share repurchases'] - }, - signTransform: 'invert' - }), - cashFlow({ - key: 'dividends_paid', - label: 'Common Share Dividends Paid', - category: 'financing', - order: 210, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['DividendsCommonStockCash', 'PaymentsOfDividendsCommonStock', 'PaymentsOfDividends'], - allowedLabelPhrases: ['common share dividends paid', 'dividends paid'] - }, - signTransform: 'invert' - }), - cashFlow({ - key: 'other_financing_activities', - label: 'Other Financing Activities', - category: 'financing', - order: 220, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['ProceedsFromPaymentsForOtherFinancingActivities'], - allowedLabelPhrases: ['other financing activities'] - } - }), - cashFlow({ - key: 'financing_cash_flow', - label: 'Cash from Financing Activities', - category: 'financing', - order: 230, - unit: 'currency', - selectionPolicy: 'prefer_exact_local_name', - matchers: { - exactLocalNames: ['NetCashProvidedByUsedInFinancingActivities'], - allowedLabelPhrases: ['cash from financing activities'] - } - }), - cashFlow({ - key: 'free_cash_flow', - label: 'Free Cash Flow', - category: 'free_cash_flow', - order: 240, - unit: 'currency', - selectionPolicy: 'formula_only', - matchers: {}, - fallbackFormula: { - kind: 'sum', - sourceKeys: ['operating_cash_flow', 'capital_expenditures'] - } - }) - ] -}; diff --git a/lib/server/financials/standardize.ts b/lib/server/financials/standardize.ts index c9ba849..d9e1340 100644 --- a/lib/server/financials/standardize.ts +++ b/lib/server/financials/standardize.ts @@ -1,31 +1,11 @@ import type { - DerivedFinancialRow, DimensionBreakdownRow, FinancialStatementKind, FinancialStatementPeriod, - FinancialUnit, StandardizedFinancialRow, TaxonomyFactRow, TaxonomyStatementRow } from '@/lib/types'; -import { - STANDARD_FINANCIAL_TEMPLATES, - type StandardTemplateRowDefinition, - type TemplateFormula -} from '@/lib/server/financials/standard-template'; - -function normalizeToken(value: string) { - return value.trim().toLowerCase(); -} - -function tokenizeLabel(value: string) { - return value - .toLowerCase() - .replace(/[^a-z0-9]+/g, ' ') - .trim() - .split(/\s+/) - .filter((token) => token.length > 0); -} function valueOrNull(values: Record, periodId: string) { return periodId in values ? values[periodId] : null; @@ -39,326 +19,6 @@ function sumValues(values: Array, treatNullAsZero = false) { return values.reduce((sum, value) => sum + (value ?? 0), 0); } -function subtractValues(left: number | null, right: number | null) { - if (left === null || right === null) { - return null; - } - - return left - right; -} - -function divideValues(left: number | null, right: number | null) { - if (left === null || right === null || right === 0) { - return null; - } - - return left / right; -} - -type CandidateMatchKind = 'exact_local_name' | 'secondary_local_name' | 'label_phrase'; - -type StatementRowCandidate = { - row: TaxonomyStatementRow; - matchKind: CandidateMatchKind; - aliasRank: number; - unit: FinancialUnit; - labelTokenCount: number; - matchedPhraseTokenCount: number; -}; - -type FactCandidate = { - fact: TaxonomyFactRow; - matchKind: Exclude; - aliasRank: number; - unit: FinancialUnit; -}; - -type ResolvedCandidate = - | { - sourceType: 'row'; - matchKind: CandidateMatchKind; - aliasRank: number; - unit: FinancialUnit; - labelTokenCount: number; - matchedPhraseTokenCount: number; - row: TaxonomyStatementRow; - } - | { - sourceType: 'fact'; - matchKind: Exclude; - aliasRank: number; - unit: FinancialUnit; - fact: TaxonomyFactRow; - }; - -type DerivedRole = 'expense' | 'addback'; - -type InternalRowMetadata = { - derivedRoleByPeriod: Record; -}; - -function resolvedCandidatesForPeriod(input: { - definition: StandardTemplateRowDefinition; - candidates: StatementRowCandidate[]; - factCandidates: FactCandidate[]; - period: FinancialStatementPeriod; -}) { - const rowCandidates = input.candidates - .filter((candidate) => input.period.id in candidate.row.values && candidate.row.values[input.period.id] !== null) - .map((candidate) => ({ - sourceType: 'row' as const, - ...candidate - })); - const factCandidates = input.factCandidates - .filter((candidate) => factMatchesPeriod(candidate.fact, input.period)) - .map((candidate) => ({ - sourceType: 'fact' as const, - ...candidate - })); - - if (input.definition.selectionPolicy === 'aggregate_multiple_components') { - const aggregateCandidates = [...rowCandidates, ...factCandidates] - .sort((left, right) => compareResolvedCandidates(left, right, input.definition)); - const dedupedCandidates: ResolvedCandidate[] = []; - const seenConcepts = new Set(); - - for (const candidate of aggregateCandidates) { - const conceptKey = candidate.sourceType === 'row' - ? candidate.row.key - : candidate.fact.conceptKey; - if (seenConcepts.has(conceptKey)) { - continue; - } - - seenConcepts.add(conceptKey); - dedupedCandidates.push(candidate); - } - - return dedupedCandidates; - } - - const resolvedCandidate = [...rowCandidates, ...factCandidates] - .sort((left, right) => compareResolvedCandidates(left, right, input.definition))[0]; - - return resolvedCandidate ? [resolvedCandidate] : []; -} - -const GLOBAL_EXCLUDE_LABEL_PHRASES = [ - 'pro forma', - 'reconciliation', - 'acquiree', - 'business combination', - 'assets acquired', - 'liabilities assumed' -] as const; - -function inferUnit(rawUnit: string | null, fallback: FinancialUnit) { - const normalized = (rawUnit ?? '').toLowerCase(); - if (!normalized) { - return fallback; - } - - if (normalized.includes('usd') || normalized.includes('iso4217')) { - return 'currency'; - } - - if (normalized.includes('shares')) { - return 'shares'; - } - - if (normalized.includes('pure') || normalized.includes('percent')) { - return fallback === 'percent' ? 'percent' : 'ratio'; - } - - return fallback; -} - -function rowUnit(row: TaxonomyStatementRow, fallback: FinancialUnit) { - return inferUnit(Object.values(row.units)[0] ?? null, fallback); -} - -function isUnitCompatible(expected: FinancialUnit, actual: FinancialUnit) { - if (expected === actual) { - return true; - } - - if ((expected === 'percent' || expected === 'ratio') && (actual === 'percent' || actual === 'ratio')) { - return true; - } - - return false; -} - -function phraseTokens(phrase: string) { - return tokenizeLabel(phrase); -} - -function labelContainsPhrase(labelTokens: string[], phrase: string) { - const target = phraseTokens(phrase); - if (target.length === 0 || target.length > labelTokens.length) { - return false; - } - - for (let index = 0; index <= labelTokens.length - target.length; index += 1) { - let matched = true; - for (let offset = 0; offset < target.length; offset += 1) { - if (labelTokens[index + offset] !== target[offset]) { - matched = false; - break; - } - } - - if (matched) { - return true; - } - } - - return false; -} - -function matchRank(matchKind: CandidateMatchKind) { - switch (matchKind) { - case 'exact_local_name': - return 0; - case 'secondary_local_name': - return 1; - case 'label_phrase': - return 2; - } -} - -function aliasRank(localName: string, aliases: readonly string[] | undefined) { - const normalizedLocalName = normalizeToken(localName); - const matchIndex = (aliases ?? []).findIndex((alias) => normalizeToken(alias) === normalizedLocalName); - return matchIndex === -1 ? Number.MAX_SAFE_INTEGER : matchIndex; -} - -function applySignTransform(value: number | null, transform: StandardTemplateRowDefinition['signTransform']) { - if (value === null || !transform) { - return value; - } - - if (transform === 'invert') { - return value * -1; - } - - return Math.abs(value); -} - -function classifyStatementRowCandidate( - row: TaxonomyStatementRow, - definition: StandardTemplateRowDefinition -) { - if (definition.selectionPolicy === 'formula_only') { - return null; - } - - const rowLocalName = normalizeToken(row.localName); - if ((definition.matchers.excludeLocalNames ?? []).some((localName) => normalizeToken(localName) === rowLocalName)) { - return null; - } - - const labelTokens = tokenizeLabel(row.label); - const excludedLabelPhrases = [ - ...GLOBAL_EXCLUDE_LABEL_PHRASES, - ...(definition.matchers.excludeLabelPhrases ?? []) - ]; - if (excludedLabelPhrases.some((phrase) => labelContainsPhrase(labelTokens, phrase))) { - return null; - } - - const unit = rowUnit(row, definition.unit); - if (!isUnitCompatible(definition.unit, unit)) { - return null; - } - - if ((definition.matchers.exactLocalNames ?? []).some((localName) => normalizeToken(localName) === rowLocalName)) { - return { - row, - matchKind: 'exact_local_name', - aliasRank: aliasRank(row.localName, definition.matchers.exactLocalNames), - unit, - labelTokenCount: labelTokens.length, - matchedPhraseTokenCount: 0 - } satisfies StatementRowCandidate; - } - - if ((definition.matchers.secondaryLocalNames ?? []).some((localName) => normalizeToken(localName) === rowLocalName)) { - return { - row, - matchKind: 'secondary_local_name', - aliasRank: aliasRank(row.localName, definition.matchers.secondaryLocalNames), - unit, - labelTokenCount: labelTokens.length, - matchedPhraseTokenCount: 0 - } satisfies StatementRowCandidate; - } - - const matchedPhrase = (definition.matchers.allowedLabelPhrases ?? []) - .map((phrase) => ({ - phrase, - tokenCount: phraseTokens(phrase).length - })) - .filter(({ phrase }) => labelContainsPhrase(labelTokens, phrase)) - .sort((left, right) => right.tokenCount - left.tokenCount)[0]; - - if (!matchedPhrase) { - return null; - } - - if (row.hasDimensions) { - return null; - } - - return { - row, - matchKind: 'label_phrase', - aliasRank: Number.MAX_SAFE_INTEGER, - unit, - labelTokenCount: labelTokens.length, - matchedPhraseTokenCount: matchedPhrase.tokenCount - } satisfies StatementRowCandidate; -} - -function classifyFactCandidate( - fact: TaxonomyFactRow, - definition: StandardTemplateRowDefinition -) { - if (!fact.isDimensionless) { - return null; - } - - const localName = normalizeToken(fact.localName); - if ((definition.matchers.excludeLocalNames ?? []).some((entry) => normalizeToken(entry) === localName)) { - return null; - } - - const unit = inferUnit(fact.unit ?? null, definition.unit); - if (!isUnitCompatible(definition.unit, unit)) { - return null; - } - - if ((definition.matchers.exactLocalNames ?? []).some((entry) => normalizeToken(entry) === localName)) { - return { - fact, - matchKind: 'exact_local_name', - aliasRank: aliasRank(fact.localName, definition.matchers.exactLocalNames), - unit - } satisfies FactCandidate; - } - - if ((definition.matchers.secondaryLocalNames ?? []).some((entry) => normalizeToken(entry) === localName)) { - return { - fact, - matchKind: 'secondary_local_name', - aliasRank: aliasRank(fact.localName, definition.matchers.secondaryLocalNames), - unit - } satisfies FactCandidate; - } - - return null; -} - export function factMatchesPeriod(fact: TaxonomyFactRow, period: FinancialStatementPeriod) { if (period.periodStart) { return fact.periodStart === period.periodStart && fact.periodEnd === period.periodEnd; @@ -367,390 +27,6 @@ export function factMatchesPeriod(fact: TaxonomyFactRow, period: FinancialStatem return (fact.periodInstant ?? fact.periodEnd) === period.periodEnd; } -function compareStatementRowCandidates( - left: StatementRowCandidate, - right: StatementRowCandidate, - definition: StandardTemplateRowDefinition -) { - const matchDelta = matchRank(left.matchKind) - matchRank(right.matchKind); - if (matchDelta !== 0) { - return matchDelta; - } - - if (left.aliasRank !== right.aliasRank) { - return left.aliasRank - right.aliasRank; - } - - if (left.row.hasDimensions !== right.row.hasDimensions) { - return left.row.hasDimensions ? 1 : -1; - } - - if (definition.selectionPolicy === 'prefer_primary_statement_concept' && left.row.isExtension !== right.row.isExtension) { - return left.row.isExtension ? 1 : -1; - } - - if (left.row.order !== right.row.order) { - return left.row.order - right.row.order; - } - - if (left.matchedPhraseTokenCount !== right.matchedPhraseTokenCount) { - return right.matchedPhraseTokenCount - left.matchedPhraseTokenCount; - } - - if (left.labelTokenCount !== right.labelTokenCount) { - return left.labelTokenCount - right.labelTokenCount; - } - - return left.row.label.localeCompare(right.row.label); -} - -function compareFactCandidates(left: FactCandidate, right: FactCandidate) { - const matchDelta = matchRank(left.matchKind) - matchRank(right.matchKind); - if (matchDelta !== 0) { - return matchDelta; - } - - if (left.aliasRank !== right.aliasRank) { - return left.aliasRank - right.aliasRank; - } - - return left.fact.qname.localeCompare(right.fact.qname); -} - -function compareResolvedCandidates( - left: ResolvedCandidate, - right: ResolvedCandidate, - definition: StandardTemplateRowDefinition -) { - const matchDelta = matchRank(left.matchKind) - matchRank(right.matchKind); - if (matchDelta !== 0) { - return matchDelta; - } - - if (left.aliasRank !== right.aliasRank) { - return left.aliasRank - right.aliasRank; - } - - if (left.sourceType === 'row' && right.sourceType === 'row') { - return compareStatementRowCandidates(left, right, definition); - } - - if (left.sourceType === 'fact' && right.sourceType === 'fact') { - return compareFactCandidates(left, right); - } - - if (left.sourceType === 'row' && right.sourceType === 'fact') { - return left.row.hasDimensions ? 1 : -1; - } - - if (left.sourceType === 'fact' && right.sourceType === 'row') { - return right.row.hasDimensions ? -1 : 1; - } - - return 0; -} - -function buildTemplateRow( - definition: StandardTemplateRowDefinition, - candidates: StatementRowCandidate[], - factCandidates: FactCandidate[], - periods: FinancialStatementPeriod[] -) { - const sourceConcepts = new Set(); - const sourceRowKeys = new Set(); - const sourceFactIds = new Set(); - const matchedRowKeys = new Set(); - const values: Record = Object.fromEntries(periods.map((period) => [period.id, null])); - const resolvedSourceRowKeys: Record = Object.fromEntries(periods.map((period) => [period.id, null])); - const metadata: InternalRowMetadata = { - derivedRoleByPeriod: Object.fromEntries(periods.map((period) => [period.id, null])) - }; - let unit = definition.unit; - let hasDimensions = false; - - for (const period of periods) { - const resolvedCandidates = resolvedCandidatesForPeriod({ - definition, - candidates, - factCandidates, - period - }); - - if (resolvedCandidates.length === 0) { - continue; - } - - if (definition.key === 'depreciation_and_amortization') { - metadata.derivedRoleByPeriod[period.id] = resolvedCandidates.some((candidate) => { - const localName = candidate.sourceType === 'row' - ? candidate.row.localName - : candidate.fact.localName; - return normalizeToken(localName) === normalizeToken('CostOfGoodsAndServicesSoldDepreciationAndAmortization'); - }) - ? 'expense' - : 'addback'; - } - - values[period.id] = definition.selectionPolicy === 'aggregate_multiple_components' - ? sumValues(resolvedCandidates.map((candidate) => { - if (candidate.sourceType === 'row') { - return applySignTransform(candidate.row.values[period.id] ?? null, definition.signTransform); - } - - return applySignTransform(candidate.fact.value ?? null, definition.signTransform); - })) - : (() => { - const resolvedCandidate = resolvedCandidates[0]!; - if (resolvedCandidate.sourceType === 'row') { - return applySignTransform(resolvedCandidate.row.values[period.id] ?? null, definition.signTransform); - } - - return applySignTransform(resolvedCandidate.fact.value ?? null, definition.signTransform); - })(); - resolvedSourceRowKeys[period.id] = resolvedCandidates.length === 1 - ? (resolvedCandidates[0]!.sourceType === 'row' - ? resolvedCandidates[0]!.row.key - : resolvedCandidates[0]!.fact.conceptKey ?? null) - : null; - - for (const resolvedCandidate of resolvedCandidates) { - unit = resolvedCandidate.unit; - - if (resolvedCandidate.sourceType === 'row') { - hasDimensions = hasDimensions || resolvedCandidate.row.hasDimensions; - matchedRowKeys.add(resolvedCandidate.row.key); - sourceConcepts.add(resolvedCandidate.row.qname); - sourceRowKeys.add(resolvedCandidate.row.key); - for (const factId of resolvedCandidate.row.sourceFactIds) { - sourceFactIds.add(factId); - } - continue; - } - - sourceConcepts.add(resolvedCandidate.fact.qname); - sourceRowKeys.add(resolvedCandidate.fact.conceptKey); - sourceFactIds.add(resolvedCandidate.fact.id); - } - } - - return { - row: { - key: definition.key, - label: definition.label, - category: definition.category, - templateSection: definition.category, - order: definition.order, - unit, - values, - sourceConcepts: [...sourceConcepts].sort((left, right) => left.localeCompare(right)), - sourceRowKeys: [...sourceRowKeys].sort((left, right) => left.localeCompare(right)), - sourceFactIds: [...sourceFactIds].sort((left, right) => left - right), - formulaKey: null, - hasDimensions, - resolvedSourceRowKeys - } satisfies StandardizedFinancialRow, - matchedRowKeys, - metadata - }; -} - -function computeFormulaValue( - formula: TemplateFormula, - rowsByKey: Map, - periodId: string -) { - switch (formula.kind) { - case 'sum': - return sumValues( - formula.sourceKeys.map((key) => valueOrNull(rowsByKey.get(key)?.values ?? {}, periodId)), - formula.treatNullAsZero ?? false - ); - case 'subtract': - return subtractValues( - valueOrNull(rowsByKey.get(formula.left)?.values ?? {}, periodId), - valueOrNull(rowsByKey.get(formula.right)?.values ?? {}, periodId) - ); - case 'divide': - return divideValues( - valueOrNull(rowsByKey.get(formula.numerator)?.values ?? {}, periodId), - valueOrNull(rowsByKey.get(formula.denominator)?.values ?? {}, periodId) - ); - } -} - -function rowValueForPeriod( - rowsByKey: Map, - key: string, - periodId: string -) { - return valueOrNull(rowsByKey.get(key)?.values ?? {}, periodId); -} - -function computeOperatingIncomeFallbackValue( - rowsByKey: Map, - rowMetadataByKey: Map, - periodId: string -) { - const grossProfit = rowValueForPeriod(rowsByKey, 'gross_profit', periodId); - const sellingGeneralAndAdministrative = rowValueForPeriod(rowsByKey, 'selling_general_and_administrative', periodId); - const researchAndDevelopment = rowValueForPeriod(rowsByKey, 'research_and_development', periodId) ?? 0; - const depreciationAndAmortization = rowValueForPeriod(rowsByKey, 'depreciation_and_amortization', periodId); - const depreciationRole = rowMetadataByKey.get('depreciation_and_amortization')?.derivedRoleByPeriod[periodId] ?? null; - - if ( - depreciationRole === 'expense' - && grossProfit !== null - && sellingGeneralAndAdministrative !== null - && depreciationAndAmortization !== null - ) { - return grossProfit - sellingGeneralAndAdministrative - researchAndDevelopment - depreciationAndAmortization; - } - - const pretaxIncome = rowValueForPeriod(rowsByKey, 'pretax_income', periodId); - if (pretaxIncome === null) { - return null; - } - - const interestExpense = rowValueForPeriod(rowsByKey, 'interest_expense', periodId) ?? 0; - const interestIncome = rowValueForPeriod(rowsByKey, 'interest_income', periodId) ?? 0; - const otherNonOperatingIncome = rowValueForPeriod(rowsByKey, 'other_non_operating_income', periodId) ?? 0; - - return pretaxIncome + interestExpense - interestIncome - otherNonOperatingIncome; -} - -function computeFallbackValueForDefinition( - definition: StandardTemplateRowDefinition, - rowsByKey: Map, - rowMetadataByKey: Map, - periodId: string -) { - if (definition.key === 'operating_income') { - return computeOperatingIncomeFallbackValue(rowsByKey, rowMetadataByKey, periodId); - } - - if (!definition.fallbackFormula) { - return null; - } - - return computeFormulaValue(definition.fallbackFormula, rowsByKey, periodId); -} - -function applyFormulas( - rowsByKey: Map, - rowMetadataByKey: Map, - definitions: StandardTemplateRowDefinition[], - periods: FinancialStatementPeriod[] -) { - for (let pass = 0; pass < definitions.length; pass += 1) { - let changed = false; - - for (const definition of definitions) { - if (!definition.fallbackFormula && definition.key !== 'operating_income') { - continue; - } - - const target = rowsByKey.get(definition.key); - if (!target) { - continue; - } - - let usedFormula = target.formulaKey !== null; - for (const period of periods) { - if (definition.selectionPolicy !== 'formula_only' && target.values[period.id] !== null) { - continue; - } - - const computed = computeFallbackValueForDefinition(definition, rowsByKey, rowMetadataByKey, period.id); - if (computed === null) { - continue; - } - - target.values[period.id] = applySignTransform(computed, definition.signTransform); - target.resolvedSourceRowKeys[period.id] = null; - usedFormula = true; - changed = true; - } - - if (usedFormula) { - target.formulaKey = definition.key; - } - } - - if (!changed) { - break; - } - } -} - -export function buildStandardizedRows(input: { - rows: TaxonomyStatementRow[]; - statement: Extract; - periods: FinancialStatementPeriod[]; - facts: TaxonomyFactRow[]; -}) { - const definitions = STANDARD_FINANCIAL_TEMPLATES[input.statement]; - const rowsByKey = new Map(); - const rowMetadataByKey = new Map(); - const matchedRowKeys = new Set(); - - for (const definition of definitions) { - const candidates = input.rows - .map((row) => classifyStatementRowCandidate(row, definition)) - .filter((candidate): candidate is StatementRowCandidate => candidate !== null); - const factCandidates = input.facts - .map((fact) => classifyFactCandidate(fact, definition)) - .filter((candidate): candidate is FactCandidate => candidate !== null); - const templateRow = buildTemplateRow(definition, candidates, factCandidates, input.periods); - - for (const rowKey of templateRow.matchedRowKeys) { - matchedRowKeys.add(rowKey); - } - - const hasAnyValue = Object.values(templateRow.row.values).some((value) => value !== null); - if (hasAnyValue || definition.fallbackFormula || definition.key === 'operating_income') { - rowsByKey.set(definition.key, templateRow.row); - rowMetadataByKey.set(definition.key, templateRow.metadata); - } - } - - applyFormulas(rowsByKey, rowMetadataByKey, definitions, input.periods); - - const templateRows = definitions - .filter((definition) => definition.includeInOutput !== false) - .map((definition) => rowsByKey.get(definition.key)) - .filter((row): row is StandardizedFinancialRow => row !== undefined); - const coveredTemplateSourceRowKeys = new Set(templateRows.flatMap((row) => row.sourceRowKeys)); - - const unmatchedRows = input.rows - .filter((row) => !matchedRowKeys.has(row.key)) - .filter((row) => !(row.hasDimensions && coveredTemplateSourceRowKeys.has(row.key))) - .map((row) => ({ - key: `other:${row.key}`, - label: row.label, - category: 'other', - templateSection: 'other', - order: 10_000 + row.order, - unit: inferUnit(Object.values(row.units)[0] ?? null, 'currency'), - values: { ...row.values }, - sourceConcepts: [row.qname], - sourceRowKeys: [row.key], - sourceFactIds: [...row.sourceFactIds], - formulaKey: null, - hasDimensions: row.hasDimensions, - resolvedSourceRowKeys: Object.fromEntries( - input.periods.map((period) => [period.id, period.id in row.values ? row.key : null]) - ) - } satisfies StandardizedFinancialRow)); - - return [...templateRows, ...unmatchedRows].sort((left, right) => { - if (left.order !== right.order) { - return left.order - right.order; - } - - return left.label.localeCompare(right.label); - }); -} - export function buildDimensionBreakdown( facts: TaxonomyFactRow[], periods: FinancialStatementPeriod[], diff --git a/lib/server/financials/surface.ts b/lib/server/financials/surface.ts deleted file mode 100644 index 49e5171..0000000 --- a/lib/server/financials/surface.ts +++ /dev/null @@ -1,320 +0,0 @@ -import type { - DetailFinancialRow, - FinancialStatementKind, - FinancialStatementPeriod, - NormalizationSummary, - StructuredKpiRow, - SurfaceDetailMap, - SurfaceFinancialRow, - TaxonomyFactRow, - TaxonomyStatementRow -} from '@/lib/types'; -import { buildStandardizedRows } from '@/lib/server/financials/standardize'; - -type CompactStatement = Extract; - -type SurfaceDefinition = { - key: string; - label: string; - category: string; - order: number; - unit: SurfaceFinancialRow['unit']; - rowKey?: string; - componentKeys?: string[]; - formula?: { - kind: 'subtract'; - left: string; - right: string; - }; -}; - -const EMPTY_SURFACE_ROWS: Record = { - income: [], - balance: [], - cash_flow: [], - equity: [], - comprehensive_income: [] -}; - -const EMPTY_DETAIL_ROWS: Record = { - income: {}, - balance: {}, - cash_flow: {}, - equity: {}, - comprehensive_income: {} -}; - -const SURFACE_DEFINITIONS: Record = { - income: [ - { key: 'revenue', label: 'Revenue', category: 'surface', order: 10, unit: 'currency', rowKey: 'revenue' }, - { key: 'cost_of_revenue', label: 'Cost of Revenue', category: 'surface', order: 20, unit: 'currency', rowKey: 'cost_of_revenue' }, - { key: 'gross_profit', label: 'Gross Profit', category: 'surface', order: 30, unit: 'currency', rowKey: 'gross_profit' }, - { - key: 'operating_expenses', - label: 'Operating Expenses', - category: 'surface', - order: 40, - unit: 'currency', - componentKeys: ['selling_general_and_administrative', 'research_and_development', 'depreciation_and_amortization'] - }, - { key: 'operating_income', label: 'Operating Income', category: 'surface', order: 50, unit: 'currency', rowKey: 'operating_income' }, - { - key: 'interest_and_other', - label: 'Interest and Other', - category: 'surface', - order: 60, - unit: 'currency', - formula: { - kind: 'subtract', - left: 'pretax_income', - right: 'operating_income' - } - }, - { key: 'pretax_income', label: 'Pretax Income', category: 'surface', order: 70, unit: 'currency', rowKey: 'pretax_income' }, - { key: 'income_taxes', label: 'Income Taxes', category: 'surface', order: 80, unit: 'currency', rowKey: 'income_tax_expense' }, - { key: 'net_income', label: 'Net Income', category: 'surface', order: 90, unit: 'currency', rowKey: 'net_income' } - ], - balance: [ - { key: 'cash_and_equivalents', label: 'Cash and Equivalents', category: 'surface', order: 10, unit: 'currency', rowKey: 'cash_and_equivalents' }, - { key: 'receivables', label: 'Receivables', category: 'surface', order: 20, unit: 'currency', rowKey: 'accounts_receivable' }, - { key: 'inventory', label: 'Inventory', category: 'surface', order: 30, unit: 'currency', rowKey: 'inventory' }, - { key: 'current_assets', label: 'Current Assets', category: 'surface', order: 40, unit: 'currency', rowKey: 'current_assets' }, - { key: 'ppe', label: 'Property, Plant & Equipment', category: 'surface', order: 50, unit: 'currency', rowKey: 'property_plant_equipment' }, - { - key: 'goodwill_and_intangibles', - label: 'Goodwill and Intangibles', - category: 'surface', - order: 60, - unit: 'currency', - componentKeys: ['goodwill', 'intangible_assets'] - }, - { key: 'total_assets', label: 'Total Assets', category: 'surface', order: 70, unit: 'currency', rowKey: 'total_assets' }, - { key: 'current_liabilities', label: 'Current Liabilities', category: 'surface', order: 80, unit: 'currency', rowKey: 'current_liabilities' }, - { key: 'debt', label: 'Debt', category: 'surface', order: 90, unit: 'currency', rowKey: 'total_debt' }, - { key: 'total_liabilities', label: 'Total Liabilities', category: 'surface', order: 100, unit: 'currency', rowKey: 'total_liabilities' }, - { key: 'shareholders_equity', label: 'Shareholders Equity', category: 'surface', order: 110, unit: 'currency', rowKey: 'total_equity' } - ], - cash_flow: [ - { key: 'operating_cash_flow', label: 'Operating Cash Flow', category: 'surface', order: 10, unit: 'currency', rowKey: 'operating_cash_flow' }, - { key: 'capital_expenditures', label: 'Capital Expenditures', category: 'surface', order: 20, unit: 'currency', rowKey: 'capital_expenditures' }, - { key: 'acquisitions', label: 'Acquisitions', category: 'surface', order: 30, unit: 'currency', rowKey: 'acquisitions' }, - { key: 'investing_cash_flow', label: 'Investing Cash Flow', category: 'surface', order: 40, unit: 'currency', rowKey: 'investing_cash_flow' }, - { key: 'financing_cash_flow', label: 'Financing Cash Flow', category: 'surface', order: 50, unit: 'currency', rowKey: 'financing_cash_flow' }, - { key: 'free_cash_flow', label: 'Free Cash Flow', category: 'surface', order: 60, unit: 'currency', rowKey: 'free_cash_flow' } - ] -}; - -function rowHasAnyValue(row: { values: Record }) { - return Object.values(row.values).some((value) => value !== null); -} - -function sumValues(values: Array) { - if (values.every((value) => value === null)) { - return null; - } - - return values.reduce((sum, value) => sum + (value ?? 0), 0); -} - -function valueForPeriod( - rowByKey: Map, - rowKey: string, - periodId: string -) { - return rowByKey.get(rowKey)?.values[periodId] ?? null; -} - -function maxAbsValue(values: Record) { - return Object.values(values).reduce((max, value) => Math.max(max, Math.abs(value ?? 0)), 0); -} - -function detailUnit(row: SurfaceFinancialRow, faithfulRow: TaxonomyStatementRow | undefined) { - if (faithfulRow) { - return Object.values(faithfulRow.units)[0] ?? null; - } - - switch (row.unit) { - case 'currency': - return 'USD'; - case 'shares': - return 'shares'; - case 'percent': - return 'pure'; - default: - return null; - } -} - -function buildDetailRow(input: { - row: SurfaceFinancialRow; - parentSurfaceKey: string; - faithfulRowByKey: Map; -}): DetailFinancialRow { - const sourceRowKey = input.row.sourceRowKeys.find((key) => input.faithfulRowByKey.has(key)) ?? input.row.sourceRowKeys[0] ?? input.row.key; - const faithfulRow = sourceRowKey ? input.faithfulRowByKey.get(sourceRowKey) : undefined; - const qname = faithfulRow?.qname ?? input.row.sourceConcepts[0] ?? input.row.key; - const [prefix, ...rest] = qname.split(':'); - const localName = faithfulRow?.localName ?? (rest.length > 0 ? rest.join(':') : qname); - - return { - key: input.row.key, - parentSurfaceKey: input.parentSurfaceKey, - label: input.row.label, - conceptKey: faithfulRow?.conceptKey ?? sourceRowKey, - qname, - namespaceUri: faithfulRow?.namespaceUri ?? (prefix && rest.length > 0 ? `urn:unknown:${prefix}` : 'urn:surface'), - localName, - unit: detailUnit(input.row, faithfulRow), - values: { ...input.row.values }, - sourceFactIds: [...input.row.sourceFactIds], - isExtension: faithfulRow?.isExtension ?? false, - dimensionsSummary: faithfulRow?.hasDimensions ? ['has_dimensions'] : [], - residualFlag: input.parentSurfaceKey === 'unmapped' - }; -} - -function baselineForStatement(statement: CompactStatement, rowByKey: Map) { - const anchorKey = statement === 'balance' ? 'total_assets' : 'revenue'; - return maxAbsValue(rowByKey.get(anchorKey)?.values ?? {}); -} - -function materialityThreshold(statement: CompactStatement, baseline: number) { - if (statement === 'balance') { - return Math.max(5_000_000, baseline * 0.005); - } - - return Math.max(1_000_000, baseline * 0.01); -} - -export function buildCompactHydrationModel(input: { - periods: FinancialStatementPeriod[]; - faithfulRows: Record; - facts: TaxonomyFactRow[]; - kpiRows?: StructuredKpiRow[]; -}) { - const surfaceRows = structuredClone(EMPTY_SURFACE_ROWS); - const detailRows = structuredClone(EMPTY_DETAIL_ROWS); - let surfaceRowCount = 0; - let detailRowCount = 0; - let unmappedRowCount = 0; - let materialUnmappedRowCount = 0; - - for (const statement of Object.keys(SURFACE_DEFINITIONS) as CompactStatement[]) { - const faithfulRows = input.faithfulRows[statement] ?? []; - const facts = input.facts.filter((fact) => fact.statement === statement); - const fullRows = buildStandardizedRows({ - rows: faithfulRows, - statement, - periods: input.periods, - facts - }); - const rowByKey = new Map(fullRows.map((row) => [row.key, row])); - const faithfulRowByKey = new Map(faithfulRows.map((row) => [row.key, row])); - const statementDetails: SurfaceDetailMap = {}; - - for (const definition of SURFACE_DEFINITIONS[statement]) { - const contributingRows = definition.rowKey - ? [rowByKey.get(definition.rowKey)].filter((row): row is SurfaceFinancialRow => row !== undefined) - : (definition.componentKeys ?? []) - .map((key) => rowByKey.get(key)) - .filter((row): row is SurfaceFinancialRow => row !== undefined); - - const values = Object.fromEntries(input.periods.map((period) => { - const nextValue = definition.rowKey - ? valueForPeriod(rowByKey, definition.rowKey, period.id) - : definition.formula - ? (() => { - const left = valueForPeriod(rowByKey, definition.formula!.left, period.id); - const right = valueForPeriod(rowByKey, definition.formula!.right, period.id); - return left === null || right === null ? null : left - right; - })() - : sumValues(contributingRows.map((row) => row.values[period.id] ?? null)); - - return [period.id, nextValue]; - })) satisfies Record; - - if (!rowHasAnyValue({ values })) { - continue; - } - - const sourceConcepts = [...new Set(contributingRows.flatMap((row) => row.sourceConcepts))].sort((left, right) => left.localeCompare(right)); - const sourceRowKeys = [...new Set(contributingRows.flatMap((row) => row.sourceRowKeys))].sort((left, right) => left.localeCompare(right)); - const sourceFactIds = [...new Set(contributingRows.flatMap((row) => row.sourceFactIds))].sort((left, right) => left - right); - const hasDimensions = contributingRows.some((row) => row.hasDimensions); - const resolvedSourceRowKeys = Object.fromEntries(input.periods.map((period) => [ - period.id, - definition.rowKey - ? rowByKey.get(definition.rowKey)?.resolvedSourceRowKeys[period.id] ?? null - : null - ])); - - const rowsForDetail = definition.componentKeys - ? contributingRows - : []; - const details = rowsForDetail - .filter((row) => rowHasAnyValue(row)) - .map((row) => buildDetailRow({ - row, - parentSurfaceKey: definition.key, - faithfulRowByKey - })); - - statementDetails[definition.key] = details; - detailRowCount += details.length; - - surfaceRows[statement].push({ - key: definition.key, - label: definition.label, - category: definition.category, - templateSection: definition.category, - order: definition.order, - unit: definition.unit, - values, - sourceConcepts, - sourceRowKeys, - sourceFactIds, - formulaKey: definition.formula ? definition.key : null, - hasDimensions, - resolvedSourceRowKeys, - statement, - detailCount: details.length - }); - surfaceRowCount += 1; - } - - const baseline = baselineForStatement(statement, rowByKey); - const threshold = materialityThreshold(statement, baseline); - const residualRows = fullRows - .filter((row) => row.key.startsWith('other:')) - .filter((row) => rowHasAnyValue(row)) - .map((row) => buildDetailRow({ - row, - parentSurfaceKey: 'unmapped', - faithfulRowByKey - })); - - if (residualRows.length > 0) { - statementDetails.unmapped = residualRows; - detailRowCount += residualRows.length; - unmappedRowCount += residualRows.length; - materialUnmappedRowCount += residualRows.filter((row) => maxAbsValue(row.values) >= threshold).length; - } - - detailRows[statement] = statementDetails; - } - - const normalizationSummary: NormalizationSummary = { - surfaceRowCount, - detailRowCount, - kpiRowCount: input.kpiRows?.length ?? 0, - unmappedRowCount, - materialUnmappedRowCount, - warnings: [] - }; - - return { - surfaceRows, - detailRows, - normalizationSummary - }; -} diff --git a/lib/server/taxonomy/materialize.ts b/lib/server/taxonomy/materialize.ts deleted file mode 100644 index c5b4299..0000000 --- a/lib/server/taxonomy/materialize.ts +++ /dev/null @@ -1,400 +0,0 @@ -import type { Filing, FinancialStatementKind, TaxonomyStatementRow } from '@/lib/types'; -import type { TaxonomyConcept, TaxonomyFact, TaxonomyPresentationConcept } from '@/lib/server/taxonomy/types'; -import type { FilingTaxonomyPeriod } from '@/lib/server/repos/filing-taxonomy'; -import { classifyStatementRole, conceptStatementFallback } from '@/lib/server/taxonomy/classifiers'; - -function compactAccessionNumber(value: string) { - return value.replace(/-/g, ''); -} - -function isUsGaapNamespace(namespaceUri: string) { - return /fasb\.org\/us-gaap/i.test(namespaceUri) || /us-gaap/i.test(namespaceUri); -} - -function splitConceptKey(conceptKey: string) { - const index = conceptKey.lastIndexOf('#'); - if (index < 0) { - return { - namespaceUri: 'urn:unknown', - localName: conceptKey - }; - } - - return { - namespaceUri: conceptKey.slice(0, index), - localName: conceptKey.slice(index + 1) - }; -} - -function localNameToLabel(localName: string) { - return localName - .replace(/([a-z0-9])([A-Z])/g, '$1 $2') - .replace(/([A-Z]+)([A-Z][a-z])/g, '$1 $2') - .replace(/_/g, ' ') - .trim(); -} - -function createStatementRecord(factory: () => T): Record { - return { - income: factory(), - balance: factory(), - cash_flow: factory(), - equity: factory(), - comprehensive_income: factory() - }; -} - -function periodSignature(fact: TaxonomyFact) { - const start = fact.periodStart ?? ''; - const end = fact.periodEnd ?? ''; - const instant = fact.periodInstant ?? ''; - return `start:${start}|end:${end}|instant:${instant}`; -} - -function periodDate(fact: TaxonomyFact, fallbackDate: string) { - return fact.periodEnd ?? fact.periodInstant ?? fallbackDate; -} - -function parseEpoch(value: string | null) { - if (!value) { - return Number.NaN; - } - - return Date.parse(value); -} - -function sortPeriods(periods: FilingTaxonomyPeriod[]) { - return [...periods].sort((left, right) => { - const leftDate = parseEpoch(left.periodEnd ?? left.filingDate); - const rightDate = parseEpoch(right.periodEnd ?? right.filingDate); - - if (Number.isFinite(leftDate) && Number.isFinite(rightDate) && leftDate !== rightDate) { - return leftDate - rightDate; - } - - return left.id.localeCompare(right.id); - }); -} - -function pickPreferredFact(facts: T[]) { - if (facts.length === 0) { - return null; - } - - const ordered = [...facts].sort((left, right) => { - const leftScore = left.isDimensionless ? 1 : 0; - const rightScore = right.isDimensionless ? 1 : 0; - if (leftScore !== rightScore) { - return rightScore - leftScore; - } - - const leftDate = parseEpoch(left.periodEnd ?? left.periodInstant); - const rightDate = parseEpoch(right.periodEnd ?? right.periodInstant); - if (Number.isFinite(leftDate) && Number.isFinite(rightDate) && leftDate !== rightDate) { - return rightDate - leftDate; - } - - return Math.abs(right.value) - Math.abs(left.value); - }); - - return ordered[0] ?? null; -} - -export function materializeTaxonomyStatements(input: { - filingId: number; - accessionNumber: string; - filingDate: string; - filingType: '10-K' | '10-Q'; - facts: TaxonomyFact[]; - presentation: TaxonomyPresentationConcept[]; - labelByConcept: Map; -}) { - const periodBySignature = new Map(); - const compactAccession = compactAccessionNumber(input.accessionNumber); - - for (const fact of input.facts) { - const signature = periodSignature(fact); - if (periodBySignature.has(signature)) { - continue; - } - - const date = periodDate(fact, input.filingDate); - const id = `${date}-${compactAccession}-${periodBySignature.size + 1}`; - - periodBySignature.set(signature, { - id, - filingId: input.filingId, - accessionNumber: input.accessionNumber, - filingDate: input.filingDate, - periodStart: fact.periodStart, - periodEnd: fact.periodEnd ?? fact.periodInstant ?? input.filingDate, - filingType: input.filingType, - periodLabel: fact.periodInstant && !fact.periodStart - ? 'Instant' - : fact.periodStart && fact.periodEnd - ? `${fact.periodStart} to ${fact.periodEnd}` - : 'Filing Period' - }); - } - - const periods = sortPeriods([...periodBySignature.values()]); - const periodIdBySignature = new Map( - [...periodBySignature.entries()].map(([signature, period]) => [signature, period.id]) - ); - - const presentationByConcept = new Map(); - for (const node of input.presentation) { - const existing = presentationByConcept.get(node.conceptKey); - if (existing) { - existing.push(node); - } else { - presentationByConcept.set(node.conceptKey, [node]); - } - } - - const enrichedFacts = input.facts.map((fact, index) => { - const nodes = presentationByConcept.get(fact.conceptKey) ?? []; - const bestNode = nodes[0] ?? null; - const statementKind = bestNode - ? classifyStatementRole(bestNode.roleUri) - : conceptStatementFallback(fact.localName); - - return { - ...fact, - __sourceFactId: index + 1, - statement_kind: statementKind, - role_uri: bestNode?.roleUri ?? null - }; - }); - - const rowsByStatement = createStatementRecord(() => []); - const conceptByKey = new Map(); - const groupedByStatement = createStatementRecord>(() => new Map()); - - for (const fact of enrichedFacts) { - if (!fact.statement_kind) { - continue; - } - - const group = groupedByStatement[fact.statement_kind].get(fact.conceptKey); - if (group) { - group.push(fact); - } else { - groupedByStatement[fact.statement_kind].set(fact.conceptKey, [fact]); - } - } - - for (const statement of Object.keys(rowsByStatement) as FinancialStatementKind[]) { - const conceptKeys = new Set(); - - for (const node of input.presentation) { - if (classifyStatementRole(node.roleUri) !== statement) { - continue; - } - - conceptKeys.add(node.conceptKey); - } - - for (const conceptKey of groupedByStatement[statement].keys()) { - conceptKeys.add(conceptKey); - } - - const orderedConcepts = [...conceptKeys] - .map((conceptKey) => { - const presentationNodes = input.presentation.filter( - (node) => node.conceptKey === conceptKey && classifyStatementRole(node.roleUri) === statement - ); - const presentationOrder = presentationNodes.length > 0 - ? Math.min(...presentationNodes.map((node) => node.order)) - : Number.MAX_SAFE_INTEGER; - const presentationDepth = presentationNodes.length > 0 - ? Math.min(...presentationNodes.map((node) => node.depth)) - : 0; - const roleUri = presentationNodes[0]?.roleUri ?? null; - const parentConceptKey = presentationNodes[0]?.parentConceptKey ?? null; - return { - conceptKey, - presentationOrder, - presentationDepth, - roleUri, - parentConceptKey - }; - }) - .sort((left, right) => { - if (left.presentationOrder !== right.presentationOrder) { - return left.presentationOrder - right.presentationOrder; - } - - return left.conceptKey.localeCompare(right.conceptKey); - }); - - for (const orderedConcept of orderedConcepts) { - const facts = groupedByStatement[statement].get(orderedConcept.conceptKey) ?? []; - const { namespaceUri, localName } = splitConceptKey(orderedConcept.conceptKey); - const qname = facts[0]?.qname ?? `unknown:${localName}`; - const label = input.labelByConcept.get(orderedConcept.conceptKey) ?? localNameToLabel(localName); - const values: Record = {}; - const units: Record = {}; - - const factGroups = new Map(); - for (const fact of facts) { - const signature = periodSignature(fact); - const group = factGroups.get(signature); - if (group) { - group.push(fact); - } else { - factGroups.set(signature, [fact]); - } - } - - const sourceFactIds: number[] = []; - let hasDimensions = false; - for (const [signature, group] of factGroups.entries()) { - const periodId = periodIdBySignature.get(signature); - if (!periodId) { - continue; - } - - const preferred = pickPreferredFact(group); - if (!preferred) { - continue; - } - - values[periodId] = preferred.value; - units[periodId] = preferred.unit; - const sourceFactId = (preferred as { __sourceFactId?: number }).__sourceFactId; - if (typeof sourceFactId === 'number') { - sourceFactIds.push(sourceFactId); - } - - if (group.some((entry) => !entry.isDimensionless)) { - hasDimensions = true; - } - } - - if (Object.keys(values).length === 0) { - continue; - } - - const row: TaxonomyStatementRow = { - key: orderedConcept.conceptKey, - label, - conceptKey: orderedConcept.conceptKey, - qname, - namespaceUri, - localName, - isExtension: !isUsGaapNamespace(namespaceUri), - statement, - roleUri: orderedConcept.roleUri, - order: Number.isFinite(orderedConcept.presentationOrder) - ? orderedConcept.presentationOrder - : rowsByStatement[statement].length + 1, - depth: orderedConcept.presentationDepth, - parentKey: orderedConcept.parentConceptKey, - values, - units, - hasDimensions, - sourceFactIds - }; - - rowsByStatement[statement].push(row); - - if (!conceptByKey.has(orderedConcept.conceptKey)) { - conceptByKey.set(orderedConcept.conceptKey, { - concept_key: orderedConcept.conceptKey, - qname, - namespace_uri: namespaceUri, - local_name: localName, - label, - is_extension: !isUsGaapNamespace(namespaceUri), - balance: null, - period_type: null, - data_type: null, - statement_kind: statement, - role_uri: orderedConcept.roleUri, - authoritative_concept_key: null, - mapping_method: null, - surface_key: null, - detail_parent_surface_key: null, - kpi_key: null, - residual_flag: false, - presentation_order: row.order, - presentation_depth: row.depth, - parent_concept_key: row.parentKey, - is_abstract: /abstract/i.test(localName) - }); - } - } - } - - for (const fact of enrichedFacts) { - if (conceptByKey.has(fact.conceptKey)) { - continue; - } - - conceptByKey.set(fact.conceptKey, { - concept_key: fact.conceptKey, - qname: fact.qname, - namespace_uri: fact.namespaceUri, - local_name: fact.localName, - label: input.labelByConcept.get(fact.conceptKey) ?? localNameToLabel(fact.localName), - is_extension: !isUsGaapNamespace(fact.namespaceUri), - balance: null, - period_type: null, - data_type: fact.dataType, - statement_kind: fact.statement_kind, - role_uri: fact.role_uri, - authoritative_concept_key: null, - mapping_method: null, - surface_key: null, - detail_parent_surface_key: null, - kpi_key: null, - residual_flag: false, - presentation_order: null, - presentation_depth: null, - parent_concept_key: null, - is_abstract: /abstract/i.test(fact.localName) - }); - } - - const concepts = [...conceptByKey.values()]; - const factRows = enrichedFacts.map((fact) => ({ - concept_key: fact.conceptKey, - qname: fact.qname, - namespace_uri: fact.namespaceUri, - local_name: fact.localName, - data_type: fact.dataType, - statement_kind: fact.statement_kind, - role_uri: fact.role_uri, - authoritative_concept_key: null, - mapping_method: null, - surface_key: null, - detail_parent_surface_key: null, - kpi_key: null, - residual_flag: false, - context_id: fact.contextId, - unit: fact.unit, - decimals: fact.decimals, - precision: fact.precision, - nil: fact.nil, - value_num: fact.value, - period_start: fact.periodStart, - period_end: fact.periodEnd, - period_instant: fact.periodInstant, - dimensions: fact.dimensions, - is_dimensionless: fact.isDimensionless, - source_file: fact.sourceFile, - })); - - const dimensionsCount = enrichedFacts.reduce((total, fact) => { - return total + fact.dimensions.length; - }, 0); - - return { - periods, - statement_rows: rowsByStatement, - concepts, - facts: factRows, - dimensionsCount - }; -} diff --git a/rust/taxonomy/fiscal/v1/core.income-bridge.json b/rust/taxonomy/fiscal/v1/core.income-bridge.json index bbe5d04..f2a1500 100644 --- a/rust/taxonomy/fiscal/v1/core.income-bridge.json +++ b/rust/taxonomy/fiscal/v1/core.income-bridge.json @@ -27,6 +27,31 @@ "not_meaningful_for_pack": false, "warning_codes_when_used": [] }, + "cost_of_revenue": { + "direct_authoritative_concepts": [ + "us-gaap:CostOfRevenue" + ], + "direct_source_concepts": [ + "CostOfRevenue", + "CostOfGoodsSold", + "CostOfSales", + "CostOfGoodsAndServicesSold", + "CostOfGoodsAndServiceExcludingDepreciationDepletionAndAmortization", + "CostOfProductsSold", + "CostOfServices" + ], + "component_surfaces": { + "positive": [], + "negative": [] + }, + "component_concept_groups": { + "positive": [], + "negative": [] + }, + "formula": "direct", + "not_meaningful_for_pack": false, + "warning_codes_when_used": [] + }, "gross_profit": { "direct_authoritative_concepts": [ "us-gaap:GrossProfit" @@ -38,24 +63,13 @@ "positive": [ "revenue" ], - "negative": [] + "negative": [ + "cost_of_revenue" + ] }, "component_concept_groups": { "positive": [], - "negative": [ - { - "name": "cost_of_revenue", - "concepts": [ - "us-gaap:CostOfRevenue", - "us-gaap:CostOfGoodsSold", - "us-gaap:CostOfSales", - "us-gaap:CostOfGoodsAndServicesSold", - "us-gaap:CostOfGoodsAndServiceExcludingDepreciationDepletionAndAmortization", - "us-gaap:CostOfProductsSold", - "us-gaap:CostOfServices" - ] - } - ] + "negative": [] }, "formula": "subtract", "not_meaningful_for_pack": false, diff --git a/rust/taxonomy/fiscal/v1/core.surface.json b/rust/taxonomy/fiscal/v1/core.surface.json index 755b471..468e443 100644 --- a/rust/taxonomy/fiscal/v1/core.surface.json +++ b/rust/taxonomy/fiscal/v1/core.surface.json @@ -16,6 +16,64 @@ "detail_grouping_policy": "top_level_only", "materiality_policy": "income_default" }, + { + "surface_key": "cost_of_revenue", + "statement": "income", + "label": "Cost of Revenue", + "category": "surface", + "order": 20, + "unit": "currency", + "rollup_policy": "direct_only", + "allowed_source_concepts": [ + "us-gaap:CostOfRevenue", + "us-gaap:CostOfGoodsSold", + "us-gaap:CostOfSales", + "us-gaap:CostOfGoodsAndServicesSold", + "us-gaap:CostOfGoodsAndServiceExcludingDepreciationDepletionAndAmortization", + "us-gaap:CostOfProductsSold", + "us-gaap:CostOfServices" + ], + "allowed_authoritative_concepts": ["us-gaap:CostOfRevenue"], + "formula_fallback": null, + "detail_grouping_policy": "top_level_only", + "materiality_policy": "income_default" + }, + { + "surface_key": "gross_profit", + "statement": "income", + "label": "Gross Profit", + "category": "surface", + "order": 30, + "unit": "currency", + "rollup_policy": "direct_or_formula", + "allowed_source_concepts": ["us-gaap:GrossProfit"], + "allowed_authoritative_concepts": ["us-gaap:GrossProfit"], + "formula_fallback": { + "op": "subtract", + "sources": ["revenue", "cost_of_revenue"], + "treat_null_as_zero": false + }, + "detail_grouping_policy": "top_level_only", + "materiality_policy": "income_default" + }, + { + "surface_key": "gross_margin", + "statement": "income", + "label": "Gross Margin", + "category": "derived", + "order": 35, + "unit": "percent", + "rollup_policy": "formula_only", + "allowed_source_concepts": [], + "allowed_authoritative_concepts": [], + "formula_fallback": { + "op": "divide", + "sources": ["gross_profit", "revenue"], + "treat_null_as_zero": false + }, + "detail_grouping_policy": "top_level_only", + "materiality_policy": "income_default" + }, { "surface_key": "operating_expenses", "statement": "income", @@ -24,12 +82,257 @@ "order": 40, "unit": "currency", "rollup_policy": "aggregate_children", - "allowed_source_concepts": ["us-gaap:SellingGeneralAndAdministrativeExpense", "us-gaap:ResearchAndDevelopmentExpense"], + "allowed_source_concepts": ["us-gaap:OperatingExpenses"], "allowed_authoritative_concepts": ["us-gaap:OperatingExpenses"], "formula_fallback": "sum(detail_rows)", "detail_grouping_policy": "group_all_children", "materiality_policy": "income_default" }, + { + "surface_key": "selling_general_and_administrative", + "statement": "income", + "label": "Selling, General & Administrative", + "category": "surface", + "order": 45, + "unit": "currency", + "rollup_policy": "direct_or_formula", + "allowed_source_concepts": [ + "us-gaap:SellingGeneralAndAdministrativeExpense", + "us-gaap:SellingGeneralAndAdministrativeExpenseExcludingEmployeeStockOptionPlanSpecialDividendCompensation" + ], + "allowed_authoritative_concepts": ["us-gaap:SellingGeneralAndAdministrativeExpense"], + "formula_fallback": { + "op": "sum", + "sources": ["sales_and_marketing", "general_and_administrative"], + "treat_null_as_zero": true + }, + "detail_grouping_policy": "top_level_only", + "materiality_policy": "income_default" + }, + { + "surface_key": "research_and_development", + "statement": "income", + "label": "Research & Development", + "category": "surface", + "order": 50, + "unit": "currency", + "rollup_policy": "direct_only", + "allowed_source_concepts": ["us-gaap:ResearchAndDevelopmentExpense"], + "allowed_authoritative_concepts": ["us-gaap:ResearchAndDevelopmentExpense"], + "formula_fallback": null, + "detail_grouping_policy": "top_level_only", + "materiality_policy": "income_default" + }, + { + "surface_key": "depreciation_and_amortization", + "statement": "income", + "label": "Depreciation & Amortization", + "category": "surface", + "order": 55, + "unit": "currency", + "rollup_policy": "direct_only", + "allowed_source_concepts": [ + "us-gaap:DepreciationDepletionAndAmortization", + "us-gaap:DepreciationAmortizationAndAccretionNet", + "us-gaap:DepreciationAndAmortization", + "us-gaap:DepreciationAmortizationAndOther", + "us-gaap:CostOfGoodsAndServicesSoldDepreciationAndAmortization" + ], + "allowed_authoritative_concepts": [ + "us-gaap:DepreciationDepletionAndAmortization", + "us-gaap:DepreciationAmortizationAndAccretionNet", + "us-gaap:DepreciationAndAmortization", + "us-gaap:DepreciationAmortizationAndOther" + ], + "formula_fallback": null, + "detail_grouping_policy": "top_level_only", + "materiality_policy": "income_default" + }, + { + "surface_key": "stock_based_compensation", + "statement": "income", + "label": "Stock-Based Compensation", + "category": "surface", + "order": 58, + "unit": "currency", + "rollup_policy": "direct_only", + "allowed_source_concepts": [ + "us-gaap:ShareBasedCompensation", + "us-gaap:AllocatedShareBasedCompensationExpense" + ], + "allowed_authoritative_concepts": [ + "us-gaap:ShareBasedCompensation", + "us-gaap:AllocatedShareBasedCompensationExpense" + ], + "formula_fallback": null, + "detail_grouping_policy": "top_level_only", + "materiality_policy": "income_default" + }, + { + "surface_key": "operating_income", + "statement": "income", + "label": "Operating Income", + "category": "surface", + "order": 60, + "unit": "currency", + "rollup_policy": "direct_or_formula", + "allowed_source_concepts": [ + "us-gaap:OperatingIncomeLoss", + "us-gaap:IncomeFromOperations", + "us-gaap:OperatingProfit" + ], + "allowed_authoritative_concepts": ["us-gaap:OperatingIncomeLoss"], + "formula_fallback": { + "op": "subtract", + "sources": ["gross_profit", "operating_expenses"], + "treat_null_as_zero": false + }, + "detail_grouping_policy": "top_level_only", + "materiality_policy": "income_default" + }, + { + "surface_key": "operating_margin", + "statement": "income", + "label": "Operating Margin", + "category": "derived", + "order": 65, + "unit": "percent", + "rollup_policy": "formula_only", + "allowed_source_concepts": [], + "allowed_authoritative_concepts": [], + "formula_fallback": { + "op": "divide", + "sources": ["operating_income", "revenue"], + "treat_null_as_zero": false + }, + "detail_grouping_policy": "top_level_only", + "materiality_policy": "income_default" + }, + { + "surface_key": "interest_income", + "statement": "income", + "label": "Interest Income", + "category": "surface", + "order": 70, + "unit": "currency", + "rollup_policy": "direct_only", + "allowed_source_concepts": [ + "us-gaap:InterestIncomeOther", + "us-gaap:InvestmentIncomeInterest" + ], + "allowed_authoritative_concepts": ["us-gaap:InterestIncomeOther"], + "formula_fallback": null, + "detail_grouping_policy": "top_level_only", + "materiality_policy": "income_default" + }, + { + "surface_key": "interest_expense", + "statement": "income", + "label": "Interest Expense", + "category": "surface", + "order": 75, + "unit": "currency", + "rollup_policy": "direct_only", + "allowed_source_concepts": [ + "us-gaap:InterestIncomeExpenseNonoperatingNet", + "us-gaap:InterestExpense", + "us-gaap:InterestAndDebtExpense" + ], + "allowed_authoritative_concepts": ["us-gaap:InterestExpense"], + "formula_fallback": null, + "detail_grouping_policy": "top_level_only", + "materiality_policy": "income_default", + "sign_transform": "absolute" + }, + { + "surface_key": "other_non_operating_income", + "statement": "income", + "label": "Other Non-Operating Income", + "category": "surface", + "order": 78, + "unit": "currency", + "rollup_policy": "direct_only", + "allowed_source_concepts": [ + "us-gaap:OtherNonoperatingIncomeExpense", + "us-gaap:NonoperatingIncomeExpense" + ], + "allowed_authoritative_concepts": ["us-gaap:OtherNonoperatingIncomeExpense"], + "formula_fallback": null, + "detail_grouping_policy": "top_level_only", + "materiality_policy": "income_default" + }, + { + "surface_key": "pretax_income", + "statement": "income", + "label": "Pretax Income", + "category": "surface", + "order": 80, + "unit": "currency", + "rollup_policy": "direct_or_formula", + "allowed_source_concepts": [ + "us-gaap:IncomeLossFromContinuingOperationsBeforeIncomeTaxesExtraordinaryItemsNoncontrollingInterest", + "us-gaap:IncomeBeforeTaxExpenseBenefit", + "us-gaap:PretaxIncome" + ], + "allowed_authoritative_concepts": ["us-gaap:IncomeBeforeTaxExpenseBenefit"], + "formula_fallback": { + "op": "sum", + "sources": ["net_income", "income_tax_expense"], + "treat_null_as_zero": false + }, + "detail_grouping_policy": "top_level_only", + "materiality_policy": "income_default" + }, + { + "surface_key": "income_tax_expense", + "statement": "income", + "label": "Income Tax Expense", + "category": "surface", + "order": 85, + "unit": "currency", + "rollup_policy": "direct_only", + "allowed_source_concepts": ["us-gaap:IncomeTaxExpenseBenefit"], + "allowed_authoritative_concepts": ["us-gaap:IncomeTaxExpenseBenefit"], + "formula_fallback": null, + "detail_grouping_policy": "top_level_only", + "materiality_policy": "income_default" + }, + { + "surface_key": "effective_tax_rate", + "statement": "income", + "label": "Effective Tax Rate", + "category": "derived", + "order": 87, + "unit": "percent", + "rollup_policy": "formula_only", + "allowed_source_concepts": [], + "allowed_authoritative_concepts": [], + "formula_fallback": { + "op": "divide", + "sources": ["income_tax_expense", "pretax_income"], + "treat_null_as_zero": false + }, + "detail_grouping_policy": "top_level_only", + "materiality_policy": "income_default" + }, + { + "surface_key": "ebitda", + "statement": "income", + "label": "EBITDA", + "category": "derived", + "order": 88, + "unit": "currency", + "rollup_policy": "formula_only", + "allowed_source_concepts": [], + "allowed_authoritative_concepts": [], + "formula_fallback": { + "op": "sum", + "sources": ["operating_income", "depreciation_and_amortization"], + "treat_null_as_zero": true + }, + "detail_grouping_policy": "top_level_only", + "materiality_policy": "income_default" + }, { "surface_key": "net_income", "statement": "income", @@ -44,6 +347,100 @@ "detail_grouping_policy": "top_level_only", "materiality_policy": "income_default" }, + { + "surface_key": "net_income_attributable_to_common", + "statement": "income", + "label": "Net Income Attributable to Common", + "category": "surface", + "order": 92, + "unit": "currency", + "rollup_policy": "direct_only", + "allowed_source_concepts": [ + "us-gaap:NetIncomeLossAvailableToCommonStockholdersBasic" + ], + "allowed_authoritative_concepts": [ + "us-gaap:NetIncomeLossAvailableToCommonStockholdersBasic" + ], + "formula_fallback": null, + "detail_grouping_policy": "top_level_only", + "materiality_policy": "income_default" + }, + { + "surface_key": "diluted_eps", + "statement": "income", + "label": "Diluted EPS", + "category": "surface", + "order": 100, + "unit": "currency", + "rollup_policy": "direct_only", + "allowed_source_concepts": [ + "us-gaap:EarningsPerShareDiluted", + "us-gaap:DilutedEarningsPerShare" + ], + "allowed_authoritative_concepts": [ + "us-gaap:EarningsPerShareDiluted" + ], + "formula_fallback": null, + "detail_grouping_policy": "top_level_only", + "materiality_policy": "income_default" + }, + { + "surface_key": "basic_eps", + "statement": "income", + "label": "Basic EPS", + "category": "surface", + "order": 105, + "unit": "currency", + "rollup_policy": "direct_only", + "allowed_source_concepts": [ + "us-gaap:EarningsPerShareBasic", + "us-gaap:BasicEarningsPerShare" + ], + "allowed_authoritative_concepts": [ + "us-gaap:EarningsPerShareBasic" + ], + "formula_fallback": null, + "detail_grouping_policy": "top_level_only", + "materiality_policy": "income_default" + }, + { + "surface_key": "diluted_shares", + "statement": "income", + "label": "Diluted Shares Outstanding", + "category": "surface", + "order": 110, + "unit": "shares", + "rollup_policy": "direct_only", + "allowed_source_concepts": [ + "us-gaap:WeightedAverageNumberOfDilutedSharesOutstanding", + "us-gaap:WeightedAverageNumberOfShareOutstandingDiluted" + ], + "allowed_authoritative_concepts": [ + "us-gaap:WeightedAverageNumberOfDilutedSharesOutstanding" + ], + "formula_fallback": null, + "detail_grouping_policy": "top_level_only", + "materiality_policy": "income_default" + }, + { + "surface_key": "basic_shares", + "statement": "income", + "label": "Basic Shares Outstanding", + "category": "surface", + "order": 115, + "unit": "shares", + "rollup_policy": "direct_only", + "allowed_source_concepts": [ + "us-gaap:WeightedAverageNumberOfSharesOutstandingBasic", + "us-gaap:WeightedAverageNumberOfShareOutstandingBasicAndDiluted" + ], + "allowed_authoritative_concepts": [ + "us-gaap:WeightedAverageNumberOfSharesOutstandingBasic" + ], + "formula_fallback": null, + "detail_grouping_policy": "top_level_only", + "materiality_policy": "income_default" + }, { "surface_key": "cash_and_equivalents", "statement": "balance",