How to Synthesize Data When You Don’t Have It: A Deterministic Rule-Based Framework for Deriving Enterprise Metrics That No Source System Records

Abstract

Measures such as the cost to serve an individual customer, profit by distribution channel, emissions per unit of product, are regularly taken as enterprise decisions, which are not reported by any source system. Faced with this gap, the common response by practitioners is to give up on the analysis or resort to generative synthetic-data methods that attempt to address some other problem. The paper formalizes a third method deterministic, rule-based data synthesis whereby operative data is used combined with the explicit and versioned business rules to form measures that were never directly measured. The method is broken down into three modes of composition, direct fetch, driver-based allocation and manual percentage allocation. It has roots in activity-based costing, however, the framing focuses on cloud data warehouses and orchestrated transformation pipelines as opposed to purpose-built costing software. A worked example is an end-to-end tracing of profitability of retail-bank customer, a combination of direct revenue capture, branch, contact-centre and digital channel driver-based cost allocation, a regulatory capital charge based on risk-sensitive assets, and an overhead layer based on judges, resulting in a calculated per-customer profit amount that lives nowhere in the source system. A reference implementation to a commodity cloud stack is defined, in which business rules in business-owned configuration are independent of transformation logic in SQL and orchestration logic in directed acyclic graphs, causing one engine to derive any limited metric. The proposed four-condition applicability test separates the problems to which deterministic synthesis is appropriate, and those to which statistical, probabilistic or generative techniques are required, along with the governance controls making the judgement layer auditable, as opposed to arbitrary.