Alpharetta – November 03, 2026 -- Generative large language models (LLMs) computing financial figures directly achieved an exact precision rate of 63.5%, against 100% for SigmaCore.ai's deterministic Eliza engine, according to a benchmark study released by the Georgia-based AI developer.
Code execution lifts LLM accuracy to 94.7% but adds formula-selection risk
The study, titled "Can We Trust the AI to Do the Math?", covered 3,400 financial test cases. LLMs with code execution enabled reached 94.7% exact precision.
The company says the augmented models reduced minor arithmetic slips but introduced logic errors, because they picked the wrong formula. In two retail financial scenarios, accuracy fell to 18% with code tools active. The models generated flawed formulas "with high structural confidence," per the study.
The research found no reliable proxy, such as answer length or operational difficulty, that warns users of an error. Precision degraded sharply as arithmetic complexity increased.
Test design spans 34 calculations and 10,200 executions
The multi-model study logged 10,200 experimental executions across 34 standard commercial financial calculations. These included loan amortization payment schedules, three-year financial projections and break-even assessments used in Small Business Administration (SBA) application streams. Each calculation ran across 100 randomized market scenarios.
SigmaCore.ai has not published the full technical framework, validation methodology or dataset access terms. It provides them on request. The results are self-reported by the vendor of the engine being tested.
Eliza engine carries a pending US patent application
Eliza, covered by US Patent Application 19/770,254, guides small and mid-sized business owners through conversation-driven workflows. It generates bank-ready business plans, three-year financial models and SBA loan documentation. SigmaCore.ai also sells Lex, which automates compliance routing and legal workflows, and Azile, which provides automated financial verification and audit functionality.
The study is the latest release in the company's research program on generative AI hallucination. CEO Subhashish Samaddar leads the program, with Ravi Periasamy as resident AI expert and Shyam Dasari as AI drift researcher.
SigmaCore.ai reaches Georgia AI Pitch Competition finalist stage
SigmaCore.ai was selected as a finalist in the Georgia AI Pitch Competition, hosted with Eficens Systems, AWS, the SBA and the Georgia SBDC. The selection brings $25,000 in proof-of-concept funds, and the company qualifies for the $100,000 grand prize.
The company is also in advanced discussions with multiple financial institutions about potential Eliza deployments. It did not name them.