Financial Product Agent

Evidence-grounded AI analyst for financial products

Issue / Solution

The Issue: Questions like "Which sellable KRW bonds are rated AA- or higher?" or "Which domestic ETFs hold this company, and at what weight?" can only be answered by filtering and aggregating tens of thousands of product records: bonds, domestic and overseas ETFs, and public funds, each with its own schema, codes, and intentionally missing values. General-purpose LLMs answer these fluently but invent products and numbers, and they rarely admit when the data cannot support an answer. In finance, a confident wrong answer is worse than none.

The Solution: I built Financial Product Agent, a question-answering API that treats the LLM as an analyst who must look things up. Each question is pre-analyzed (entity resolution and ontology validation), then deterministically routed to only the stores it needs: a SQLite database, a 1.08M-triple knowledge graph, hybrid lexical/vector search, or reference documents. An LLM tool-calling loop then finishes the computation in SQL. Every response carries its evidence (tables, as-of dates, the exact SQL or SPARQL, product codes) and a numbered reasoning trace, and any answer produced without a single lookup is rejected and sent back.

Overview

Financial Product Agent is a Python/FastAPI service over four Korean product masters (domestic bonds, domestic ETFs/ETNs, overseas ETFs/ETNs, and public funds) totaling 53,375 product records, plus 22,599 ETF constituent holdings. I built it for the financial-product agent challenge at the Mirae Asset Securities AI Festival 2026, where answer generation had to run on NCP HyperCLOVA X within a 300-second limit per question.

The ontology is an execution layer, not documentation. Six Turtle files are loaded at runtime to generate the LLM's schema cards (column meanings, units, allowed values), validate a question's values before any query runs, and drive inference inside the knowledge graph.

System Architecture

A request flows down the left column: pre-analysis, then a retrieval plan that prefetches facts before the model runs. The agent works with the knowledge base on the right through tools, and every answer exits through a strict three-state policy.

System Architecture

What I Built

An ETL pipeline that loads the four Excel product masters and a scraped ETF-constituent dataset into SQLite, with ISO dates, a redefined "sellable" flag, fund-family keys, normalized asset managers, and five views exposing all 215 columns.
Six domain ontologies in Turtle (bonds, domestic ETFs, overseas ETFs, funds, ETF constituents, and shared classes) that the runtime turns into LLM schema cards and value validators.
A knowledge graph of 1,082,874 triples (965,823 asserted, 117,051 inferred) built from the database plus curated dictionaries of 77 asset managers, 48 bilingual themes, and 97 companies, queryable through SPARQL and neighbor lookups.
Hybrid retrieval over 75,089 documents: BM25 with Korean syllable bigrams and exact-name boosting, plus a TF-IDF + LSA vector index that runs deterministically without any embedding API key.
A deterministic retrieval router that prefetches only the stores a question needs before the model runs, and records which stores it chose and why in the trace.
The tool-calling agent loop with a time budget, evidence and trace collection, and a three-state answer policy, running on HyperCLOVA X with Anthropic and OpenAI-compatible providers available for comparison.
A FastAPI service (GET /answer, GET /health) that always returns 200 with the same five-field schema, even on internal errors.
An evaluation harness: a 54-question answer key recomputed from the data at grading time, a 1,313-query retrieval audit, and a 194-test suite.

Grounding Before Fluency

The central design decision is that the model is never trusted to answer from memory. In testing, five runs of the same question split cleanly: every run that called a tool was correct, and every run that answered without one was wrong. So the agent rejects any final answer submitted without at least one lookup and sends the model back to retrieve.

Answers are held to the data in three more ways. Question values are checked against the ontology before any query runs, so a request for a nonexistent credit grade like "AAAA" or a risk grade of 7 is answered as unconfirmable instead of producing an empty or invented list. Every answer must declare one of three states (answered, cannot confirm, or clarify), and a refusal ends as a refusal, with no list of substitute products appended. Missing values stay missing: nulls are never counted or sorted as zero, and the answer says the data is unavailable.

Resolved identifiers are carried all the way into SQL. For 32 companies, the name a user types differs from how the source data stores it (TSMC is stored as "TSMC ADR", for example), so a name-based filter would silently return zero rows. The agent is handed the store's actual company codes instead, and graph results ship with the product IDs that SQL needs.

Retrieval, Decided by Measurement

The hybrid search design came from measurement, not preference. Reciprocal Rank Fusion was the first approach, but on this corpus it let loosely related funds that both retrievers ranked mid-list push out the best lexical match; a query for battery cathode products returned an unrelated fund of funds first. The final design gives the head of the result list to lexical matches and reserves roughly the bottom 30% of slots for documents only semantic search found, so hybrid can never rank worse than lexical at the top.

Vector quality was tuned the same way. Korean product names are full of boilerplate terms such as "securities investment trust" and "exchange-traded", which dominated the LSA space. Dropping the most frequent tokens (max_df = 0.005) raised hits from 1 to 6 of 7 semantic probes.

I then measured how much the vector index actually adds, using 1,313 queries generated from the data across seven types (official names, codes, abbreviations, holding names, themes, typos, and synonyms). Recall@10 by query type:

Official names: lexical 100.0%, vector 60.0%, hybrid 100.0%
Codes and tickers: lexical 49.4%, vector 1.9%, hybrid 49.4%
Typos: lexical 93.1%, vector 45.0%, hybrid 93.1%
Themes: lexical 87.0%, vector 55.8%, hybrid 90.5%

I kept the honest conclusion: hybrid matches lexical exactly at recall@1 and @5, the vector index's net gain is +3.5 percentage points of recall@10 on theme queries, and typo tolerance comes from BM25's Korean bigrams rather than from semantic search.

Knowledge Graph & Ontology

At startup, the ontology's class definitions (TBox) are merged into the same graph as the instance data, and three rules are materialized: superclass inheritance (a domestic ETF is also an ETF and a product), inverse relations (managedBy and manages), and domain/range type inference. Queries like "every product this manager runs" then work without any extra loading.

Modeling choices kept the graph clean. ETF holding weights were first written as one predicate per company, which ballooned the schema to 5,082 predicates. Moving constituents into a SQLite table made weight an ordinary column and cut that to 73, and because the graph's holdings are generated from that same table, SQL and graph answers cannot disagree. Company affiliate relations are derived from group-themed ETFs: 608 edges across 8 business groups and 65 companies, with no false positives found. They are labeled affiliates rather than subsidiaries, because the data contains no ownership stakes.

Data Quality

Missing values in the provided data were intentional, so I left them unfilled, but I repaired gaps in columns I derived. Fund asset-manager names were 25.4% missing because they had been guessed from fund-name prefixes; deriving them from the institution code the source provides for every row raised coverage from 74.6% to 91.2%, with the fill source recorded on each row. The constituent scraper had been discarding stock codes, and keeping them gave 79% of holdings an exact company identifier. Companies are now matched only by code, never by fuzzy name similarity, after an early substring match misattributed 677 rows.

Results

The official evaluation questions were not published, so I built a 54-question answer key whose correct answers are recomputed from the provided data with SQL and graph queries at grading time. Measured with HyperCLOVA X (HCX-005):

Answer Accuracy

47 / 54 (87%)

By difficulty: easy 8/10, medium 14/16, and hard 6/9.

Unanswerable Questions

19 / 19 refused

Every question the data cannot support was correctly answered as unconfirmable.

Latency

20.6s average

Longest response 87.4s, with no responses over the 300-second limit.

Test Suite

194 passing tests

Unit and contract tests on a stub LLM, plus a slow full-ETL validation run.

Deployment

The API runs in Docker Compose on a Naver Cloud VPC server (Ubuntu 24.04, 4 vCPU / 16 GB), served on port 80. The image loads the product store and warms the vector cache at build time. Building the knowledge graph with inference takes about three minutes at startup, so the health check allows a 420-second start period, and the container restarts automatically after crashes or reboots. API keys live in an .env file that is excluded from the image and read at runtime.

Stack

Python 3.12, FastAPI, Uvicorn, Pydantic, SQLite, pandas, openpyxl, rdflib (RDF, OWL, SPARQL), Turtle ontologies, rank-bm25, scikit-learn (TF-IDF, Truncated SVD), NumPy, PyMuPDF, NCP HyperCLOVA X (function calling), Anthropic and OpenAI SDKs, Docker Compose, Naver Cloud Platform, pytest.

(go back)