Case study

FinPilot.

AI agents that work where fintech teams work.

2025
Year
Concept
Type
Fintech
Domain
AI / UX
Focus
Scroll
Context

An AI agent platform for enterprise fintech.

Train it on company knowledge. Connect it to enterprise tools. Deploy it inside the products teams already use. Not a chatbot. An intelligent layer that actually does things.

I designed the full system — research, architecture, interaction model, builder experience, and end-user interface across four fintech roles.

Core question

How do you design an AI system that financial professionals will trust with real money decisions?

Regulatory

Every agent action must be auditable. Compliance is non-negotiable.

Latency

Traders need sub-second responses. 3 seconds = missed opportunity.

Trust

One wrong margin number could cost millions. Accuracy beats speed.

The problem

Six tools. Two hundred PDFs.
Zero connection.

Financial teams aren't slow because they're bad at their jobs. They're slow because their tools don't talk to each other and their knowledge is buried in documents nobody reads.

01

Knowledge locked in PDFs

Finding a margin policy takes 20 minutes of searching through a 240-page manual. Multiply by 15 queries per day.

02

Every task spans 3+ systems

CRM → OMS → compliance rules → Jira. Four tools, four logins, zero integration between any of them.

03

Manual work eats senior time

Reconciliation, compliance checks, and reports are rule-based — but they consume hours that should go toward judgment work.

Research

The people behind the terminals.

Conceptual research based on 12 published fintech workflow studies, SEC enforcement data, and enterprise AI deployment post-mortems. Personas are synthesized archetypes — not direct interviews.

SR
Sarah R.
Senior options trader · 8 years

Manages 40+ positions daily. Needs market data, risk metrics, and execution speed in one view.

Loses 30+ min daily searching for policy details mid-tradeMissed a hedge window because risk data was in a separate system"If it gives me wrong margin numbers, that's my job"
MK
Marcus K.
Compliance analyst · 5 years

Reviews flagged transactions, prepares audit reports. Citation accuracy is everything.

Spends 3+ hours daily on routine reviews that follow clear rulesCan't find which policy version applies to a 6-month-old trade"If I can't cite it, I can't use it"
LP
Lisa P.
Risk manager · 12 years

Monitors portfolio risk across funds. Needs to spot problems before they escalate.

Dashboards show data but don't explain why it changedStress tests need data from 3 systems + manual Excel workReports are outdated by the time they're ready
JT
James T.
Operations analyst · 3 years

Handles settlement, reconciliation, and exception management.

30% of exceptions take 80% of timeNew hires take 4-6 months to handle edge cases alone"The SOP says one thing, reality is different"

What I found.

Insight 01

Trust is the product, not intelligence. An agent that's right 95% but can't show its work is worse than one at 90% that cites everything.

Insight 02

Users don't distrust AI — they distrust outputs they can't verify quickly. If citation + verification takes 30 seconds, adoption follows. If 5 minutes, they'll search the PDF themselves.

Insight 03

The high-value work is connecting systems, not answering questions. An agent that orchestrates across CRM + risk + Jira is 10x more valuable than one that chats.

Insight 04

Different roles need fundamentally different agent personalities. Same brain. Different error tolerances.

Insight 05

The builder (admin) is the forgotten user. Their frustration directly determines agent quality.

"I don't need AI to be smart. I need it to be right, and I need to prove it was right when auditors come knocking."

— Synthesized from compliance analyst patterns

AI is not the interface.
Trust is.

This insight became the foundation for every design decision.

The solution

One brain. Multiple surfaces.
Role-based behavior.

FinPilot has two sides. The builder side — upload knowledge, connect tools, set permissions, test. The user side — the agent deployed inside the products teams already use.

Same trained agent, different face depending on who's using it. Traders get speed. Compliance gets citations. Ops gets checklists.

Rejected

Standalone AI app

Research showed teams won't adopt another tool. The agent must live inside existing workflows.

Rejected

Single interface for all roles

A trader needs action buttons. Compliance needs audit trails. One interface can't serve both.

Three phases.

01

Build the brain

Train on company documents using RAG. Upload → chunk → embed → retrieve.

02

Give it hands

Connect OMS, risk engine, CRM, compliance DB. Agent reasons, calls tools, synthesizes.

03

Deploy into products

Embedded copilot, proactive alerts, standalone workspace, backend automation.

User flow

Marcus investigates a flagged transaction.

Step by step. Where the agent helps. Where the human decides. Where it breaks.

1

Trigger: proactive alert

FinPilot detects a trade pattern matching wash sale criteria. Marcus sees a notification — he didn't search for it.

Alert: "Trade #7842 (Acme Capital) — MSFT sell + repurchase within 14 days. Potential wash sale. Confidence: 87%."
2

Agent pre-compiles context

One click opens a side panel with both trades, time gap, client details, and the relevant regulation — all pulled automatically.

Tools called:
OMS.get_trade_history(client: 4521, ticker: MSFT, 30d)
ComplianceDB.get_rule(wash_sale_irs_1091)
CRM.get_client_profile(4521)
3

Analysis with sources

Agent summarizes the case. Every claim links to its source. Confidence: 87% — the 13% gap is flagged explicitly.

Trust mechanism: Each fact links to its source document. Marcus can click any citation to see the original text. The agent doesn't hide what it doesn't know.
4

Edge case: ambiguous account type

Sell was in taxable account, repurchase in IRA. The agent doesn't guess — it flags: "Cross-account rules apply but enforcement varies. Recommend manual review."

Edge case

The agent states what it doesn't know. Provides the regulation text. Flags for human judgment. This is where Marcus's expertise matters.

5

Marcus makes the call

After reviewing cross-account guidance, he confirms: wash sale violation. Agent executes the mechanical follow-up — not the compliance decision.

Human decision point. The agent presented evidence and ambiguity. Marcus applied judgment. The agent cannot make compliance decisions.
6

Agent handles documentation

Audit record, Jira ticket, compliance database entry — all generated automatically with full citation trail.

ComplianceDB.log_investigation(WS-2025-0342)
Jira.create_ticket(client_notification, medium)
OMS.flag_trade(7842, wash_sale_confirmed)
7

12 minutes. Not 2 hours.

Same investigation. Fraction of the time. Because the agent did the research, and the human did the judgment.

Key principle

The agent never made the compliance decision. It gathered evidence, flagged ambiguity, and documented the outcome. The human made the call.

Impact

Same task. Different reality.

Compliance investigation — before and after. Estimated from workflow simulation.

Before

Manual workflow

1

Scan monitoring report for flags

2

Log into OMS, search trade history

3

Open compliance PDF, search wash sale rules

4

Cross-reference trade dates manually

5

Log into CRM for client details

6

Write summary in Word

7

Create Jira ticket manually

8

Log findings in compliance DB

~2h 15m · 4 systems · 8 manual steps
After

Agent-assisted

Proactive alert with pre-compiled context

Agent pulls trade history, regulation, client data

Analysis with source citations + confidence

Ambiguity flagged — human makes the call

Audit record, Jira ticket, compliance log auto-generated

~12m · 1 interface · 2 human decisions
Estimated based on workflow simulation
Trust & safety

Designing for the moment the agent is wrong.

One wrong margin number = millions in liability. The trust layer isn't a feature. It's the product.

Source citations

Every claim links to document, section, and version. Click to see original text. Not optional — hard requirement for compliance.

Confidence scoring

90%+ = green badge. Below 80% = explicit uncertainty flag. Below 60% = refuses to answer definitively.

Reasoning trace

Expand any response to see: which tools called, which docs retrieved, how the answer was synthesized. Makes the agent's thinking auditable.

Failure handling

Doesn't know? Says so. Tool fails? Clear error + retry. Outdated source? Flags the last-updated date. No silent failures.

Human override

Auto-execute: Read-only queries, data lookups, reports.

Confirm to proceed: Trade modifications, escalations. Agent suggests, human approves.

Human only: Policy changes, regulatory submissions, compliance decisions.

Audit logging

Every interaction logged: query, tools called, sources retrieved, response, user action. Immutable and timestamped.

Constraint

Audit logs retained 7 years per SEC Rule 17a-4. Written to immutable store, not app cache.

01
Phase 1

Build the brain.

Company knowledge → searchable AI memory.

RAG in plain terms.

Documents get broken into chunks, converted to vectors, stored in a specialized database. When someone asks a question, the system finds the most relevant chunks and gives them to the AI as context. No retraining needed.

Why RAG, not fine-tuning?

Fine-tuning = retrain the whole model. Expensive, slow, hard to update. RAG = upload a new doc, it's searchable in minutes.

Trade-off

RAG for factual, document-grounded answers. Fine-tuning for domain-specific reasoning. FinPilot uses RAG primarily.

Ingestion pipeline.

01

Collect

Compliance manuals, trading policies, SOPs, filings.

02

Parse

Extract clean text. Handle PDF layouts, tables, images.

03

Chunk

Semantic splits at topic boundaries. Size matters.

04

Embed

Convert chunks to vectors. Similar meaning = similar numbers.

05

Store

Vector DB + metadata: source, page, section, date.

06

Retrieve

Query → embed → find nearest chunks → feed to LLM.

02
Phase 2

Give it hands.

Tools that let the agent act, not just talk.

OMS

Order management

Trade status, orders, positions

RSK

Risk engine

Stress tests, exposure, VaR

CRM

Client database

Account info, history, records

CDB

Compliance DB

Rules, violations, audit logs

MKT

Market data

Prices, VIX, historical

COM

Communication

Slack, email, Jira

The reasoning loop.

Think → Plan → Execute → Observe → Respond. If insufficient, loop again.

Think
Analyze
Plan
Pick tools
Execute
Call APIs
Observe
Read results
Respond
Or loop
03
Phase 3

Deploy into products.

Four ways the agent reaches users.

Embedded copilot

Inside the existing app

Side panel. Context-aware. Knows what screen you're on. Ask a question, get an answer without leaving your workflow.

Standalone workspace

Deep analysis hub

Full-screen for complex multi-step conversations. Scenarios, comparisons, report generation.

Proactive agent

Monitors and pushes

Doesn't wait. Watches metrics, pushes alerts with context and recommended actions.

Backend automation

Invisible workflows

No UI. Nightly reconciliation, daily compliance checks. Only surfaces exceptions.

The agent handles the research.
The human handles the judgment.

Architecture

Six layers.

Interface
Embedded copilot
Standalone workspace
Alert system
Admin panel
Orchestration
Agent orchestrator
ReAct loop
Workflow engine
Tools / API
OMS
Risk engine
CRM
Compliance DB
Market data
Knowledge
Vector database
Document store
Embedding pipeline
AI / Model
Foundation LLM
Fine-tuned layer
Evaluation suite
Security
Role-based access
Guardrails
Audit logging
7-year retention
Builder experience

The admin who configures it
determines how good it is.

Most AI case studies skip this. But the person uploading documents and debugging wrong outputs is the most underserved user.

"I uploaded our entire compliance library and the agent still gives wrong answers about margin calls. No idea which document is causing it."

— Synthesized from enterprise AI admin frustrations

The real journey.

1

Upload documents

340+ files. Inconsistent formatting, scanned pages, watermarks, broken tables.

UX: Per-file status with actionable errors. "3 files have unreadable pages (scanned without OCR)" — not just a progress bar.
2

System chunks and embeds

Admin doesn't need vector math. They need to verify the system understood their documents.

UX: chunk preview with highlighting. Browse any document, see chunk boundaries overlaid. If a table got split across chunks, you see it immediately.
3

Test and find problems

Agent gives wrong margin answer. Most tools show the wrong answer but no way to trace why.

UX: side-by-side debug. Left: agent response. Right: which chunks retrieved, from which docs, relevance scores. Admin sees: "Pulled from 2019 policy instead of 2024 update."
Failure: wrong answers

Debugging panel shows retrieval chain. Wrong chunks retrieved = document pollution. Right chunks but wrong answer = model reasoning issue. UI flags the distinction.

4

Debug missing data

"Agent says it doesn't know our options SOP, but I uploaded it last week."

UX: knowledge gap detector. Run coverage check — input questions, see which have strong matches vs gaps. For gaps: "Not yet processed? Scanned without OCR? Terminology mismatch?" — with the fix for each.
5

Iterate and deploy

Saved test suite. Automatic accuracy scoring by topic. Hit 90%+ → promote to production.

UX: evaluation dashboard. "Compliance: 94%. Trading policy: 87% — 3 questions consistently wrong." Click failing questions → direct to debug mode.
Rejected

Technical chunking controls (token size, overlap %)

Target admin is PM/ops, not ML engineer. Replaced with 3 presets: Detailed / Balanced / Broad — with visual preview of how each splits a sample document.

Wireframes

Both sides of FinPilot.

Styled low-fidelity. Layout and interaction patterns — not visual polish.

Builder side Admin configures the agent
B-01

Builder dashboard

The home base for agent administrators. Shows agent health, usage metrics, knowledge base status, and quick actions. Designed for a PM or ops lead who manages the agent.

Dashboard Knowledge base Tool connections Permissions Testing playground Settings Agent dashboard FinPilot v2.1 · Last updated 2h ago AGENT STATUS Online Uptime 99.7% · 30d QUERIES TODAY 1,247 ↑ 18% vs yesterday ACCURACY RATE 94.2% Based on user feedback KNOWLEDGE BASE 342 Documents · 12.4k chunks QUERY VOLUME · 30 DAYS RECENT ACTIVITY New doc uploaded: Q4_Compliance_Update.pdf 2m ago Agent answered 23 queries from Trading desk 15m ago Tool connection test failed: Risk Engine API 1h ago Permissions updated for Compliance role 3h ago Weekly evaluation score: 94.2% accuracy 6h ago CONNECTED TOOLS Order Mgmt Latency: 45ms ● Connected Risk Engine Latency: 320ms ● Degraded CRM Latency: 62ms ● Connected Compliance DB Latency: 78ms ● Connected Market Data Latency: 12ms ● Connected
Dashboard provides at-a-glance agent health, usage trends, recent activity feed, and tool connection status with latency indicators
Design decision

Tool health cards show latency numbers, not just green/red status. In financial systems, a "connected" tool with 320ms latency is functionally degraded — traders notice. Showing the number lets admins act before users complain.

B-02

Knowledge base management

Where admins upload, manage, and monitor the company documents that power the agent's knowledge. Shows upload interface, document list, chunk preview, and embedding status.

FinPilot Knowledge base Knowledge base 342 documents · 12,438 chunks · Last sync 2h ago Drop files here or click to upload Supports PDF, DOCX, MD, TXT, CSV · Max 50MB per file Search docs... All types Status ▾ DOCUMENT NAME TYPE CHUNKS STATUS UPDATED PDF Compliance_Manual_v4.2.pdf Policy 847 Embedded Dec 14, 2025 DOC Trading_Policies_2025.docx Policy 324 Embedded Dec 10, 2025 PDF SEC_Regulatory_Filing_Q3.pdf Regulatory 512 Processing Dec 18, 2025 MD Client_Onboarding_SOP.md SOP 156 Embedded Nov 28, 2025 CHUNK PREVIEW · Compliance_Manual_v4.2.pdf · Chunk #142 "Section 4.1 — Margin Requirements: All client accounts must maintain a minimum margin ratio of 25% for equity positions. Accounts falling below this threshold will trigger an automated margin call notification to the client within 24 hours..."
Document management with drag-drop upload, status tracking per file, chunk counts, and inline chunk preview for verification
Why chunk previews exist

Admins need to verify the agent's knowledge is correct before deployment. The chunk preview lets them see exactly what text the agent will retrieve for a given topic — catching bad parsing, broken tables, or garbled PDFs before they cause wrong answers in production.

B-03

Testing playground

Where admins test agent responses before deployment. Shows the conversation, source citations, tool calls made, and confidence scoring — so teams can verify accuracy before going live.

FinPilot Testing playground Testing playground Role: Trader Role: Compliance Role: Risk What's the margin requirement for AAPL options? For AAPL options positions, the margin requirement depends on the strategy: • Long calls/puts: 100% of premium (no margin) • Covered calls: No additional margin required • Naked puts: 20% of underlying + premium Source: Trading_Policies_2025.docx · Section 3.4 94% confident Check if client #4521 has enough margin for this ↗ CRM.get_client_portfolio(4521) Client #4521 (Acme Capital) has $2.4M in available margin. That's sufficient for the naked put strategy on AAPL. Current utilization is at 62%. 98% confident Ask a test question... EVALUATION PANEL Sources used ✓ Trading_Policies_2025.docx ✓ Margin_Requirements_v3.pdf Tools called CRM.get_client_portfolio(4521) ✓ Success · 62ms Reasoning trace 1. Parse question → margin + AAPL options 2. Search KB → Trading_Policies §3.4 3. User asks about client → call CRM 4. Compare margin available vs required 5. Synthesize answer with source cite Rate this response 👍 👎 Add note
Split view: live conversation testing (left) with evaluation panel showing sources, tool calls, reasoning trace, and feedback controls (right)
Why the evaluation panel is always visible

Early concepts hid the reasoning trace behind a "Show details" toggle. We surfaced it permanently because admins testing the agent need to see not just what it said, but why — which sources it used, which tools it called, and how long each took. Hidden details don't get checked. Visible details build trust before deployment.

User side Agent in fintech products
U-01

Embedded copilot — trading terminal

The agent appears as a collapsible side panel within the trader's existing terminal. It has full context of the current view and can execute actions directly. Speed-optimized with one-click confirmations.

TradeView Pro MARKET ORDERS POSITIONS AAPL $198.42 +2.14 (+1.09%) OPEN POSITIONS AAPL 200C Dec +50 contracts +$12,400 NVDA 480P Jan +25 contracts -$3,200 SPY 580C Feb +100 contracts +$8,750 QUICK ORDER AAPL BUY SELL FinPilot Trader × Viewing: AAPL · 3 open positions Best hedge for my AAPL calls? Given your 50 AAPL 200C Dec: 1. Protective put (195P) — cost: $4.2k 2. Collar (195P/210C) — cost: $1.8k 3. Reduce position by 40% — no cost Based on current IV: 28.4% · Greeks analyzed Execute collar → Details 96% confident Show unusual activity in tech ↗ MarketData.scan_unusual() 3 unusual activity alerts: • MSFT 420C — 5x avg vol • META 520P — 3x avg vol • GOOGL 180C — 2x avg vol Ask FinPilot... QUICK ACTIONS Risk check P&L summary Market pulse
Copilot panel (right) is context-aware — it knows the trader is viewing AAPL with 3 open positions. Responses include actionable buttons for one-click execution.
Why one-click execution, not chat-only

Traders operate in milliseconds. Typing "execute the collar strategy" and waiting for a response costs time. The one-click "Execute collar →" button appears directly in the agent's recommendation — reducing the action to a single tap. The agent suggests, the trader confirms with muscle memory, not conversation.

U-02

Proactive alert — risk dashboard

The agent monitors risk metrics continuously and pushes alerts when thresholds are breached. Shows the proactive notification banner, the risk dashboard context, and suggested actions — the agent reaches out before the user asks.

RiskView Pro ! FinPilot Alert: Portfolio VaR exceeded threshold by 18% — Energy sector exposure spiked due to crude oil movement Triggered 3 min ago · Auto-analysis complete · 3 recommended actions View analysis → Portfolio risk overview VALUE AT RISK (95%) $4.8M ↑ $2.1M → $4.8M (+128%) BREACH DELTA EXPOSURE $12.3M ↑ 23% vs. yesterday SHARPE RATIO 1.42 Within target range MARGIN UTILIZATION 78% Approaching 80% warning FinPilot risk analysis Auto-generated · 3 min ago What happened Crude oil futures spiked 4.2% following OPEC production cut announcement. Your energy sector allocation (32% of portfolio) drove VaR above the $4M limit. Recommended actions Option 1: Reduce energy exposure by 15% Sell $1.8M in XLE positions. Estimated VaR reduction to $3.6M. Cost: ~$2.4k in spreads. Execute Option 2: Hedge with crude oil puts Buy CLF 72P contracts. Covers 80% of exposure. Premium: $34k. VaR reduction to $3.2M. Review Option 3: Accept and monitor Set enhanced monitoring. Alert if VaR exceeds $5.5M or energy allocation exceeds 35%. Accept Sources: Risk_Limits_Policy.pdf §2.3 · Market data (real-time) · Portfolio analytics engine SECTOR EXPOSURE BREAKDOWN Energy 32% Technology 25% Financials 18% Healthcare 12% Consumer 8% Other 5%
Proactive alert banner (top) triggers automatically when VaR breaches threshold. FinPilot analysis panel (left) provides context, cause, and 3 actionable options with one-click execution. Sector breakdown (right) for immediate visual context.
Why three options, not one recommendation

Early designs had the agent recommend a single "best" action. Rejected because risk management is judgment-heavy — the "best" option depends on market outlook, client relationships, and risk appetite that the agent can't fully assess. Three ranked options let the risk manager apply their expertise to the agent's analysis. The agent does the math; the human chooses the strategy.

U-03

Standalone agent workspace

A dedicated, full-screen environment for deep multi-step analysis. Used when embedded panels aren't enough — complex report generation, multi-system queries, and iterative research with the agent.

FinPilot Workspace + New conversation RECENT CONVERSATIONS Q4 portfolio stress test Client onboarding analysis Compliance audit prep Market volatility report Run a full stress test on our Q4 portfolio against a 2008-style financial crisis scenario. ↗ RiskEngine.stress_test() ↗ CRM.get_portfolio_holdings() ↗ MarketData.get_historical(2008_crisis) 2008-style stress test results Under this scenario, your portfolio would experience: Portfolio loss: -$18.4M (-34.2%) Worst sector: Financials (-52.8%) Best sector: Utilities (-8.3%) VaR would peak at $8.2M (3.4x current limit) Margin calls triggered on 4 of 12 accounts Portfolio value under stress (simulated 12mo) Sources: Historical crisis data · Risk engine · Current holdings Generate a PDF report I can share with the team ✓ Report generated Ask a follow-up or start a new analysis... CONTEXT PANEL Tools used in this session Risk Engine · CRM · Market Data Sources referenced Risk_Limits_Policy.pdf Stress_Testing_Guidelines_v2.docx Historical_Crisis_Data_2008.csv Generated artifacts 📄 Q4_Stress_Test_Report.pdf Download · Share Session confidence 92% overall · High confidence
Full-screen workspace for complex analysis. Left sidebar shows conversation history with color-coded personas. Center is the deep chat. Right panel tracks tools used, sources referenced, generated artifacts, and session confidence.
Principles

Five rules.

01

Embedded, not separate

Lives inside existing workflows. Never pulls users away.

02

Transparent by default

Show sources, reasoning, confidence. If you can't cite it, don't say it.

03

Progressive autonomy

Suggestions first. Earn trust. Then automate.

04

Role-aware

Same brain, different behavior. Different error tolerances.

05

Fail loudly

Doesn't know? Says so. Tool fails? Shows why. No silent failures.

Validation

What I'd test next.

Key hypotheses.

H1

Do users trust confidence scores? Show same response with/without confidence to compliance analysts. Measure manual verification rate.

H2

Will compliance teams accept AI-generated logs? Present auto-generated investigation records. Ask: "Would you submit this to auditors?"

H3

Is the 80% confidence threshold right? Run 200 queries at varying confidence. Calibrate against analyst error tolerance.

Testing plan.

Week 1–2

Prototype compliance flow. Test with 5 analysts. Measure: completion time, verification steps, trust (1-5).

Week 3–6

Builder prototype. Can 3 ops admins identify and fix a wrong answer within 10 minutes?

Month 2–3

A/B test: citations-only vs citations + confidence vs citations + reasoning trace. Lowest manual verification wins.

Open question

Before/after estimates (~2h → ~12m) are from workflow simulation. Live numbers would differ — likely better for routine, worse for edge cases.

Design the system,
not just the screen.

Thank you

Designed by Dheeraj Allu.

Research, architecture, interaction design, trust engineering — and the product decisions that make AI systems safe enough for financial professionals to rely on.

I'm looking for roles where I can design AI products at this depth.

UX / Product Design · 6+ years enterprise platforms