October 1, 2026 · GRAFOLIO
Why Investment-Desk AI Agents Stall in Pilot
Why Investment-Desk AI Agents Stall in Pilot
Process maps, agreed definitions, source trails and audit logs decide whether a pilot reaches production — not the model.
Pramod Thaz · Grafolio
Most investment firms have tried AI. Few have put agents into production on the desk.
The demo lands well. An analyst pastes a rent roll or a reporting pack into a chat window and gets a tidy answer. Leadership asks for a pilot. Six months later the same team is still copying numbers by hand, and the pilot is still a pilot. The model is rarely the problem. The process, the definitions, the source trail and the control environment are.
This is the pattern we see when firms try to move from a promising GenAI trial to agents that do real work for asset managers, pensions, REITs, and real estate and private credit managers.
The process was never written down
An agent can only automate a process someone has mapped. On most desks, the process lives in people's heads, email threads and spreadsheet tabs.
Who pulls the T-12? Who reconciles it to the rent roll? Who decides what counts as NOI for this fund? Who signs the variance note? Who handles a statement with no NOI line?
Until those steps, owners and exception paths are written down, an agent has nowhere safe to operate. Skip the map and you get impressive demos and brittle workflows. The first real exception — a missing line, a fund that defines occupancy differently, a workbook the agent was never shown — stops the agent cold, or worse, encourages it to invent a fill-in.
Map first. Mark each step as automate, assist, or keep human. That map becomes the agent's specification and the control document operations, technology and compliance can sign off on.
Definitions disagree by fund
Ask three portfolio managers for trailing-twelve-month NOI and you may get three answers. One includes other income. One annualizes incomplete months. One treats model units as occupied.
Humans negotiate those differences in the hallway. Agents cannot. If the firm has not written down how each key metric is calculated — and which version is approved for reporting — the agent will either refuse to answer or produce a number nobody can defend in committee.
Write the definition once, in plain language. Apply it the same way every time. When the document does not support it — no NOI line, for example — say so. Do not estimate quietly.
There is no source trail
Investment committees, auditors, consultants and regulators ask the same question: where did this number come from?
A chat answer without a citation to the source line, cell or system record is not production-ready. It may help research. It is not something you put in an IC memo, an LP pack or a board book.
Every figure for desk use should be traceable: property, period, calculation, and the document or workbook cell behind it. If the figure is not in the source, say it is missing — do not compose a substitute. Cite or flag: that habit separates a tool reviewers will trust from one that never leaves pilot.
Compliance asks for the logs
Supervisors expect firms to show their work. FINRA's 2026 Regulatory Oversight Report describes effective practices for generative AI — not new rules — that include storing prompt and output logs, tracking which model version was used and when, and keeping a human in the loop for review. For AI agents, it adds monitoring system access, tracking agent actions and limiting what agents can do. The SEC's examination program continues to focus on how advisers use AI, including whether disclosures match practice and whether policies and procedures are adequate. The NIST AI Risk Management Framework is becoming the shared vocabulary for AI governance across Govern, Map, Measure and Manage.
There is also a gap. In April 2026, the federal banking agencies' revised model risk guidance, SR 26-2, superseded SR 11-7, which many investment firms had borrowed as their model-governance template. SR 26-2 explicitly places generative and agentic AI outside its scope. It defers to each organization's own risk management and governance practices. Firms now need to design those controls themselves.
A pilot without prompt, output and tool-call logs; without model and prompt version tracking; without scoped permissions and human checkpoints where policy requires them cannot survive a compliance review. Building governance after the agent is live is how pilots stall. Build it with the agent, or the agent stays in the sandbox.
Excel is still the system of record
On underwriting and asset-management desks, the approved workbook is often the system of record. Cap rates, NOI bridges, debt schedules and sensitivity tables live in Excel because that is where the formulas are reviewed, versioned and signed off.
Agents that recompute those figures in a chat window — or rebuild the model in a new system — create a second set of numbers the committee did not approve. Pilots that ignore the workbook usually fail the first time a reviewer asks, "Does this match our model?"
The production pattern is different: the agent drives the approved workbook, leaves formulas as formulas, and cites the cells. Model changes stay under human review. Replacing Excel math with AI math is why many CRE and private-credit pilots never leave the lab.
Exceptions get invented
Documents do not always reconcile. Totals do not always tie. A property in the demo portfolio — call it Vesper Lofts — may have an operating statement with no NOI line. A rent roll may show cash-balance rows that do not sum.
A careful analyst flags the gap. An unconstrained agent fills it. That is the failure mode compliance teams fear most: a fluent answer that looks complete and is quietly wrong.
Production agents need an exception queue. When data is missing or does not reconcile, the agent says so and holds the item for a person. Filling the gap is not helpfulness. It is inventing.
What actually gets an agent to production
The firms that move past pilot do five things before they scale the model:
- Map the workflow — inputs, systems, decisions, approvals, exceptions and owners.
- Agree the definitions — firm metrics in plain language, approved before the agent uses them.
- Require a source trail — every figure cited to a document line, cell or system record.
- Install the logs and checkpoints — prompts, outputs, model versions, permissions, human review where policy requires it.
- Leave exceptions visible — missing or unreconciled data goes to a queue, not into the answer.
None of that is glamorous. All of it is what turns a demo into something an investment committee, an auditor and an examiner can rely on.
Start with one workflow
Pick one process that consumes too much analyst time each quarter: a morning variance note, a lease-expiration chase list, a first-pass IC memo when a deal arrives. Map it. Write the definitions. Decide what the agent may read, write and send. Put logging and review in place before you widen the scope.
That is how agents leave pilot.
Grafolio designs, deploys and governs autonomous AI agents for institutional investment management. Meridian by Grafolio is our AI-first portfolio management platform for commercial real estate — the reference experience we use when we build agents on real CRE documents and Excel models. The guardrails we recommend to clients run in our own product first.
Technology and process advisory. Not investment, legal or regulatory advice. Illustrative examples only; no client results are implied. FINRA materials cited are effective practices described in the 2026 Regulatory Oversight Report, not new rules.