Finance Helpdesk AI Automation

Business context

A finance helpdesk receives a steady stream of tickets — supplier statements, payment queries, invoice disputes, banking changes and more. Tickets only get resolved quickly when they reach the right agent with the right classification. The goal was to build an agent that could reliably identify certain groups of ticket requests, then quickly and accurately assign each one to the correct agent with the correct classification, so it could be resolved without delay.

Challenge

The first target was supplier statement tickets. To handle them, the AI agent had to:

• pick out a supplier statement ticket from among all the other tickets, by reading through its contents accurately rather than relying on the subject line;
• assign the correct classification to the ticket;
• assign it to the agent who would work on it — without making errors.

Language models are good at reading unstructured requests but are not deterministic by default. A workflow that touches finance processes also has to behave the same way when a call is retried, when an external API fails, or when the model returns something unexpected.

My contribution

• Extended the existing baseline helpdesk workflow; I did not build the original intake system.
• Built the structured LLM classification step, so the agent’s output always conforms to a defined schema.
• Implemented the assignment logic that routes each classified ticket to the correct agent.
• Added the internal audit note that is written to every assigned ticket.
• Added retry handling and idempotent processing so repeated events never create duplicate assignments.
• Integrated with the helpdesk APIs through Azure Functions in Python, and deployed the solution.
• Wrote an offline test suite: 88 unit tests in 15 test classes.

Investigation and methodology

The design began with failure modes rather than the happy path: what happens if the model returns invalid output, if a ticket looks like a supplier statement but is not one, if a downstream API times out, or if the same ticket is delivered twice? Each became a requirement and then a test.

Solution design and implementation

Each incoming ticket is read and classified into a fixed set of categories with a structured response, which is validated before anything changes. A ticket the agent cannot classify confidently is left for a person instead of being guessed. Once a ticket is classified, it is assigned to the correct agent.

Every assignment then adds an internal note to the ticket, visible only to the agents working on the system. The note records how the ticket was identified, how it was classified and why it went to that agent. Together, these notes form an audit trail: anyone can check the agent’s reasoning on a single ticket, and errors can be caught and corrected quickly.

Operations that change a ticket are keyed so they can be safely retried, and the test suite runs offline with external services replaced by test doubles, so it can be run repeatedly without touching live systems.

Results and business value

• Deployed: the agent runs on the live helpdesk, identifying, classifying and assigning its target ticket types.
• Faster routing: targeted tickets reach the correct agent with the correct classification as they arrive, rather than waiting for manual triage.
• Auditability: every assignment carries an internal note explaining the decision, giving the team a trail to review and catch errors.
• Reliability: behaviour is covered by 88 offline unit tests across 15 classes, including duplicate deliveries, API failures and invalid model output.

Technical details

• Runtime: Python on Azure Functions, integrated with the helpdesk through its REST APIs.
• Classification: LLM output constrained to a schema and validated; schema failures handled explicitly.
• Audit note: an agent-only internal note is posted with each assignment, recording identification, classification and assignee.
• Reliability: retries for transient failures; idempotency keys so duplicate deliveries do not repeat side effects.
• Testing: 88 offline unit tests in 15 classes, with external APIs and model calls stubbed.

Lessons, constraints and improvement opportunities

• In finance workflows, predictability matters more than cleverness.
• An audit note on every decision makes an AI agent easier to trust, review and correct.
• Natural next steps are to extend the agent to further ticket groups and to track its accuracy against agent corrections over time.