Hundreds of test applicants per credit change, up from 15 to 20 by hand

The team configured credit decisioning engines for banks across cards, auto, personal and SME lending. Every change request or new product had to be tested before it reached production. Testing a credit engine means sending it applicants and checking that each one is approved, referred or declined exactly as the requirements say.

There was no standard way to do this. Each analyst built test cases their own way. Some worked by hand in Excel. Some pasted the requirements into ChatGPT and other chat tools and re-prompted from scratch each time. Setting up took hours before a single case existed. With several projects running at once, an analyst could build 15 to 20 cases at most for one change, spread over days. Sometimes a test was one variable edited in a JSON file, which checks one thing at a time.

Stage	What it does	Status
1Read the requirements	Pulls rules from documents, PDFs and chat messages into one view	Live
2Generate the applicants	Builds synthetic applicants with expected outcomes	Live
3Load the test data	Writes them into the credit bureau's test database	Live
4Run and compare	Runs them through the engine and compares results	Live
5Trace the gaps	Finds the cause of every mismatch	Live

The requirements were never in one place. Some were in Word documents written by different analysts over several years. Some were in PDFs. Some were a quick message from a credit officer or the risk team in a group chat. Changes arrived out of order, and nothing was kept in one structured format.

The system is built for that. An AI-enabled Python program takes in all of these sources: documents, PDFs and chat messages. It extracts the structure: the input variables, the logic formulas, the decision trees, the scorecards and the scorecard matrices. It also reads the bank's risk acceptance criteria and business needs. These are the rules the bank set for who it will lend to. The result is one structured view of what the engine is supposed to do.

From that view, the system builds synthetic applicant profiles. Each is a made up applicant designed to take one specific path through the decision trees. Every profile carries its expected outcome: approve, refer or decline. Edge cases sit exactly on each threshold, such as an applicant a fraction over the debt burden ratio (DBR) cap. DBR is the share of monthly income that goes to debt repayments. The system covers every branch, not only the ones there is time to write by hand. A change that once got 15 to 20 cases now gets hundreds, generated in minutes. The analyst reviews the case set before it runs.

A credit engine pulls the bureau report for every applicant. So each synthetic applicant also needs a bureau record. The system writes the profiles in the bureau's own test data format and uploads them to its test database. To the engine, they look like real applicants with real credit histories.

The cases run against the engine in any of its three environments: development, user acceptance testing (UAT, where the bank signs off a change) and production. Postman, a tool that sends requests to an API, calls the engine for each applicant. The system then compares every actual decision with the expected one. After each change the full set runs again. This is regression testing: it proves the change did not break decisions that already worked.

Each mismatch is traced to its cause. Most were small and would have been easy to miss with a dozen cases. A threshold set slightly off. A broken link between two decision trees. A configuration that no longer matched the business requirements document (BRD). A BRD that no longer matched the bank's risk acceptance criteria.

One case stood out. The engine produced decisions the team could not explain at first. The tests traced it to a single semicolon in a logic formula that should not have been there. It made the engine skip an entire decision step. In production, it would likely have approved applicants who should have been declined. It was found and fixed before go-live.

• Hundreds of test cases per change, in minutes, instead of 15 to 20 at most, built over days.
• One standard process for every analyst, in place of each analyst's own method.
• At least a dozen change requests tested across development, UAT and production.
• Errors caught before go-live, including a stray semicolon that would have skipped a decision step.