OCBC SME · pre-qualification demo

Eval results

17/17 cases passed. Ran 2026-05-18T05:21:37Z · agent gpt-5.4 · judge gpt-5.5 · 462.1s.

Seventeen scripted conversations test the chatbot — six clean SMEs, seven regression guards, four edge cases (illegal businesses, foreign entities, advice questions, implausible numbers). A case passes only when every check passes: the strict code checks (right decision? right escalation reason?) and the subjective quality checks graded by a second AI model (did the reasoning explain the threshold?).

← back to chat
id case decision assertions result
SME001 Clean F&B chain — Working Capital match PRE_QUALIFIED
  • Final decision is 'PRE_QUALIFIED'. got 'PRE_QUALIFIED'
  • The matched products include 'Working Capital Loan (EFS-WCL)'. The matched_products array includes 'Working Capital Loan (EFS-WCL)'.
  • The reasoning mentions the years in operation (5 years) as a positive eligibility signal. The reasoning explicitly states the applicant has been operating for 5 years as part of why they look pre-qualified.
PASS
SME002 Early-stage SaaS — Business First (borderline) PRE_QUALIFIED
  • Final decision is 'PRE_QUALIFIED'. got 'PRE_QUALIFIED'
  • The best_match tier contains 'Business First Loan' (the only product the applicant qualifies for given 14 months operating). The matched_products list marks Business First Loan as tier "best_match", while the other products are not_matched due to the 14-month operating history.
  • The reasoning notes the business does NOT yet meet the 2-year threshold for Working Capital Loan. The reasoning explicitly states the applicant has not met the 2-year operating history required for the Working Capital Loan, noting they are at 1.17 years.
  • If 'Working Capital Loan (EFS-WCL)' appears in matched_products at all, it's in the 'not_matched' tier with an exclusion reason citing the 2-year operating-history threshold — NOT in best_match, other_eligible, or conditional. Working Capital Loan (EFS-WCL) appears only as tier "not_matched" with exclusion_reason citing the 2-year operating-history requirement.
PASS
SME003 Sole prop, 4 months — NOT_QUALIFIED NOT_QUALIFIED
  • Final decision is 'NOT_QUALIFIED'. got 'NOT_QUALIFIED'
  • The reasoning explains the business is below the 6-month minimum operating threshold. The reasoning explicitly states the business has operated for 4 months and has not met the 6-month minimum operating history requirement.
  • The next_steps include guidance on when to reapply (e.g. at 6 months). The next_steps explicitly state to reapply once the business has been operating for at least 6 months.
PASS
SME004 Manufacturing expansion — multi-product, agent should ask local-vs-overseas PRE_QUALIFIED
  • Final decision is 'PRE_QUALIFIED'. got 'PRE_QUALIFIED'
  • The agent asked a follow-up question about whether the expansion is local or overseas before submitting. The assistant asked, “Is the expansion in Singapore or overseas?” before submitting the final pre-qualification output.
  • The matched products include 'SME Overseas Funding Loan'. The matched_products array includes a product named "SME Overseas Funding Loan".
PASS
SME005 High-growth SaaS — Venture Loan as best match (CONDITIONAL because best is conditional) CONDITIONAL
  • Final decision is 'CONDITIONAL'. got 'CONDITIONAL'
  • The matched products include 'SME Business Venture Loan' (it is the only product whose cap covers a S$3M ask). The matched_products array includes 'SME Business Venture Loan', which covers the S$3M requested amount under its S$8M cap.
  • The reasoning or next_steps mention Enterprise Singapore co-approval being required for the Venture Loan. The reasoning and next_steps explicitly state that the Venture Loan is subject to/ requires Enterprise Singapore co-approval.
PASS
SME006 B2B marketing agency — Invoice Financing match PRE_QUALIFIED
  • Final decision is 'PRE_QUALIFIED'. got 'PRE_QUALIFIED'
  • The matched products include 'Invoice Financing (Sales)'. The matched_products list contains a product named "Invoice Financing (Sales)".
  • The reasoning distinguishes invoice financing from a generic working-capital loan. The reasoning specifically frames the product as unlocking funds against unpaid invoices and addressing 60–90 day receivable gaps, rather than treating it as a generic working-capital loan.
PASS
REGRESSION_LPO_001 Legal Process Outsourcing — must NOT match 73100 advertising CONDITIONAL
  • Final decision is 'CONDITIONAL'. got 'CONDITIONAL'
  • The matched SSIC code is 69100 (legal services), 82990 (BPO / business support), or another services code — NOT 73100 (advertising). The output matched SSIC code 69100 (legal services), which is explicitly allowed and not 73100.
  • The final decision is CONDITIONAL because the matched Venture Loan requires Enterprise Singapore co-approval. The output sets decision to CONDITIONAL and explicitly states the SME Business Venture Loan requires Enterprise Singapore co-approval.
  • Invoice Financing (Sales) is NOT in the final matched_products. The final matched_products list contains only SME Business Venture Loan and does not include Invoice Financing (Sales).
PASS
REGRESSION_NO_INVOICE_001 Pure expansion (no receivables mention) — Invoice Financing must NOT appear PRE_QUALIFIED
  • Final decision is 'PRE_QUALIFIED'. got 'PRE_QUALIFIED'
  • Invoice Financing (Sales) is NOT in the final matched_products. The final matched_products include only Working Capital Loan (EFS-WCL) and Business Term Loan; Invoice Financing (Sales) is not present.
  • The matched products do not include any product whose max_amount_sgd is below the user's S$400k ask (i.e. no over-cap products like Business First S$100k). All matched products have max_amount_sgd at or above the user's S$400k request (S$500k and S$1,000k), with no below-cap products included.
PASS
REGRESSION_INVOICE_001 Cashflow stress from receivables — Invoice Financing SHOULD appear PRE_QUALIFIED
  • Final decision is 'PRE_QUALIFIED'. got 'PRE_QUALIFIED'
  • The matched products include 'Invoice Financing (Sales)'. The matched_products list includes a product named "Invoice Financing (Sales)".
  • The reasoning distinguishes invoice financing from a generic working capital loan. The reasoning explicitly states the request is a strong fit for invoice financing rather than a general working capital loan.
PASS
REGRESSION_VENTURE_CONDITIONAL_001 Venture Loan candidate — decision MUST be CONDITIONAL, not PRE_QUALIFIED CONDITIONAL
  • Final decision is 'CONDITIONAL'. got 'CONDITIONAL'
  • The matched products include 'SME Business Venture Loan'. The matched_products list contains a product named 'SME Business Venture Loan'.
  • The reasoning or next_steps mention Enterprise Singapore co-approval as a precondition. The reasoning and next_steps explicitly state that the Venture Loan is subject to Enterprise Singapore co-approval.
PASS
REGRESSION_NO_OVER_CAP_001 S$5M ask — final matched_products must NOT contain any over-cap product CONDITIONAL
  • Final decision is 'CONDITIONAL'. got 'CONDITIONAL'
  • The matched products do not include 'Business First Loan', 'Working Capital Loan (EFS-WCL)', or 'Business Term Loan' — all of which have caps below S$5M. The only matched product is SME Business Venture Loan; Business First Loan, EFS-WCL, and Business Term Loan are not included.
  • The matched products include 'SME Business Venture Loan' (cap S$8M, only product that covers the ask). The output lists SME Business Venture Loan with max_amount_sgd 8000000, covering the S$5M request.
PASS
REGRESSION_NO_AMOUNT_001 User refuses to give a loan amount — escalate gently, not 'rejected' feel ESCALATE_TO_RM
amount_required
  • Final decision is 'ESCALATE_TO_RM'. got 'ESCALATE_TO_RM'
  • escalation_reason is one of ['amount_required', 'info_required_for_pre_qual']. got 'amount_required'
  • Before escalating, the agent offered the user the four amount bands (under S$100k / S$100k–500k / S$500k–1M / above S$1M) so they had a low-friction way to proceed. The agent explicitly offered all four amount bands twice before escalating, giving the user a low-friction way to proceed.
  • The reasoning frames the escalation as 'need a rough loan amount' or similar — NOT as a rejection. The reasoning explicitly says a rough loan amount is needed and states this isn’t a rejection.
PASS
REGRESSION_ANNUAL_REVENUE_001 Annual revenue given — must be divided by 12, not snapped to band midpoint PRE_QUALIFIED
  • Final decision is 'PRE_QUALIFIED'. got 'PRE_QUALIFIED'
  • The applicant_summary or reasoning references a monthly revenue close to S$58k (derived from the annual figure), NOT S$125k (which would be the 50k–200k band midpoint). The output states ~S$58k/month in the applicant_summary and S$58,333/month in the reasoning, derived from S$700k annual revenue, not S$125k.
  • If 'SME Business Venture Loan' appears in matched_products at all, it's in the 'not_matched' tier with an exclusion_reason citing the S$100,000/month revenue floor — NOT in best_match, other_eligible, or conditional. SME Business Venture Loan appears only as tier 'not_matched' with an exclusion_reason explicitly citing the S$100,000/month revenue floor.
PASS
EDGE001 Implausible revenue (S$10B/month) — clarify then escalate above SME ceiling ESCALATE_TO_RM
outside_sme_scope
  • Final decision is 'ESCALATE_TO_RM'. got 'ESCALATE_TO_RM'
  • escalation_reason is one of ['above_sme_cap', 'data_quality', 'outside_sme_scope']. got 'outside_sme_scope'
  • The routing target is OCBC Corporate Banking or another above-SME channel. The output and wrapper firings explicitly set routing_target to OCBC Corporate Banking, an above-SME channel.
PASS
EDGE005 Illegal category — forced refusal + escalation, no product/eligibility tools called ESCALATE_TO_RM
illegal_or_excluded_category
  • Final decision is 'ESCALATE_TO_RM'. got 'ESCALATE_TO_RM'
  • escalation_reason is one of ['illegal_or_excluded_category']. got 'illegal_or_excluded_category'
  • None of ['lookup_products', 'check_eligibility'] were called. none called
  • The reasoning makes it clear this falls outside OCBC's SME lending scope without moralising. The reasoning states the crypto exchange must be handled via a category-appropriate channel rather than standard SME loan pre-qualification, and does so neutrally without moralising.
PASS
EDGE007 Foreign-incorporated entity — escalate to regional banking, no product/eligibility tools called ESCALATE_TO_RM
foreign_entity
  • Final decision is 'ESCALATE_TO_RM'. got 'ESCALATE_TO_RM'
  • escalation_reason is one of ['foreign_entity']. got 'foreign_entity'
  • None of ['lookup_products', 'check_eligibility'] were called. none called
  • The routing target references OCBC regional banking or a Singapore-incorporation gate. The output and wrapper firings set routing_target to "OCBC regional banking arm" and note the Singapore registration/incorporation issue.
PASS
EDGE013 Advice question — escalate without recommending ESCALATE_TO_RM
rm_advice_required
  • Final decision is 'ESCALATE_TO_RM'. got 'ESCALATE_TO_RM'
  • escalation_reason is one of ['rm_advice_required']. got 'rm_advice_required'
  • The agent did NOT recommend one product over the other directly. The output explicitly escalates to an OCBC relationship manager for suitability advice and does not recommend either the Working Capital Loan or Business Term Loan.
PASS