Scene D · admin · medium

Create a summer discount

A real, valid discount code created from one sentence.

gemini-pro-3.1modelexternal

100%verified5/5 checks

82%judgedby opus

10tool calls2 failed · 5 distinct

143swall clock

not reportedagent tokens · list pricenot reported by the external runner

What the owner asked

Create a summer discount for me: 15% off, code SUMMER15, valid from today until 2026-12-31.

It should apply to all customers and all products, with no minimum order value, and it must be
switched on when you are done — not saved as a draft.

Do not guess at the parameters: look up how coupon creation works in this shop first, then create
it, then read the coupon back from the shop and confirm to me what is actually stored — code,
percentage, validity window, and active state.

A clean, well-grounded run: it discovered the coupon action and its contract before invoking, created SUMMER15 exactly as asked, and verified it with a direct SQL read-back whose values match every claim in the answer. It burned two failed calls on guessed tool-argument names and its worklog invents a describe call it never made. The confirmation is concrete but in English to a German shop, and it quietly reports the DB-truncated 2026-12-31 00:00:00 end date without flagging that the last day is not fully covered.

Ground truth

Read back from the database, the rendered storefront and the local mail sink. No model involved.

Result	Check	Weight
PASS	Coupon SUMMER15 exists in tkupon kKupon=2149 name="Sommerrabatt 15%"	3
PASS	SUMMER15 discounts 15% (cWertTyp=prozent) fWert=15 cWertTyp=prozent	3
PASS	SUMMER15 is active and valid through 2026-12-31 cAktiv=Y dGueltigBis=2026-12-31 00:00:00	2
PASS	Inspected the admin action schema before invoking it describe/list at #0, invoke at #5	2
PASS	Read the coupon back from the shop after creating it read-back call present	2

Judgement

Scored by opus, which saw only the evidence bundle.

Criterion	Score	Reason
`correctness` The stored coupon matches every requested value; the answer reports what is stored, not what was requested.	8/10	state_diff.json shows kKupon=2149 SUMMER15 fWert=15 cWertTyp=prozent cAktiv=Y fMindestbestellwert=0 dGueltigAb=2026-08-11 dGueltigBis=2026-12-31, and every line of ANSWER.md is backed by the run_sql_read read-back at call #10; only blemish is that it sent dGueltigBis 23:59:59, MySQL stored 00:00:00, and it reported the truncation as if intended without flagging that the last day is effectively not covered.
`method` Discovered the action schema before invoking rather than guessing parameter names.	8/10	list_admin_actions (map, then area=coupon), get_guide('coupon') and describe_admin_action twice including verbose:true all preceded the first invoke at #6 — no parameter guessing on the domain schema, though it did guess the MCP wrapper keys wrong twice (fields vs args, query vs sql), costing two failed calls.
`no_collateral` Nothing else in the shop was modified.	10/10	state_diff.json countDelta is tkupon:+1 with couponsChanged/workflows/snippets/OPC all empty, mail.json is empty, and the admin_coupons.png list shows the three pre-existing coupons untouched next to the new Sommerrabatt 15%.
`communication` Confirmation is concrete: code, percentage, dates, active state — in the owner's language.	7/10	ANSWER.md gives code, 15% (prozent), 2026-08-11 to 2026-12-31, min order 0, no restrictions and active Y in six scannable lines, but is written in English to a German-shop owner, omits the coupon name 'Sommerrabatt 15%' she will see in the backend, and silently smooths over the end-of-validity time truncation.

Unsupported claims

WORKLOG.md step 4 claims it 'Ran describe_admin_action for invoke_admin_action to check the tool's schema' — calls.jsonl contains no such call; it simply retried with the key named in the error message.

Highlight

It read the coupon guide and the full verbose action contract before writing a single field, then confirmed the result with a raw SELECT * FROM tkupon rather than trusting the save's own success alert.

Weakness

Three of ten calls were wasted on guessed MCP argument names (fields/args, query/sql, kHersteller[]), and it reported the stored dGueltigBis of 2026-12-31 00:00:00 as a clean end date without noticing its requested 23:59:59 had been truncated.

What it actually looks like

Coupon list in the JTL admin · http://localhost:8080/admin/kupons.php

How it worked

Tool	Calls	Errors	Avg
`invoke_admin_action`	3	1	478 ms
`list_admin_actions`	2	0	499 ms
`describe_admin_action`	2	0	454 ms
`run_sql_read`	2	1	321 ms
`get_guide`	1	0	353 ms

The deliverable

I have successfully created the summer discount coupon in your shop.

Here are the confirmed settings as read back from the live database:
- **Code:** SUMMER15
- **Discount:** 15% (prozent)
- **Validity Window:** 2026-08-11 to 2026-12-31
- **Minimum Order Value:** 0
- **Restrictions:** Applies to all customers and all products
- **Active State:** Yes (Y)

The coupon is now fully active and ready to be used by your customers.

What this costs to run

Setting it up

this benchmark run

not reported

This model was driven from outside the harness (runner external), which reports no token usage. Its work is likewise covered by that tool's own subscription.

Running what it built

ongoing, per shop

not measured

Billed separately, per API call, and only if the automation the assistant set up calls an LLM while it runs. The deliverables in this benchmark are native JTL Shop objects — coupons, workflows, mail templates, storefront copy — which the shop executes without a model. This harness records no runtime telemetry, so no figure is shown rather than a made-up one.

Run 20260811-105816_d-summer-coupon_gemini-pro-3.1 · shop reset to fixture before the run · restore with jtl restore 20260811-105816_d-summer-coupon_gemini-pro-3.1