webkd
University of Utah

Teaching Agents in Their Own Style: Black-Box Teacher Distillation for Web Agents

Md Farhan IshmamKenneth Marino

Kahlert School of Computing · University of Utah

Paper Code
ANATOMY OF A SMALL CORRECTION

The student’s task

Find the price of the most expensive bookshelf.

BEFOREfill('15', 'bookshelf')
AFTERfill('31', 'bookshelf')
Right idea.
  Wrong element.
FIG. 01   /   ILLUSTRATIVE CORRECTION, ADAPTED FROM THE PAPER

Teaching Agents in Their Own Style:
Black-Box Teacher Distillation for Web Agents

01 Overview

Style-preserving
teacher distillation

WEBKD trains web agents using text-only corrections from a black-box teacher. The teacher identifies the first error in a student trajectory and edits the remaining responses at their original contexts.

Corrections preserve the student’s reasoning style while changing incorrect decisions.

Example correction
OBSERVATION
[15] link ‘Simple Shop’
[31] textbox ‘Search’
[47] button ‘Search’
Think

I need to search for “bookshelf”. The search box is 31, so I’ll type there.

Plan

1. Type bookshelf → 2. Search → 3. Compare prices.

Actionfill('31', 'bookshelf')

The action is corrected, while the short, direct phrasing stays close to the student’s style.

ILLUSTRATIVE EXAMPLE

Adapted from Figure 1; text simplified for illustration.

02 The method

Method

Training procedure and objective
Method + loss
Speed 1×

Fixed-context editing, action-informative weights, and the role-asymmetric training objective.

View the static diagram ↗
Read the animation as text
  1. A behavior-cloned student makes a browser attempt, shown as four original contexts and their responses.
  2. The teacher identifies the first erroneous step. Responses before this point are kept verbatim.
  3. Each response from the error onward is independently corrected at its original student context, in the student’s style. Edited actions do not generate a new teacher trajectory.
  4. The perplexity premium measures extra mean per-token surprisal under the frozen student. In the schematic comparison, original student text has a short bar, a raw teacher edit has a long bar, and a style-conditioned edit has a short bar. Bar length, not color, represents surprisal.
  5. For each reasoning span, the frozen seed scores the corrected action with and without that span. The difference is its action contribution.
  6. More informative spans receive more training weight; less informative spans are downweighted, not discarded. When a step’s contributions are nearly tied, their weights move toward a shared midpoint. Corrected action tokens retain full imitation weight.
  7. The objective adds a weighted mean of cross-entropies to a normalized pointwise KL surrogate. The role mask is 1 on think, plan, and memory, and 10⁻⁶ on action and step.
  8. The anchor keeps reasoning close to the seed without freezing it. The action is almost unanchored so the student can adopt corrected decisions.
P1 / INITIALIZE

Behavior cloning

Behavior-clone successful teacher demonstrations to give the student its initial web skills.

P2 / EXPERIENCE

Student rollouts

Collect its own trajectories. Failed attempts become edit targets; successes provide examples of its style.

P3 / LOCALIZE

Error localization

The teacher identifies the earliest error that sends the task off course. Earlier correct steps stay intact.

P4 / CORRECT

Style-preserving edits

Correct each response in the faulty suffix at the student’s original context, using its writing as a guide.

Unpack the contribution scores, weight map, and anchor
Eq. 5 / Training objective

Weighted imitation and rationale anchoring

L WD equals the sum of w i times CE i divided by the sum of w i, plus beta times the sum of gamma i times kl i divided by the sum of gamma i.
Weighted imitation
Emphasize reasoning spans that support the corrected action. Action and step tokens receive full imitation weight.
Rationale anchor
Limit drift from the frozen seed on reasoning tokens. Action and step tokens are almost unanchored.

Each term is normalized by its own weight sum; β controls the anchor strength.

Eq. 2 / The mismatch

Perplexity premium

The premium is teacher-supervision mean surprisal minus student-response mean surprisal under the frozen seed.

This is a difference in log perplexities, not raw perplexities. Uniform training can spend its largest loss on unfamiliar wording instead of the useful decision.

Eq. 4 / Credit assignment

Leave-one-out action contribution

Delta s is the frozen student's log probability of the corrected action with the span minus without the span.

Both scores use the frozen behavior-cloned student. A span earns credit when its presence increases the probability of the corrected action. Negative contributions stay in the ranking.

Eq. 10 / From rank to weight

Within-step rank weights

Rank-derived weights are contracted toward the midpoint according to contribution spread, then clipped between phi and one.

Ranks are normalized within each step; ties share their average rank. With φ = 0.1 and τ = 0.05 nats, identical contributions yield 0.55. Action and step CE weights are pinned to 1.

Eq. 5 / How far to move

Role-asymmetric anchoring

The pointwise penalty is exp d minus d minus one; d is the seed log probability minus the current policy log probability of a target token.

Here pᵢ is the current student’s target-token probability and qᵢ is the frozen seed’s. On corpus target tokens, use γ = 1 for think/plan/memory and γ = 10⁻⁶ elsewhere. This pointwise surrogate is not an exact full-vocabulary KL. β = 0.03 in the reported configuration.

Notation: πBC is the frozen behavior-cloned seed, πθ is the student being trained. Explore the formulas and a numerical example ↗ · Appendix B.4–B.5 ↗

03 Evaluation

Experimental results

TimeWarp and OnlineMind2Web evaluation. Select a student model to compare methods.

TimeWarp v6

Evaluation on the interface version seen during training.

Table 1 · p. 7 ↗
TimeWarp v6

See Table 1 in the linked preprint for the reported values.

Success rate (%)

Baseline WEBKD ±1 standard error

Mean ± standard error over three runs. Selected baselines; the complete table is available below.

All methods and exact values

TimeWarp v1–v5

Evaluation across five interface versions held out from training.

Table 1 · p. 7 ↗
TimeWarp v1

See Table 1 in the linked preprint for the reported values.

Success rate (%)

Baseline WEBKD ±1 standard error

Mean ± standard error over three runs. The table reports all five unseen versions for the selected student.

All methods and all five versions

OnlineMind2Web

Live-web evaluation of Qwen 3.5 4B and Qwen 3.5 9B.

Original paper figure ↗
OnlineMind2Web · Overall0–25% scale, fixed across students and difficulties

Success rates are provided in the supplied results table.

Success rate (%)

Baseline WEBKD ±1 standard error

Success rate (%) ± standard error over three runs. Difficulty views report success within each subset: 61 easy, 106 medium, and 61 hard tasks (228 total).

All methods and task difficulties

04 Paper and citation

Teaching Agents in Their Own Style: Black-Box Teacher Distillation for Web Agents

Md Farhan Ishmam · Kenneth Marino

Kahlert School of Computing, University of Utah

Read on arXiv
BibTeX · arXiv
@misc{ishmam2026webkd,
  title={Teaching Agents in their Own Style:
    Blackbox Teacher Distillation for Web Agents},
  author={Ishmam, Md Farhan and Marino, Kenneth},
  year={2026},
  eprint={2610.19834},
  archivePrefix={arXiv},
  primaryClass={cs.AI},
  url={https://arxiv.org/abs/2610.19834}
}

arXiv:2610.19834 [cs.AI] · 2026