Elephant Index
목록으로
AI Nativeness2분EN

What Should Teams Learn from Using AI?

Working with AI · Part 10

연재 「AI와 일하는 회사」 20번째 글

한국어로 읽기

AI drafts a refusal for a refund. An experienced agent checks the original sales channel and terms, which can change the answer.

Record why the decision changed

Keep the relevant facts in anonymized training cases. Have staff identify what they would check before showing the AI draft. Discuss assumptions behind disagreements and document the agreed approach. Review recurring corrections weekly and check whether people applied the new guidance in subsequent cases.

How a support study examined learning

Brynjolfsson, Li, and Raymond reported a 15% rise in resolutions per hour on average, with greater gains for less experienced agents. They used temporary outages to explore learning. A support team can use unaided training cases for a similar check without interrupting live service.

Brynjolfsson, Li, and Raymond in the Quarterly Journal of Economics

Turning corrections into team learning

Save the channel and policy in effect at purchase so colleagues can apply the reasoning to new cases.

Workflow and decision points
  1. Save draft and evidence
  2. Record why it changed
  3. Review cases and revise guidance
  4. Test without tool assistance
Accuracy on comparable support cases (%)
Before training: 11 of 20
55%
After training: 16 of 20
80%

Illustrative assessments with different cases of comparable difficulty and no AI assistance.

Before attributing a change to training, check case difficulty, scoring, and the participant mix. Review evidence people routinely missed.

Series index

목록으로