🔍 Read the full analysis: Amazon’s Advice For Engineers: Measure What AI Achieves on ThorstenMeyerAI.com
Get business pricing on office and shipping supplies
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
Amazon published new guidance for software engineers on October 27, saying AI usage metrics such as token counts should not be treated as performance targets. The memo says those measures can help teams spot patterns, but urges leaders to judge AI by its impact on customers, the business and engineering work.
Amazon published revised guidance for its software engineers on October 27, warning that token counts and other AI-use measures can distort behavior when treated as targets. The memo says engineers and managers should assess whether AI improves customer, business and team outcomes, rather than reward people for using AI more often.
The memo, published on Amazon’s own news site, was written by senior engineers and introduces a revised role guideline for software engineers. It was authored by Terk Terkowitz, a vice president and distinguished engineer in Amazon’s stores business. The guidance does not prescribe a fixed set of AI tools or usage quotas, arguing that technology changes too quickly for such definitions to remain useful.
Amazon identifies token usage, AI-tool adoption, code changes and reviews as possible indicators of activity, but says they are flawed when used as goals. Tokenmaxxing—using an AI system more simply to increase token consumption or rank—conflicts with the company’s stated emphasis on frugality, the memo says. Targeting lines of code can also conflict with the principle of simplifying work.
The memo does not say those indicators are useless. It says they currently correlate with important outputs, while stressing that correlation is not a reason to make them individual or team targets. Leaders are told to examine both low and high adoption: investigate what prevents some teams from benefiting, and check whether the heaviest users are producing better results. Engineers who find that AI does not help their work are asked to explain and share that finding.
Six months at Amazon
From token leaderboards to an outcomes memo
Amazon spent 2026 learning the same lesson twice: once with how its engineers use AI, once with its own AI models.
- May 2026Token leaderboards backfireFinancial Times (via The Decoder): staff give an internal agent tool pointless tasks to inflate token counts. Reported, anonymous sources.
- Late May 2026KiroRank “deprecated”Amazon spokesperson to Business Insider. SVP Dave Treadwell: “Please don’t use AI just for the sake of using AI.”
- 24 Jul 2026San Francisco AGI Lab site closesConfirmed by Amazon to GeekWire. Frontier model research continues under Pieter Abbeel.
- 28 Jul 2026Most flagship Nova models wound downBusiness Insider, people familiar: Premier, Omni, Reel, Canvas to “keep the lights on”. Amazon: models “remain one of the most important things we’re working on”.
- 14 / 30 Sep 2026Nova Sonic v1, Nova Reel, Nova Canvas reach end of lifeAWS Bedrock model lifecycle page. Requests to them now fail.
- 26 Oct 2026Anthropic files its S-1Amazon notes: $6.2 billion principal, 12% ownership cap that Amazon can waive. Anthropic commits at least $110 billion to Amazon through April 2036.
- 27 Oct 2026The memo: outcomes, not tokenmaxxingUsage and throughput metrics are “deeply flawed” as targets. “Tokenmaxxing is at odds with frugality.”
- 29 Oct 2026Amazon’s third-quarter resultsAnnounced by Amazon. Not yet published when this article went live.
Sources: Amazon engineering memo (27 Oct 2026); Business Insider (29 May and 28 Jul 2026); GeekWire (24 Jul 2026); The Decoder (12 May 2026, summarising the Financial Times); AWS Bedrock model lifecycle (read 28 Oct 2026); Anthropic S-1 (26 Oct 2026).
Powered by Thorsten Meyer AI
How Amazon Wants AI Work Judged
The guidance matters because companies are trying to gauge whether investment in AI tools produces measurable value. Usage is easy to count; improved customer experience, faster delivery or better-quality work can be harder to attribute and may take longer to appear. Amazon’s memo draws a line between those activity signals and the outcomes it says should guide decisions.
For engineers, the distinction affects how work may be evaluated. The company says workers should not be penalized simply for using AI less, or rewarded for generating more tokens or code. Instead, a reasoned assessment that a tool does not suit a particular task can itself be useful evidence. That approach could also limit incentives to create unnecessary work merely to improve a usage score.
For managers, the memo points to a practical challenge: adoption data can help locate questions, but cannot answer them alone. A team with low use may need support, or may have work that current tools do not improve. A team with high use may be gaining value, or may be accumulating cost and complexity. The memo asks leaders to examine both possibilities rather than assume usage equals productivity.
Popular right now
Roborock Q7 M5+ Robot Vacuum and Mop
Philips 3300 Series Fully Automatic Espresso Machine with LatteGo Milk System
Ninja SP151 Flip Toaster Oven and Air Fryer, 8-in-1, Stainless Steel
As an affiliate, we earn on qualifying purchases.
From Usage Trackers to Official Guidance
The memo follows earlier reporting about internal AI-use tracking at Amazon. In May, the Financial Times reported that some employees gave an internal agent tool, MeshClaw, unnecessary tasks to increase token counts. The report also described a weekly AI-use target and internal leaderboards. These accounts were reported by the newspaper and are not independently established in the memo.
Amazon did confirm a related development: at the end of May, a company spokesperson told Business Insider that an employee-built leaderboard called KiroRank had been deprecated. The company described it as an informal tracker and said it had not been intended to promote AI use for its own sake. Senior vice president Dave Treadwell also told staff not to use AI merely for the sake of using it.
The new guidance extends that message into an official role guideline. It also comes amid broader questions about the returns from Amazon’s AI investments. The supplied source cites Business Insider’s July reporting that most flagship Nova models had been moved to a maintenance posture and Amazon disclosures showing a large paper valuation for its Anthropic investment. Those figures and characterizations relate to separate company developments; they do not establish that the engineering memo evaluates either investment.
“Tokenmaxxing is at odds with frugality.”
— Amazon engineering memo
How Outcomes Will Be Measured
The memo sets out a principle but does not specify a single measurement system for determining whether AI has improved customer or business results. It also does not provide a company-wide estimate of productivity gains, cost savings or the share of engineering work where AI has helped. Those outcomes can vary by task and may take time to assess.
It is also not clear from the supplied material whether Amazon will formally change performance reviews, publish further implementation rules or retire other usage trackers. The memo says usage measures can correlate with outcomes today, but offers no figures or methodology for that relationship. Correlation does not establish that higher usage caused better results.
Teams Put the Guidance Into Practice
The memo asks leaders to use AI tools on their own production work, including identifying a task where the tools helped and one where they did not, then explaining the difference. Teams are also expected to examine usage patterns at both ends of the adoption range and share findings about where current tools fall short.
Amazon’s third-quarter results were scheduled for October 29, two days after publication of the memo, according to the source material. The earnings report may provide broader financial context for the company’s technology spending, but the memo itself does not promise that Amazon will report a specific AI return measure. What remains to be seen is how the company translates its outcomes-first guidance into consistent team assessments.
Key Questions
What did Amazon tell its engineers?
Amazon’s October 27 guidance says engineers should be judged by the impact of their work, not by token counts or how often they use AI. It warns that usage measures can become harmful when treated as targets.
Is Amazon telling employees to stop using AI?
No. The memo says AI adoption currently correlates with important outputs and asks leaders to investigate both low and high adoption. Its message is to assess whether AI helps, not to maximize use for its own sake.
What does “tokenmaxxing” mean in this report?
It refers to using an AI system to generate or process more tokens in order to increase a usage measure or leaderboard ranking, rather than because the work needs it.
Will engineers be penalized if AI does not help their work?
The memo says a finding that current AI tools do not benefit a task is “a finding, not a failing.” It asks employees to explain and share that assessment. The supplied material does not detail how the guidance will affect formal performance reviews.
What remains unknown about Amazon’s policy?
The memo does not set out a standard formula for measuring AI’s effect on customer or business outcomes, nor does it report company-wide productivity gains. It is also unclear whether Amazon will issue additional rules or change performance-review practices.
Source: ThorstenMeyerAI.com
