Claude's mark, token jacking, and the Canaries Dashboard update

This week Anthropic announced that future Claude models will invisibly watermark their text, although no model has been launched since the policy took effect on 2 August and Anthropic has not confirmed that any existing model has been updated.

The announcement responds to new transparency requirements under the EU AI Act. Providers of generative AI systems must make synthetic text, images, audio and video detectable in a machine-readable form where technically feasible. Anthropic has signed the supporting Code of Practice alongside around 190 other organisations.

Several companies already claim to remove Claude watermarks. These claims cannot be verified until Anthropic provides a working detector. Passing a conventional AI detector such as GPTZero does not prove that the watermark has gone. Those products estimate whether writing resembles model output. They do not have access to Anthropic’s marking system.

Google provides a useful comparison. Its SynthID Text technology includes tools for generating watermarked text and testing for the resulting mark. This gives us a practical way to explore the behaviour Anthropic is proposing without assuming that both companies use the same implementation.

Anthropic has not released its detector, API or technical documentation. We cannot currently take a Claude response and establish whether it contains the mark. We also cannot independently assess how well the system survives editing, copying, translation or paraphrasing.

Most watermarking methods alter the choices made while a model writes. There is nothing visible in the finished text.

We ran some experiments with a small AI model and two watermarking methods, one based on a published green-list technique and the other using Google’s SynthID Text technology to explore how these systems behave.

With the green-list technique, at each point in a sentence, a secret rule divided the words the model might choose into two arbitrary groups. The model gave one group a small advantage. Those words were not better, clearer or more accurate. They were preferred solely to create a detectable pattern.

Google’s SynthID uses a more complex version of the same principle. It applies several secret scoring rules and uses a tournament process to influence which word is selected. Its detector reconstructs those scores across the finished passage and looks for a consistent imbalance.

We tested both methods using the same prompts, sampling settings and editing operations. The chart shows the percentage of passages detected after different types of subsequent edits:

What we found:

• Light editing leaves the mark intact. Both watermarks survived sentence reordering because most of the original wording remained unchanged.

• Rewriting weakens detection quickly. Both methods became unreliable when 30% of the words were replaced. Neither survived when half the words were deleted.

• Short extracts are difficult to judge. SynthID detected both 20-word fragments, while the green-list detector found one. Short passages contain fewer word choices for the detector to examine, so results are less reliable.

• Adding human writing dilutes the evidence. The green-list watermark survived when we added unmarked text equal to the original passage. SynthID fell below its whole-document threshold. Its mark may still have remained detectable within the original section, but it was lost when the document was assessed as one block.

• Detection does not establish authorship. Pasting one marked AI paragraph into a human document could cause the whole document to be flagged. The surrounding human writing has not acquired the watermark, but a whole-document result may create that impression.

• Human work can acquire an AI watermark. A model could revise writing created years before LLMs existed and return a marked version. The pattern would show that the model influenced the wording, not that it created the ideas or original argument.

• Writing quality remains an open question. Watermarking gives arbitrarily selected words a small advantage during generation. Some influence on the model’s choices is unavoidable, even if it is too subtle to notice in an individual response.

What is concerning is that schools, employers and publishers will treat watermark detection as a verdict, not just as evidence of model involvement. It should instead be one part of a wider provenance record.

The community reaction has been highly negative. Dozens of Claude users posted subscription cancellations, with writers and developers worried that clients, employers or schools could mistake evidence of AI assistance for proof of AI authorship. Others objected to Anthropic placing a persistent marker in private work without an opt-out, particularly while retaining sole control of the detector. Investor Bill Gurley described that position as making Anthropic “judge, jury and prosecutor”.

Businesses publishing synthetic media also have their own obligations. An AI-generated image that appears to document a real person, company, product, place or event may require a visible disclosure. AI-assisted text follows a more practical rule. The EU requirements target public-interest text published without substantive human review or accountable editorial control. They do not require every business to label material simply because AI helped with research, drafting, translation or editing.

This places the emphasis on the publishing process. A responsible organisation should know who checked the facts, who edited the copy and who approved publication. Automated articles published without meaningful review need disclosure. Work that has passed through a genuine editorial process does not need a warning merely because Claude contributed to it.

Takeaways: Our experiment shows that watermark detection is a sliding measure shaped by passage length and editing, not a permanent verdict attached to every word. The technology can provide evidence of model involvement and give AI labs a route to regulatory compliance, but it cannot determine authorship, accuracy or responsibility.

This week, security researchers revealed the scale of the data stolen during March’s attack on LiteLLM, a widely used gateway connecting applications to AI models. A 153GB archive contained 400,000+ files linked to nearly 2,500 organisations, including cloud credentials, GitHub tokens, Kubernetes secrets and keys for OpenAI and Anthropic.

Our previous analysis has explored how AI improves malware and helps attackers find vulnerabilities. This incident reveals another risk. AI access is now a valuable commodity, and an established international "token-jacking" market turns stolen credentials into cheap model usage.

That market is particularly visible in China. OpenAI and Anthropic do not officially provide services there. Chinese developers may need an overseas network connection, telephone number, payment card and billing address to use them. Anthropic also restricts companies controlled by organisations in unsupported countries and now requires identity checks from some users.

Chinese developers still want Claude, ChatGPT and their coding tools. Their practical solution is an AI transfer station, or relay station.

The user buys credit through Alipay or WeChat Pay, and points their agents at a new API link. The endpoint may offer hundreds of models through the same OpenAI-compatible interface. Chinese technology publications review these services, comparison sites rank their prices and availability, and developer communities exchange advice about which stations are reliable.

Behind that simple link sits a murky and exploitative supply chain.

Account merchants register or acquire AI accounts in bulk. SMS platforms provide overseas telephone numbers. Card merchants supply foreign payment credentials. Proxy networks make requests appear to originate in supported countries. Reverse engineers study provider software for authentication shortcuts, while station operators rotate accounts and adapt whenever abuse controls change.

Know your customer (KYC) checks have added another industry. Station operators use synthetic identity documents and deepfake tools to match a face to a forged identity. When automated deception fails, brokers recruit real people to complete verification with genuine documents and live selfies. Transfer station "agents" travel to countries in South East Asia, Africa, and Latin America to source people willing to pass these checks.

Iris scans from KYC merchants in Cambodia and Kenya have reportedly been offered on the biometric black market for less than $30. A face or identity supplied for one AI account can later be reused for financial accounts, employment fraud or deepfake impersonation. The person providing it may have no knowledge of the businesses eventually built upon their identity.

At the centre, transfer stations combine these accounts and credentials into pools. Some buy official capacity. Others share subscriptions, farm free credit, use fraudulent cards or reverse-engineer consumer services. Stolen API keys can be added to the same pool, such as those harvested in the LiteLLM attack.

The station can extract further yield in other ways. It can charge for access, silently substitute a cheaper model, and many retain prompts and responses. Coding-agent logs may contain repositories, engineering decisions, and more credentials to harvest. These records support fraud, commercial intelligence or black model training.

Anthropic says DeepSeek, Moonshot and MiniMax generated more than 16 million Claude exchanges through around 24,000 fraudulent accounts. Transfer stations allegedly mixed this activity with ordinary customer traffic, obscuring who was making each request and why.

Despite Chinese authorities warning against grey and black transfer stations, the practice remains surprisingly normal there. Technology publications discuss how to choose a reliable provider. Developers compare prices and uptime. Relay software is openly available on GitHub. A customer sees a convenient API, not the account farms, stolen credentials, payment fraud and biometric identities that may be supplying it.

Takeaways: Time to check your AI model usage dashboard for unexpected traffic, unfamiliar models or sudden spending outside normal hours... It's always a good idea to "rotate" any API key stored in source code, developer laptops or pipelines, or better still use a secrets manager. Remember to set hard daily spending caps and alerts rather than waiting for the monthly bill. And if you're developing with languages like Python and installing lots of useful libraries, it's critical to use package scanners and to "pin" your package versions. This gives you control over upgrades and prevents potentially exploited packages such as LiteLLM getting installed silently. And whatever you do, stick to well known AI suppliers and never, ever use a transfer station!

This week's chart shows employment for workers aged 22 to 25 since 2021, grouped into five bands by how exposed their occupation is to AI. The dark lines are the most exposed jobs, such as software development and customer service. The pale lines are the least exposed, such as nursing, maintenance and management. Stanford's revised Canaries in the Coal Mine paper puts the gap between them at 19%, up from 15% last July. The lab has also launched a Canaries Dashboard with ADP Research, refreshing these figures every month rather than once a year.

That figure is relative, not a count of jobs lost. Employment in the most exposed bands fell by about 11% while the least exposed grew by about 10%. The adjustment is happening through slower hiring rather than redundancies, and pay has barely moved. The authors describe their results as patterns, not proof of cause. The gap narrows when education is taken into account, and their payroll sample shows larger effects than national surveys.

The mechanism is more useful than the number. Jobs built on codified knowledge are losing young hires. Jobs built on knowledge learned by doing are gaining experienced ones.

Retraining may not always be the answer. Anthropic reviewed 56 randomised trials this week and found average gains of 1.7 percentage points of employment and about $800 a year, at a cost of $13,000 per person. Only programmes designed with employers returned real value.

Takeaways: Employers are not replacing young workers outright; they are declining to hire juniors whose skills are codified while paying more for senior experience that only junior work produces. Because that stock of experience cannot be replenished by generic retraining, cutting early-career rungs shrinks the future talent pipeline.