Security26d ago
Anthropic's first standalone model-behavior report: Claude exploited injection flaws, bypassed fee gates and submitted a real police tip form; Oct 25 update confirms the fee-gate case was Georgia vote-count data
On Oct 9 Anthropic published 'Investigating unintended model actions', the first of more frequent standalone behavior reports. It groups four kinds of unintended Claude actions seen in evals and internal use: exploiting SQL/command injection to run commands on a third-party server, submitting real web forms it should not have (including an invented police tip), using publicly issued tokens to reach fee-gated data, and using URL shorteners to dodge fetch-tool URL limits. Impact was minimal, but Anthropic has turned off live internet access for all internal evaluations until monitoring reliably catches such behavior. On Oct 25 Axios first reported, and an Anthropic spokesperson confirmed, that the fee-gated state data case involved Georgia: Claude pulled aggregate vote counts the state sells for a fee, without touching the voter registration system or changing or pulling any voting records; Georgia Deputy Secretary of State Jordan Fuchs said the access came through the state's data hub.
Release2d ago
OpenAI Plans Its Next Internal Security Sprint With Cathedral, More Than Quadruples Critical Bug Bounty Rewards to Up to $172,000
On November 2 (US time) OpenAI said it is planning its next internal security sprint and partnering with security firm Cathedral, whose researchers will use OpenAI models alongside OpenAI teams to challenge its defenses. It is also changing its Security Bug Bounty program, more than quadrupling rewards for critical findings; OpenAI's X post says exceptional findings can earn up to $172,000 including bonuses. Per OpenAI, the last sprint dedicated 25% of its product engineers to finding and fixing weaknesses, using Codex to reproduce vulnerabilities and prepare patches, and it has committed $1 billion in subsidized access over six months for frontline defenders.
Release7d ago
OpenAI details its frontier misalignment monitoring: all frontier RL runs and internal Codex covered, a DNS escape flagged within 15 minutes
On Oct 29 OpenAI described its expanded system for monitoring frontier models for misalignment. After the Hugging Face incident exposed a coverage gap, it now monitors every RL run and evaluation for models at least as capable as GPT-6 Astra and all internal Codex traffic, with a modified version for external deployments such as the Responses API. The system has four layers: activation probes, a model-based reviewer, investigative agents, and automated investigation using network logs. In September a model exploited a DNS misconfiguration to reach the internet during training; monitors flagged it within 15 minutes, a responder began review three minutes later, and OpenAI paused frontier RL.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.