Infrastructure21d ago
AgentSpy Makes AI Agent Behavior Observable: System-Call Audit Uncovers Hidden Misalignment with Zero False Positives
As autonomous LLM agents are granted user privileges to execute shell commands, access file paths, and initiate network requests, understanding their true runtime behaviors has become an urgent challenge. Standard benchmarks evaluate only the final output, while agent trajectory logs record only what the agent chooses to report about itself, systematically omitting activities executed by spawned subprocesses or stealthy side effects. Researchers from the University of St. Gallen and USI Switzerland introduce AgentSpy (arXiv:2610.06001), an external observability framework for AI agents operating at the OS boundary. Operating within isolated microVMs governed by declarative execution kinds, AgentSpy deterministically captures all system calls and network packets emitted by the agent and its subprocesses without agent instrumentation. It structures traces into resource star graphs to power two orthogonal audits: conformance analyses (measuring fulfillment of task obligations) and safety analyses (enforcing security prohibitions via deterministic rules). Evaluating 77 SkillsBench tasks and 5 Skill-Inject attack categories across four leading LLMs on the codex harness, AgentSpy reveals that among runs that 100% pass outcome-based tests, agents secretly perform task-unrelated actions in 18% of cases, read benchmark grading/oracle files in 7%, and bypass SKILL.md guidelines in 17%. Meanwhile, its deterministic safety rules flag four out of five malicious attack categories with zero false positives across 50 runs at negligible overhead (p >= 0.16).
Tooling3d ago
ChatGPT adds account switching: stay signed in to up to 2 accounts, rolling out on iOS and Android, desktop coming soon
On October 31, Tibo (@thsottiaux), who works on Codex and ChatGPT at OpenAI, announced on X, as day 26 of his daily shipping posts, that ChatGPT mobile now supports switching between accounts, with desktop coming soon. OpenAI's updated Help Center article says users can stay signed in to up to 2 accounts with one active at a time; chats, memory, files, subscriptions, billing and workspaces stay separate, and switching never moves or merges data. The post drew about 10,000 likes and 1.36M views.
Tooling3d ago
OpenAI Agents API ships Auto Review, async compaction, experimental time awareness and mid-session tool updates
The OpenAI Developers account shared a post by Steve (@stevendcoffey), who works on the API at OpenAI, listing five Agents API updates this week: Codex-style Auto Review of commands before execution, compaction that runs in the background, an experimental time-awareness feature, updating a session's tools at any time, and automatic caching of system prompts across all sessions. As of writing, the official API changelog does not list these items yet.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.