Benchmark13d ago
Artificial Analysis Benchmarks DeepSeek Search API: $0.75/1K Searches Delivers +36 Point Score Lift
Independent AI evaluation platform Artificial Analysis (@ArtificialAnlys) officially benchmarked DeepSeek's web search API (DeepSeek Search). Augmenting the base model with DeepSeek Search boosted its Search Index score from 33 to 69—an impressive +36 point leap (ranking 17th among 27 tested search APIs). DeepSeek Search established a market-leading price-performance ratio, priced at just $0.75 per 1,000 searches (~$0.032 per completed task), making it the most cost-effective paid search API evaluated. It recorded 70% accuracy on AA-Omniscience and 63% on BrowseComp, highlighting an optimal balance between factual grounding and operational economics.
Tooling6d ago
CrowdStrike: China-nexus VAPOR PANDA used the ARTEX multi-agent framework, mainly on DeepSeek, against 200+ organizations in 11 days
CrowdStrike reported on Oct 29 that the China-nexus espionage group VAPOR PANDA, starting Oct 17, ran the open-source agentic intrusion framework ARTEX with DeepSeek plus a second, unconfirmed commercial LLM, hitting more than 200 organizations in at least 14 countries from a single controller over 11 days. Post-exploitation was confirmed at nine organizations, where the BLOODALCHEMY backdoor was deployed. Planning agents used about 1.4 billion input tokens in a single intrusion, and recovered transcripts show a human operator steering the agents in Chinese at key decision points.
Research33d ago
CorrGRPO Solves Multi-Reward Imbalance in GRPO: Pearson Correlation Normalization Boosts Code & Tool Calling
Group Relative Policy Optimization (GRPO) forms the reinforcement learning foundation for reasoning models like DeepSeek-R1 by normalizing group rewards to derive advantage estimates. However, practical LLM alignment requires balancing multiple competing rewards, such as code execution correctness, format compliance, tool-calling precision, and safety. Standard GRPO computes advantage normalization across aggregate variance, causing large-scale, highly correlated reward components to overshadow critical fine-grained signals. HKUST researchers propose CorrGRPO (Correlation-Normalized GRPO), which normalizes reward covariances into Pearson correlation coefficients. Across code generation, tool invocation, and agent safety benchmarks spanning 0.5B to 8B models, CorrGRPO systematically outperforms standard GRPO.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.