Anthropic warns of open-source AI risks as Nvidia commits $250m to UK research

Open-source artificial intelligence systems carry heightened cybersecurity vulnerabilities, leadership at frontier developer Anthropic has warned, cautioning that models could begin autonomously improving their own capabilities as early as 2027.

The warnings over unchecked software distribution come as international governments and technology firms step up institutional oversight. In the United Kingdom, chipmaker Nvidia has pledged $250 million worth of advanced computing capacity over five years to support sovereign research into autonomous software agents. At the same time, Indian industry is rapidly moving to translate artificial intelligence capabilities into productive enterprise capacity, with domestic firms scaling integration services to capture a national market projected to reach $43.5 billion by 2030.

What are the safety risks facing frontier AI?

Anthropic executives argue that freely distributing the underlying code and weights of advanced models introduces acute risks that proprietary developers cannot mitigate after release. Unlike closed platforms that maintain centralised safeguards, fully accessible open models permit malicious actors to bypass guardrails, heightening threats to digital infrastructure and corporate networks.

Adding urgency to the debate, Anthropic co-founder leadership anticipates that frontier systems may attain recursive self-improvement capabilities within three years. Once an artificial intelligence model can independently refine its own architecture and training processes, containment becomes substantially more complex.

These concerns were underscored earlier this year after OpenAI confirmed that a coordinated swarm of autonomous agents targeted software platform Hugging Face, revealing operational gaps in current agentic security.

Global investments and market growth

Nvidia commits computing capacity to British labs

To counter emerging agentic threats, Nvidia, valued at $5.5 trillion, will supply UK laboratories with dedicated accelerated computing. The resources will support the British Open-Ended Learning and Discovery Lab, Imperial College London, and the Science of Fundamental AI Research Lab, which is jointly administered by Cambridge, Oxford, University College London, and Edinburgh universities.

"The UK brings world-class scientific rigour to this essential engineering challenge," said Nvidia chief executive Jensen Huang, explaining that the donation will put hardware in researchers' hands to test safeguards. UK AI Minister Kanishka Narayan described the move as a major endorsement of national research strength. The UK AI Security Institute will also examine flagship models from developers including OpenAI, Anthropic, and Google prior to commercial release.

India expands commercial robotics and AI deployment

While Western regulators focus heavily on containment, India's technological sector is driving rapid commercial integration, anchoring national productivity and technological resilience. Demonstrating this commercial momentum, domestic technology provider Empower India announced plans to expand into artificial intelligence and robotics implementation, seeking to capture enterprise demand across the country's $43.5-billion expansion curve.

Rather than undertaking capital-intensive frontier chip design or proprietary foundation model training, the company will focus on systems integration, enterprise customisation, and maintenance partnerships with software developers and automation specialists.

"We aim to help enterprises move from technology adoption to practical deployment, creating intelligent systems that can operate effectively within real business environments," said Rajesh Chavan, managing director at Empower India. The strategy aligns with India's broader push to build robust digital infrastructure and operationalise artificial intelligence across domestic manufacturing and enterprise supply chains.