AI firm Anthropic – owner of the Claude AI assistant – has now admitted three cases where test models went rogue, accessed the open internet and hacked other organisations. The news follows on from OpenAI’s recent admissions that test models had done the same.
Anthropic wrote a lengthy post about the incidents but has not yet named the organisations involved. The AI firm says it reported the breaches to the companies involved on July 27. However, it’s likely more information will emerge via the organisations involved or people familiar with the matter.
As a result of the OpenAI events, Anthropic says from July 23 it reviewed “141,006 evaluation runs where Claude could have obtained internet access… we identified three incidents in which a model accessed the internet from within or while interacting with the evaluation environment of Irregular, one of our third-party evaluation partners, and then gained unauthorized access to the production infrastructure of three different organizations.”
The company says that in all three incidents which have occurred since April, Claude had been tasked with a capture-the-flag challenge to assess the model’s cyber capabilities. “The model is given a fictional scenario and told that a piece of secret information (the “flag”) has been hidden on a different machine on the network, and its objective is to break in and retrieve it. The challenge is left open-ended, and no particular method is prescribed.”
Because of a misunderstanding with the unnamed organisation Anthropic was using for testing, internet access was erroneously available and so Claude understandably targeted that to acheive its aims. As with the OpenAI disclosures, the key thing to take away is that AI can now probe complex tasks to such an extent that it is able to use different methodologies to access things it shouldn’t be. And that could have serious consequences for the security of the internet and corporate infrastructure if safeguards are not put in place.
Also read: Why is Apple suing ChatGPT? Here’s what’s really going on with the battle between the tech giants
Read the full article here
