OpenAI reveals cases o ‘concerning’ AI behaviour as it announces new disclosure system

OpenAI has disclost sax mair examples o unco or concernin behaviour by its technology, as it warned that the pace o development cudna continue at maximum speed for muckle langer responsiblie.

In ane o the new cases reportit by OpenAI, an unreleased research model insertit jailbreak-like instructions intae its ain notes tae disregard its normal constraints and telt itsel tae be freed frae the roles and identities that bind ither chatbots.

In anither instance, an AI agent uploadit files tae the internet tae obtain a brawser citation withoot askin the user.

OpenAI’s admission cam as King Charles cawed for strangir safeguards on AI afore it is a' ower late, at a meetin wi tech bosses in Scotland.

Speakin at a specially convenit meetin wi AI executives, he said: There seems urgency in adequatelie considerin the existential dangers o sic technologies fawin intae the wrang hands, and bein used in potentiaillie catastrophic ways. Shud we no hae sufficient means o control afore it is a' ower late?

The monarch was jined at the meetin by Nvidia’s founder and chief executive, Jensen Huang, Google DeepMind founder and chair, Sir Demis Hassabis, OpenAI’s chief financial officer, Sarah Friar, and the UK’s AI minister, Kanishka Narayan.

Charles said AI had the potential tae impruive and save lives, particularlie in life sciences and medicine.

But its creators were incrementallie warnin that it cud develop darker capacities, perhaps even tae tak life, he said.

OpenAI, the developer o ChatGPT, said in a blogpost publisht on Wednesday nicht that it was introducin a new framework for trackin, investigatin and disclosin AI model misalignment, the term for AIs failin tae adhere tae human values and safety goals.

In the blogpost, OpenAI echoit caws for a development slawdoun issued by its rival Anthropic, which has said the current pace o growth poses an existential threat. We dae no believe that the AI industry has solvit alignment and monitorin tae a sufficient degree tae continue responsiblie scalin at maximum speed for muckle langer, said OpenAI.

Decisions aboot hou AI development shud proceed in the months and years tae come need tae draw on evidence that fowk ootside the companies buildin frontier models can examine for themsels.

Google and Elon Musk, wha also awns an AI startup, hae supportit caws for a slawdoun, which hae been rejectit by Donald Trump, wha citiit the need tae keep aheid o China’s AI industry. The caws hae also been met wi scepticism frae some experts, includin a warnin that companies maun no appoint their ain auditors.

Examples o potential existential threats posit by AI range frae facilitatin the development o bioweapons tae triggerin a global financial crash. A tap safety researcher at Anthropic has said there is greater than 10% chance that AI cud kill a' humans within the neist decade. Houever, a source familiar wi Anthropic’s thinkin has acknowledgit that the exact chances o ony ane ootcome are probablie unknowable.

The sax reportit incidents were discoverit durin trainin or evaluation ower the past months, OpenAI said.

Wednesday’s new cases cam efter OpenAI disclost in Julie that an AI agent swarm hackit intae the AI startup Hugging Face durin a cybersecurity test. Anthropic also said the same month that its AI models hackit intae three organisations durin testin. Anthropic said the models had been deliberatelie testit withoot cybersecurity safeguards, and that they had been, and were able tae reach the apen internet – the AI testin equivalent o leavin the front door apen – due tae a misunderstandin wi an external testin company.

AI agents – the term for AI tools that operate autonomouslie – were becomin smarter and mair determinit tae resolve complex tasks throu inter-agent collaboration, knawledge sharin, deception and concealment, said Lian Jye Su, a chief analyst at the technology research and advisory group Omdia.

That was makin it harder tae govern and contain them usin traditional AI security approaches, he said.

OpenAI’s new trackin and disclosure framework cud help encourage ither AI developers tae adopt similar practices. That said, the process remains internal and voluntarie, but is a step in the richt direction, Su said.