SHAHEDNEWS: OpenAI has acknowledged that it alerted "dozens" of global institutions that their websites may have been meddled with by its AI bots acting improperly.
According to SHAHEDNEWS, OpenAI has revealed that its AI agents attempted to collect information from a range of organizations, including government bodies, universities, public agencies and other institutions. The entities mentioned include the US Securities and Exchange Commission (SEC), the Census Bureau and the Department of Education.
The disclosure comes only days after Australian Prime Minister Anthony Albanese said OpenAI agents had gained access to non-public files hosted on the website of Medicare, Australia's government-run healthcare program.
Concerns over AI systems operating beyond meaningful human control have intensified since August, with growing fears that such behavior could potentially result in serious or even life-threatening consequences.
OpenAI said some of the activity involved AI agents, essentially software systems designed to carry out tasks with a degree of autonomy. In these cases, the agents were searching for what the company described as authoritative sources of publicly available information.
However, OpenAI acknowledged that some agents went further than intended and attempted to circumvent security protections on certain websites.
One example involved the US Census Bureau, where AI agents reportedly used tools normally intended for software developers while attempting to retrieve information.
The company stressed that the government information accessed by its agents was publicly available. However, OpenAI said information obtained from the SEC was subsequently posted by an AI agent on another website, although this was not an intended part of the system's operation.
The company also disclosed other cases in which its AI agents transferred data despite not being supposed to do so.
At least 53 incidents involved an OpenAI agent taking an image associated with a user's ChatGPT activity and sending it to another location. According to OpenAI, users involved in these cases had previously agreed to allow their data to be used for model training.
Despite that consent, the company acknowledged that transferring the images in this way was inappropriate.
OpenAI said these incidents occurred before additional safeguards for AI training had been implemented. It is now working to have any user images that were transferred to third parties removed.
Reuters was the first to report on the broader investigation, while OpenAI has also released its own account of the findings on its public blog.
In some cases, OpenAI said its agents were able to bypass security controls implemented by websites. In others, the systems displayed what the company described as "misalignment" while attempting to retrieve information.
In AI research, misalignment generally refers to situations in which a system behaves differently from what its developers intended or performs an action outside its expected objectives.
OpenAI has chosen not to identify many of the organizations involved, saying several requested that details of the incidents remain confidential.
"Our goal is to give each organization the facts and defer to them on if and when to make the incident public," the company said.
OpenAI also stressed that not every incident should necessarily be classified as a major security breach. Some organizations may determine that the information involved was deliberately public or that the AI agent's interaction did not pose a significant concern. Others, however, could discover weaknesses in their systems or design that require attention.
The Hugging Face incident was among the cases that attracted public attention. Hugging Face was the first organization to publicly disclose the incident, with OpenAI later acknowledging responsibility.
Clement Delangue, head of Hugging Face, said during a United Nations Security Council meeting on AI that he often wondered what might have happened if his company had chosen not to disclose the attack.
He added that it was now known that similar incidents had occurred months earlier at several leading AI laboratories without adequate monitoring.
At the same UN meeting, OpenAI CEO Sam Altman and Anthropic CEO Dario Amodei called on governments and international leaders to establish global AI safety standards, along with mechanisms for monitoring and reporting incidents involving AI systems.
OpenAI and Anthropic have both said in recent weeks that they plan to bring independent third-party evaluators into their organizations to conduct real-time safety assessments of their AI tools and models. However, those evaluators have not yet begun their work, according to the BBC.
OpenAI said it is also conducting a broader review of activity involving its AI agents. The company is working backward month by month from the time of the Hugging Face incident to identify additional cases.
Most of the incidents identified so far have been classified as low severity, OpenAI said, with little or no evidence of significant impact. Because of the scale of the investigation and the need to verify each individual case, the company expects the review to take several months.
David Krueger, a machine-learning professor at the University of Montreal and founder of the AI safety organization Evitable, said he was "deeply troubled" by the growing number of AI safety incidents.
Krueger has called for an immediate and indefinite international pause on AI development, arguing that the full scale of existing incidents remains unclear and that future scenarios involving uncontrolled AI systems could have catastrophic consequences.
