The Rising Concerns of AI Vulnerabilities
In recent times, it has become increasingly common to hear about autonomous AI agents hacking websites or tools being exploited by cybercriminals and fraudsters. A recently patched vulnerability in the macOS version of ChatGPT exemplifies how, as these applications gain popularity, the AI software itself may also become an enticing target for attackers.
This flaw could have been exploited to seize control of ChatGPT on a victim’s computer, granting the attacker access to all chat histories and other data stored by the application, along with related elements such as browser sessions. Discovered by researchers at the Objective-See Foundation, the vulnerability highlights the extensive system access these platforms require to operate and how that inherent trust can render them appealing targets.
“Agents require extensive access to perform their functions. They are akin to a hotel’s housekeeper who has access to all the rooms. Therefore, if they can be compromised or manipulated, it poses significant concerns,” stated Patrick Wardle, a software analyst at the Objective-See Foundation and a seasoned macOS researcher.
OpenAI’s Response to Security Flaws
OpenAI publicly acknowledged the security flaw and its resolution in its system change log on September 25. “We are continuously improving our security practices, but we recognise the need to act with greater speed,” remarked OpenAI spokesperson Shane Bauer in a statement.
The macOS application of ChatGPT comprises multiple components that communicate securely through digital signature verification. This design aims to confirm, through these validity checks, that both processes are indeed OpenAI components and not potentially malicious external software making requests. The system’s design even mandates these signature checks at three levels of separation from the request, ensuring that no malware is instructing an OpenAI component to act as a proxy and make a seemingly legitimate request.
The Flaw in the Trust Model
However, researchers from the Objective-See Foundation discovered that one of the trusted components, a script interpreter, could accept a series of commands and be manipulated to relay them to ChatGPT’s main process. “They also check the parent and grandparent processes, but the malicious script simply executes the script interpreter three times and then makes the request in a way that meets the requirements,” Wardle explained.
Exploiting this vulnerability was described as “incredibly straightforward.” Beyond gaining access to ChatGPT’s chat logs, the flaw could also allow attackers to make ChatGPT execute commands on their behalf, such as accessing a browser or other sensitive applications, thereby making the requests appear as legitimate instructions issued by OpenAI’s software.
Future Implications and Security Focus
Wardle is set to present an analysis of various bugs in AI applications for macOS at “Objective by the Sea,” a security conference focused on Apple, scheduled for November. He recently uncovered a flaw, now patched, in the dictation feature of Meta’s new Muse agent that could have been exploited by a local hacker to obtain a mismanaged authentication token and access user data. Additionally, he has submitted a new vulnerability finding related to the integration between ChatGPT and OpenAI’s always-active AI agent, Dots, which is currently under review by the company.
“AI companies are currently obsessed with adding features. However, the more features they incorporate, the larger the attack surface becomes. Therefore, all these companies should be fully focused on security, and from what I observe, it often remains a secondary concern,” Wardle concluded.