Large language models (LLMs) that power AI conversational chatbots like ChatGPT, are a new attack surface that will likely evolve to make some attacks more cost-effective or persistent, according to tech giant IBM.
Attackers can use AI-generated text with ChatGPT, for example, to easily construct convincing phishing e-mails that are very authentic-sounding, eliminating many indicators that it is a suspicious e-mail.
Such LLM-powered tools can make phishing campaigns easier to execute globally, even if the attacker doesn’t speak the language, said Chris Hockings, chief technology officer for cybersecurity at IBM Asia-Pacific.
“There is a massive time deficit on the defender side, and there is no need for the attacker’s side to get more speed and scale,” said Hockings.
He said that LLMs can be manipulated or ‘hypnotised’ to provide potentially dangerous responses and recommendations.
Earlier this year, IBM had attempted to ‘hypnotise’ popular LLMs to test how easy it is for threat actors to get LLMs to offer poor advice to users without carrying out a massive data poisoning attack. It successfully ‘hypnotised’ five LLMs with the English language and without using any programming language.
As a result, the LLMs leaked confidential financial information about other users, created vulnerable and malicious code, and provided weak security recommendations.
Hockings said that this means attackers can exploit LLMs by tricking it into giving the opposite advice as requested by the user, by creating a “simulation game” where the user gets responses they are not looking for.
Additionally, LLMs are vulnerable to data poisoning, where an attacker manipulates the training data or introduces vulnerabilities, backdoors, or biases that could compromise the security or effectiveness of the LLM.
“There are backdoors available to attackers for sale on the Dark Web. Imagine a backdoor that has access to a database – an attacker could sell that backdoor to somebody who then poisons the data set that trains the LLM,” said Hockings.
Solutions for protection
To counter malicious attacks, organisations need to develop a mindset around zero-trust to protect themselves and their networks from these phishing campaigns, according to IBM.
Data protection encompasses the training of data and having the appropriate controls around the authorised sharing of data to the right recipients. Threat detection is also essential as the scale and sophistication of cyber attacks grow.
Key here is the integrity of software solutions. To ensure clean and safe code, organisations should focus on protecting open-source verification processes, source code control systems, with solutions like identity access management and role-based access, says IBM.
Testing, it adds, also plays a significant role in ensuring expected outputs across various use cases.
“Attackers can easily access repositories and insert malicious code, making it essential to ensure data security and compliance with regulatory and governance requirements,” said Hockings.
As organisations start to dabble in these LLM models, they may not have security built in to their systems and processes.
“Enterprises should prioritise cybersecurity elements as a core function, with a risk mitigation strategy,” said Hockings.“Cybersecurity must be included from the beginning to ensure the integrity and trustworthiness of systems.”
