Being aware of the threats to generative AI (GenAI) workloads in the cloud and how to combat them is vital, says Kennedy Torkura, co-founder and CTO of Mitigant, a partner of Sopra Steria Ventures.
GenAI is transforming our lives in various ways, and organisations are rapidly exploiting its potential to enhance productivity, innovation, and business advantages. However, only some organisations can easily access GenAI, despite the perceived benefits.
Consequently, cloud service providers (CSPs) now provide GenAI-as-a-Service offerings, such as Amazon Bedrock, Azure AI Services, and Google Vertex AI. These GenAI-as-a-Service offerings drastically lower the entry barrier for organisations and fast-track development, deployment and maintenance of GenAI workloads without bothering much about the infrastructure and knowledge requirements.
However, these GenAI workloads present several security and safety challenges that require attention, given the increasing need for responsible AI systems, such as the EU AI Act. Organisations must understand these risks and possible countermeasures to retain customer trust and business excellence, while thwarting potential cloud attacks.
Top threats against GenAI cloud workloads
LLMJacking
LLMJacking is an attack where criminals gain illegal access to large language models (LLMs) by first stealing cloud credentials, e.g. AWS API Keys, and using the credentials to access cloud GenAI workloads e.g. LLMs. The primary motivation is to pass over the vast bills accrued to the victim account organisation (up to $46,000 of LLM consumption/day). KrebsonSecurity recently published a story on LLMJacking attacks, based on research investigation conducted by Permiso Security. The report reveals the lucrative nature of LLMJacking for cybercriminals; a prominent cyber criminal organisation behind this attack generated $1 million in annualised revenue via a business model involving the reselling of sex chatbots. LLMJacking was first reported earlier this year by Sysdig Threat Research Team.
Data poisoning
In data poisoning attacks, attackers inject malicious or nonsensical data into the training datasets to corrupt the output produced by LLMs. Data is the core of GenAI; training and fine-tuning data must be easily accessible to Cloud GenAI workloads to allow for customisation and contextualisation via Retrieval Augmented Generation (RAG) techniques.
The LLMs provided by CSPs are usually pre-trained and general-purpose. While this drastically reduces the effort and resources needed for training, the responses from the models need to be customised to meet specific use cases required by organisations. However, fine-tuning data is commonly kept in S3 buckets, thus exposing the data to data-poisoning attacks - attackers are familiar with the vulnerabilities related to S3.
Data exfiltration
Similar to the data poisoning attacks, data used by cloud GenAI workloads are attractive targets for exfiltration attacks. There are several reasons attackers would be interested in training data more than other types of data, the primary reason being the need to acquire proprietary information or intellectual property. Access to intellectual property might give a competing organisation a considerable advantage.
Prompt injection
Prompt injection attacks are one of the most commonly discussed threat vectors against LLMs. Attackers implement this attack by crafting malicious prompts that trick LLMs into acting in an unintended way, such as divulging sensitive information.
Prompt injection occurs at the application layer. However, similar to other application security attacks, there are implications and challenges when applications are deployed on the cloud, e.g. to take advantage of rapid scaling and elasticity. With the need to scale GenAI applications, organisations would prefer central management of GenAI applications for the sake of resilience and scalability. This would eventually mean exposing prompt injection attacks to the infrastructure rather than the application layer.
Model attacks
Cloud GenAI workloads might include both provider-managed and customer-managed machine learning (ML) models. Provider-managed models are less vulnerable to attacks since the providers implement extra security measures, and these are the least exposed.
However, customer-managed models are more susceptible to attacks due to higher levels of exposure. Customer-managed models are often stored in container image repositories (e.g. Azure Container Registry), virtual machines, or Kubernetes clusters. Several ML image attack vectors are documented on the MITRE ATLAS, including Backdoor ML Model, Full ML Access, Evade ML Model, and Erode ML Model Integrity. Several of these model attacks were recently demonstrated against SAP AI Core by Wiz research.
Cloud misconfiguration
Most cloud attacks are due to misconfigured cloud resources and customer faults. Naturally, GenAI cloud workloads are not isolated from this menace. Some important aspects are Identity and Access Management (IAM) controls, encryption and secret management. Each type of misconfiguration could have substantial security impacts on GenAI workloads, resulting in successful attacks.
Addressing threats against GenAI workloads
Several threats against GenAI cloud workloads are well known, and mitigation strategies are established. However, many of these threats are AI-specific and require novel mitigation strategies. Let’s examine two of these essential strategies for securing GenAI cloud workloads.
AI Security Posture Management
Several security posture management approaches, including CSPM and DSPM, are used to manage diverse aspects of the cloud. These approaches only partially cover the AI-specific gaps; hence, there is a need for a specific approach: AI Security Posture Management (AISPM).
With AISPM, organisations can quickly check for misconfigured GenAI cloud workloads and ensure that security best practices are implemented. Furthermore, organisations would leverage AISPM to monitor and ensure adherence to regulatory and compliance requirements, such as the EU AI Act.
AI red teaming
Though AISPM allows organisations to implement several security measures for GenAI workloads, it doesn’t meet all security and safety requirements. Responsible AI guidelines require organisations to ensure that GenAI workloads are secure and don’t generate toxic, biased, inappropriate or factually incorrect content (hallucination). AI red teaming is suitable for meeting these requirements via rigorous testing of Gen AI workloads.