The AI Cyberthreat Challenge: How Strong Are Your Security Controls? 

Generative AI is transforming many industries through automating processes that were once time consuming and complex. 65% of organizations surveyed by McKinsey&Co in 2024 are using generative AI — more than double last year’s figures. This rapid adoption of AI can mean that organizations may not have the proper controls in place to protect their data. 

This article will identify AI vulnerabilities that could impact your business and provides controls and best practices to secure against these threats. 

AI Behaviour and Vulnerabilities: 

There are many ways companies can use AI in their everyday practices, here are some examples and vulnerabilities that may apply to your organization. 

1. Content Creation and Marketing: generative AI tools are used to create blog posts and social media content. 

Vulnerabilities: 

• Data Breaches: Sensitive data used in AI tools could be exposed if the platform is compromised. 
• Intellectual Property Theft: AI generated content could be stolen or misused if proper safeguards are not in place. 
• Misinformation: AI tools could inadvertently generate inaccurate or misleading content, damaging client reputations. 

2. Customer Support: E-commerce stores implement chatbots to handle customer inquiries 24/7. 

Vulnerabilities: 

• Data Privacy: AI chatbots handling customer data could be targeted, leading to leaks of personal and financial information. 
• Unauthorized Access: Weak authentication mechanisms might allow attackers to gain control of AI systems and access customer databases. 
• AI Manipulation: Attackers could manipulate AI algorithms to provide false responses or redirect customers to malicious sites. 

3. Healthcare Services: Clinics can use AI-powered tools to analyze patient data and generate reports to assist in quickly diagnosing and treating conditions and symptoms. 

Vulnerabilities: 

• Patient Data Exposure: AI systems processing medical data are prime targets for cyberattacks aiming to steal sensitive health information. 
• Misdiagnosis: Cyberattacks aimed at manipulating AI models can lead to incorrect diagnoses and treatment recommendations. 
• Regulatory Non-Compliance: Failure to secure AI systems could result in breaches of healthcare regulations like HIPAA, leading to legal and financial penalties. 

4. Financial and Tax Services: AI can automate the generation of financial reports, detect anomalies, and predict cash flow. Popular financial products are now including offerings with AI capabilities that can provide real-time insights and automate routine tasks. 

Vulnerabilities: 

• Financial Data Theft: AI systems processing financial transactions and reports are attractive targets for attackers seeking sensitive financial data. 
• Fraud: AI systems could be exploited to create fraudulent transactions or financial documents. 
• System Manipulation: Attackers could manipulate AI models to generate false financial predictions and reports. 

5. Retail and E-commerce: Product recommendations based on browsing history and purchase behaviour can be done through AI. 

Vulnerabilities: 

• Customer Data Breaches: AI systems handling customer purchase history and preferences are vulnerable to data breaches. 
• Phishing Attacks: AI-generated personalized emails could be intercepted or manipulated to conduct phishing attacks. 
• Algorithm Manipulation: Attackers could manipulate recommendation algorithms to promote malicious products or content. 

6. Human Resources: AI can be used for resume screening and to generate job descriptions. 

Vulnerabilities: 

• Candidate Data Exposure: AI systems processing resumes, and personal information are vulnerable to data breaches. 
• Discrimination: Bias in AI algorithms could lead to discriminatory hiring practices, exposing the agency to legal risks. 
• Automated Decision Manipulation: Attackers could manipulate AI-driven decisions, leading to the selection of unsuitable candidates. 

7. Real Estate: AI can be used to generate property listings, market analysis reports, and personalized property recommendations for clients. AI tools can also assist in automating communication with potential buyers and sellers. 

Vulnerabilities: 

• Client Data Breaches: AI systems accessing client property preferences and financial details could be targeted. 
• Property Fraud: AI-generated property listings could be manipulated to create fraudulent listings. 
• Market Analysis Tampering: Attackers could manipulate AI models to generate false market trends and property valuations. 

Malicious Actors and AI Threats: 

Malicious actors are increasingly leveraging AI to enhance the scale, sophistication, and impact of their cyberattacks. Here are some of the ways AI is being used to launch cyber threats: 

Personalized Attacks: AI can analyze social media and other online data to craft highly personalized phishing messages, increasing the likelihood of success 

Voice and Video Manipulation: AI can create realistic deepfake videos and audio recordings that can be used to impersonate executives or other trusted individuals. These deepfakes can be used to manipulate financial transactions, steal sensitive information, or damage reputations. 

Ransomware Optimization: AI can be used to identify the most valuable targets within an organization, such as those with sensitive data or critical systems, and launch ransomware attacks that maximize potential payoffs. 

Efficient Scanning: AI can automate scanning for vulnerabilities across many systems, identifying weaknesses that can be exploited for unauthorized access or data breaches. 

By understanding how AI is being used as persistent cyberthreats, organizations can better prepare and strengthen their cybersecurity controls to defend against these advanced tactics. 

Aligning with Security Frameworks 

The best way to ensure your AI security controls are up to date is by aligning with established security frameworks. Frameworks such as NIST (National Institute of Standards and Technology) and SOC2 (Service Organization Control 2) provide comprehensive guidelines that help organizations implement robust security measures. These frameworks are continually updated to address emerging technologies, including regenerative AI. 

Frameworks Addressing AI Controls: 

NIST AI Risk Management Framework (AI RMF): 

The NIST AI RMF provides guidelines specifically for managing risks associated with AI technologies. It includes principles and practices for trustworthy AI, emphasizing transparency, accountability, and fairness. 

Key Elements: 

• Governance: Establishing policies and procedures to oversee AI deployment. 
• Data Management: Ensuring data quality and integrity used in AI systems. 
• Monitoring and Maintenance: Continuous monitoring of AI systems for anomalies and updating models as needed. 
• Security and Privacy: Implement robust security measures to protect AI systems and the data they process. 

SOC 2: 

SOC 2 primarily focuses on the controls relevant to the security, availability, processing integrity, confidentiality, and privacy of data. Although not AI-specific, its principles can mitigate AI-related risks:  

• Security: Implement access controls and intrusion detection to protect AI systems. 
• Availability: Monitor AI systems continuously and have disaster recovery plans. 
• Processing Integrity: Validate data and manage changes to AI models. 
• Confidentiality: Encrypt data and use data masking techniques. 
• Privacy: Enforce data privacy policies and obtain user consent. 

ISO/IEC 27001: 

While not exclusively focused on AI, ISO/IEC 27001 provides a comprehensive framework for information security management. Its principles can be applied to AI systems to ensure they are secure and resilient. 

Key Elements: 

• Risk Assessment: Identifying and assessing risks related to AI technologies. 
• Access Control: Implementing strict access controls to AI systems and data. 
• Incident Response: Establishing procedures for responding to security incidents involving AI. 

Compliance as a Service Platforms: 

• Implementing compliance solutions can ensure your AI risks are continuously monitored and tested in accordance with these latest frameworks. 

ProtechSuite’s risk management and internal controls modules allow organizations to identify, test and track their AI vulnerabilities ensuring they are effectively managed. 

Steps to Mitigate AI Risks 

If your organization is not yet aligned with a security framework, here are some actionable steps to manage risks associated with regenerative AI: 

1. Risk Assessment:  

• Identify and evaluate the potential risks associated with AI in your organization. Identify critical assets, potential threats, and vulnerabilities. 

2. Implement Strong Access Controls:  

• Ensure that only authorized personnel have access to AI systems and sensitive data. Use multi-factor authentication and role-based access controls. 

3. Ensure Data Quality and Integrity:  

• AI systems rely heavily on data. Implement measures to ensure data used in AI models is accurate, complete, and protected from tampering. 

4. Regularly Update and Monitor AI Systems:  

• AI models should be regularly updated to address new threats and improve performance. Continuous monitoring helps detect and respond to anomalies in real-time. 

5. Educate and Train Staff:  

• Provide training for employees on the risks associated with AI and best practices for mitigating these risks. Employees are usually the first line of defence when it comes to cyberattacks. 

6. Establish Governance and Accountability:  

• Create and regularly update clear policies and procedures for the use of AI within your organization. Assign responsibility for AI governance to ensure compliance with ethical and legal standards. 

Updating your security controls to address the risks associated with regenerative AI is essential for protecting your organization’s data and maintaining trust with stakeholders. Aligning with established security frameworks or taking proactive steps to mitigate your risks will help safeguard your organization and its AI use. 

ProtechSuite  provides an all-in-one compliance solution with these frameworks already embedded. Explore how ProtechSuite can simplify your compliance journey with a free trial or demo: https://j-sas.com/pricing/