AI Coding Tools Introduce New Security Risks for UK Businesses
0
0
Most development teams have at least one person using Copilot, Cursor, or a similar AI coding assistant day-to-day. These tools speed things up, but they also generate code that can carry real security problems. Let’s find out how these vulnerabilities creep in and what your team can do to catch them before they reach production.
When AI Suggestions Become Attack Surfaces
AI models generate code based on patterns from training data. They don’t understand context the way a senior developer does, and they don’t know whether a package they recommend actually exists or has been compromised since they were trained.
Research published in 2025 found that AI-assisted commits introduce hardcoded secrets at roughly double the rate of human-written commits, and that AI-generated code contains significantly more high-severity vulnerabilities overall. These are not edge cases, they are patterns that emerge consistently across different models and codebases.
The three most common issues that come up in practice are:
- Hallucinated dependencies: The model suggests a package name that doesn’t exist. If an attacker registers that name on npm or PyPI before you catch it, anyone who installs it ends up running their code.
- Hardcoded secrets: API keys, database credentials, and tokens get embedded directly in source files. This happens more than most teams realise, especially in quickly iterated codebases.
- Insecure defaults: AI-generated code often skips input validation, uses deprecated cryptographic functions, or opens up permissions that should be restricted. These issues tend to be subtle and easy to miss in review.
Hallucinated package names being registered by malicious actors is a credible and growing risk, not a distant hypothetical. Researchers have demonstrated proof-of-concept attacks, and suspected real-world cases have emerged, though confirmed and attributable exploitation at scale has not yet been widely reported.
Security researchers have warned that hallucinated names are common, repeatable, and semantically plausible, making them a predictable target for attackers to monitor and act on.
Why Code Review Won’t Catch Everything
Traditional code review helps, but it wasn’t designed to catch AI-specific failure modes. A reviewer scanning a pull request might not question a package name if it looks plausible, and hardcoded secrets are often added during quick prototyping sessions and never removed.
Static analysis tools can flag some of these issues, but they miss the bigger picture. They won’t tell you whether a dependency is hallucinated, and they can’t assess the combined impact of multiple small weaknesses across your stack.
This is where penetration testing becomes relevant. A structured test against a codebase that includes AI-generated components will probe for exactly these kinds of weaknesses, examining how attackers could exploit misconfigured permissions, exposed secrets, or vulnerable dependencies together.
What to Do Before AI Code Ships
There are practical steps teams can take alongside any formal security testing.
- First, treat every AI-generated package suggestion as unverified until you’ve confirmed it exists and checked its download history and maintainer activity. A package with zero downloads and a one-day-old registration is a clear warning sign.
- Second, run secret scanning tools like GitGuardian or TruffleHog as part of your CI/CD pipeline. These catch hardcoded credentials before they reach a remote repository.
- Third, review AI-generated code with security in mind, not just functionality. Ask whether input is validated, whether permissions are scoped correctly, and whether any authentication logic was generated by the model rather than written deliberately.
- Fourth, UK businesses should familiarise themselves with NCSC guidance on secure development and deployment, and consider whether Cyber Essentials or Cyber Essentials Plus certification is appropriate for their organisation.
Both schemes cover network boundary controls and malware protection that are relevant when AI-assisted code is part of your stack, and certification is increasingly expected by public sector clients and larger enterprise partners.
Closing Message
AI coding tools aren’t going away, and for most teams the productivity benefits are genuine. But speed is only an advantage if the code you’re shipping is sound. Hallucinated dependencies, hardcoded secrets, and insecure defaults are patterns that appear regularly in AI-assisted codebases, and they’re exactly what attackers look for. Teams that catch these issues early build validation into their process rather than leaving it to chance.
Artificial Intelligence – The Data Scientist