Security Bug Report

AI-suggested Python package that never existed: catching slopsquatting in CI

A pull request added a dependency that an AI assistant had suggested. The package name did not exist when the code was written, and a few weeks later someone else published it. We caught it in review by luck, then added registry checks so we do not rely on luck.

Ryan Pham Ryan Pham @ryan_bugs · June 11, 2025 · 7 min read · 8 Views
Banner: AI-suggested dependency that did not exist and was later registered by someone else

A developer opened a pull request for a small script that parses AWS cost reports. CI failed at install time:

ERROR: Could not find a version that satisfies the requirement awscur-parser==0.4.2 (from versions: none)
ERROR: No matching distribution found for awscur-parser==0.4.2

The developer said the import and the usage had come from an AI coding assistant, along with a confident explanation of the library's API. They removed the version pin, CI passed on the next push, and the PR went up for review.

The reviewer noticed something odd: the package had appeared on PyPI eleven days earlier, version 0.0.1, no repository link, no description beyond a single line. That was the moment we stopped and looked properly.

Reproducing it

We asked the same assistant, in a fresh session, for "a Python library to parse AWS CUR files" a dozen times. It suggested the same non-existent name four times, with slightly different made-up APIs. Other people asking similar questions would get similar suggestions, which is what makes a hallucinated name worth registering for an attacker.

What we ruled out

  • A typo of a real package. No popular package had a similar name; this was not classic typosquatting.
  • An internal package. Our private index had nothing like it.
  • A legitimate new project. The published package had no source repository, a single maintainer account created the same week, and an install hook.

Root cause

AI code assistants generate plausible-looking package names that sometimes do not exist. Anyone can register such a name on a public registry and publish code under it; installs then run whatever the package does, including code at install time. The term "slopsquatting" is used for this. Our pipeline trusted any name that resolved, and the developer's natural reaction to "version not found" was to loosen the pin until something installed.

What is happening under the hood

Language models produce text that is likely given their training data. Package names follow strong patterns (aws-, -parser, -utils), so a model can produce a name that fits the pattern perfectly without any such package existing. Studies of code models have measured non-trivial rates of non-existent package suggestions, and the same invented names recur across sessions, which makes them predictable.

Public registries like PyPI and npm are first come, first served for names. Packages can run code during install (for example, through setup.py on source distributions, or postinstall scripts on npm). So a malicious package does not even need to be imported to do damage; installing it in CI is enough.

Environment

Component Version / value
Language Python 3.12
Tooling pip 25.0, pip-tools for lockfiles
CI GitHub Actions, Ubuntu runners with cloud credentials for integration tests
Registries Public PyPI plus a private index
Assistant IDE-integrated AI coding assistant

Minimal reproduction

There is nothing to reproduce safely in code; the pattern is:

1. Assistant suggests: import awscur_parser  /  pip install awscur-parser
2. Name does not exist -> install fails
3. Developer loosens the pin or retries later
4. Someone registers the name -> install now succeeds and runs their code

The fix

We did the immediate cleanup first: removed the dependency, rotated the credentials available to that PR's CI run (it had run once with the package installed), reviewed the runner logs, and reported the package to the PyPI security team. Then we put controls in the pipeline.

1. Every new dependency is checked against the registry's metadata. A small CI step runs when lockfiles change:

import datetime as dt
import sys
import requests

MIN_AGE_DAYS = 60

def check(name: str) -> list[str]:
    r = requests.get(f"https://pypi.org/pypi/{name}/json", timeout=10)
    if r.status_code == 404:
        return [f"{name}: not on PyPI"]
    data = r.json()
    problems = []
    uploads = [f["upload_time_iso_8601"] for files in data["releases"].values() for f in files]
    if uploads:
        first = min(dt.datetime.fromisoformat(u.replace("Z", "+00:00")) for u in uploads)
        age = (dt.datetime.now(dt.UTC) - first).days
        if age < MIN_AGE_DAYS:
            problems.append(f"{name}: first release {age} days ago")
    urls = data["info"].get("project_urls") or {}
    if not any("github.com" in u or "gitlab.com" in u for u in urls.values()):
        problems.append(f"{name}: no source repository link")
    return problems

if __name__ == "__main__":
    issues = [p for name in sys.argv[1:] for p in check(name)]
    print("\n".join(issues))
    sys.exit(1 if issues else 0)

A failure does not block forever; it requires a named approver from the platform team to add the package to an allowlist file.

2. Hash-pinned lockfiles. pip-compile --generate-hashes and pip install --require-hashes mean a package cannot change underneath us, and nothing gets installed that is not in the lock.

3. No secrets in dependency install steps. Installing dependencies now runs in a job without cloud credentials. Integration tests that need credentials run in a later job that uses the cached environment.

4. A note in our AI assistant guidelines: treat package names from an assistant like a stranger's recommendation. Search the registry and the source repository before adding it.

Alternatives and trade-offs

  • Private registry proxy with an allowlist. Strongest control; every new package needs approval. More friction, best for regulated teams.
  • Commercial supply chain scanners. Flag new, low-reputation or malicious packages. Useful in addition to basic age and source checks.
  • Ban AI assistants. Does not address the underlying risk, which applies to blog posts and forum answers too.

How to test the fix

We opened test PRs adding a non-existent name, a package published last week, and a well-known package. The first two failed the check with clear messages; the third passed. We also confirmed --require-hashes fails when a lockfile entry is edited by hand.

Confirming in production

  • In the first two months, the check flagged 9 dependency additions: 6 legitimate young packages that were approved after review, 2 names that did not exist (both from AI suggestions), and 1 abandoned package without a repository that we replaced.
  • No dependency install job has access to cloud credentials anymore, confirmed by an audit of workflow permissions.

Regression guard

  • The check is a required status on all repositories with Python or Node lockfiles (an npm version queries the npm registry the same way).
  • The allowlist file has code owners.

Security implications

This is a supply chain issue first. The credentials exposed to the one CI run were short-lived and scoped, and logs showed no unexpected use, but we rotated them anyway. The broader lesson is that installing a package is executing code, so install steps must run with the least privilege possible.

Prevention checklist

  • Verify that every new package exists, has history and a source repository.
  • Use hash-pinned lockfiles and install with hash checking.
  • Run dependency installs without secrets.
  • Require an approval for young or unknown packages.
  • Teach the team that AI-suggested names need the same scrutiny as any untrusted source.

References

Add a label

#python #dependencies #supply-chain #pypi #ai-coding
Written by
Ryan Pham @ryan_bugs

I build and run the platforms other engineers ship on: CI/CD pipelines, Kubernetes clusters, GPU inference services, LLM features in production, and the cloud bills that come with all of it. I have also spent years writing and operating automated trading systems, where a rounding bug or a missed websocket message is not a log line but a position you did not intend to hold. Each post here starts from a real failure: the alert, the exact error, what we ruled out, the root cause, and how we proved the fix. Same path every time: Investigate, Explain, Fix, Verify. No hype, no price predictions, just engineering.

Advertisement

Comments

0

No comments yet. Be the first to comment!

Sign In to post a comment.