Bumblebee vs OSV-Scanner: Two Takes on Supply Chain Scanning

When the supply chain bites

The last two years have been rough on the ecosystem. The XZ Utils backdoor (CVE-2024-3094) was the headline, but it's the steady drip of smaller npm worms, hijacked maintainer accounts, and typosquats that really wears you down. I've personally had to triage three "did we just install the bad version?" incidents in the past year alone. None of them ended badly. The muscle was tired by December.

Most of us reach for the same tools when that pager fires: npm audit, osv-scanner, maybe Snyk if your shop pays for it. They're great at telling you what CVEs your package-lock.json is exposed to. They're less great at answering the question you actually have at 2am: did the bad package ever land on this machine, even briefly?

That's the angle bumblebee, a new tool from Perplexity, seems to be taking. According to its README, it's a read-only scanner that walks your on-disk package, extension, and developer-tool metadata to check exposure against known supply-chain compromises. I haven't run it long enough in production to give you a battle-tested take, but the framing is interesting enough that it's worth comparing against the tools you already have.

What Bumblebee does differently

The mental model I've settled on: most SCA tools look forward (what's in my lockfile right now?), while bumblebee looks backward (what did my machine actually touch?).

• osv-scanner reads manifest files (package.json, go.mod, etc.) and matches them against advisories in osv.dev.
• npm audit does the same, scoped to npm.
• Bumblebee, per its description, walks the disk — installed packages, editor extensions, dev tool caches — to detect whether known-bad artifacts ever landed.

The use case isn't redundant. You can pass an npm audit clean check and still have a compromised version cached in ~/.npm, or a malicious VS Code extension sitting in ~/.vscode/extensions that you forgot about. After a real incident, what you want is a forensic sweep of the actual filesystem, not a re-read of your manifest.

Side-by-side comparison

These aren't really competitors — they answer different questions. The mistake is treating SCA-in-CI as the whole answer.

A quick OSV-Scanner workflow

OSV-Scanner remains my default for project-level scanning. It's free, multi-ecosystem, and the data quality from osv.dev has been solid in my experience.

# Install via Go, or grab a binary from the GitHub release page
go install github.com/google/osv-scanner/cmd/osv-scanner@latest

# Scan a project — reads package-lock.json, go.sum, requirements.txt, etc.
osv-scanner --recursive .

# Or scan an SBOM directly if you generate one in CI
osv-scanner --sbom=sbom.spdx.json

The output points at GHSA/OSV IDs and affected version ranges. Solid for CI gates.

Bumblebee, based on the README

The project is fresh, so I want to be careful here — I'm describing what the README says, not what I've benchmarked. The framing on the GitHub page is "read-only developer endpoint scanner for on-disk package, extension, and developer-tool metadata."

A sensible workflow looks roughly like this:

# Clone the repo per the project's README
git clone https://github.com/perplexityai/bumblebee
cd bumblebee

# Run the scanner — check the README for current flags, the project is moving
# The key design property is "read-only": no writes, no filesystem mutations

I'm intentionally not inventing CLI flags I haven't verified. Check the project README for current usage — it's been changing.

The "read-only" part is the design choice I like most. After an incident, the last thing you want is a scanner that mutates filesystem state and confuses the forensic timeline. Compare this with some "remediation" tools that auto-bump versions — useful in normal triage, but not what you want during the first hour of an incident.

Migrating your incident-response playbook

If your current "did this hit us?" runbook leans entirely on npm audit + osv-scanner, here's the shift worth considering.

Before:

# Old playbook: re-read the manifest, hope it tells the truth
osv-scanner --recursive .
npm audit --json | jq '.vulnerabilities'

After:

# Manifest scan still gates CI
osv-scanner --recursive .

# Add a filesystem sweep on dev machines and shared runners
# (refer to the bumblebee README for the current invocation)
bumblebee scan ~

The point isn't to drop your existing tooling. It's that "manifest scan" and "filesystem sweep" answer different questions, and after a real incident you usually want both.

When to use which

• CI gating, PR checks: OSV-Scanner or Snyk. Manifest-based, fast, deterministic.
• Pre-install protection on npm/PyPI: Socket. It looks at install-time behavior signals (install scripts, network access, etc.).
• Post-incident filesystem triage: Bumblebee looks like the right fit, if it pans out in practice. I'd add it to the incident response toolkit and see how it compares during the next advisory.
• One-off quick check: npm audit still does the job for npm-only projects, and it's already on every dev's machine.

I haven't tested bumblebee thoroughly enough to recommend it as a primary tool. But the design intent — read-only, on-disk, focused on known-bad artifacts — fills a gap that manifest scanners genuinely don't cover.

Bottom line

The supply chain threat model shifted when attackers started targeting maintainer accounts and post-install scripts directly. Manifest scanning catches what your lockfile declares; it doesn't catch what landed on disk a month ago and got cleaned up after the worm spread.

Bumblebee's read-only, on-disk approach plugs that gap if the implementation holds up under load. Don't migrate away from osv-scanner — layer this on top. The cost of running an extra read-only sweep during incident response is basically zero. The upside on a bad day is large.