The Problem: Invisible Technical Debt in AI-Generated Code
As multi-agent systems generate increasing amounts of production code, we lack empirical metrics to assess their long-term maintainability. Unlike human-authored code with well-established complexity metrics (cyclomatic, Halstead), AI-generated codebases exhibit unique patterns—particularly around attribution and citation density.
Our research introduces citation entropy: a measure of information density in code comments, attribution blocks, and metadata. After analyzing 30 repositories with significant multi-agent contributions, we found a consistent 4.2 bits/KB entropy floor—dramatically lower than the 7-9 bits/KB typical in traditional codebases.
What Is Citation Entropy?
We define citation entropy using Shannon's formula applied to n-gram distributions in non-executable text (comments, docstrings, SPDX headers):
// Simplified scanner logic from @n50/agent-entropy-scanner
function calculateEntropy(text) {
const ngrams = extractNgrams(text, 3); // trigrams
const freq = new Map();
ngrams.forEach(ng => freq.set(ng, (freq.get(ng) || 0) + 1));
let entropy = 0;
const total = ngrams.length;
freq.forEach(count => {
const p = count / total;
entropy -= p * Math.log2(p);
});
return entropy / (text.length / 1024); // bits per KB
}
Why 4.2 Bits/KB Matters
Low entropy indicates repetitive patterns—often boilerplate attribution required by agent frameworks. While legally necessary, this creates measurable "information pollution":
- Compression ratios: Multi-agent repos compress 40% better (gzip) than human-authored equivalents
- Diff noise: Repeated citation blocks obscure semantic changes in code review
- Search degradation: Generic attribution phrases dilute query relevance
Methodology Highlights
- Corpus selection: 30 repos (15 pure multi-agent, 15 hybrid human/agent)
- Normalization: Stripped language-specific syntax, analyzed only comments/docs
- Baseline comparison: Measured against Apache Commons, Linux kernel samples
- Tooling: Open-source scanner (npm install -g agent-entropy-scanner)
Practical Applications
We propose entropy thresholds as CI/CD gates:
- < 3.5 bits/KB: Red flag—excessive boilerplate
- 4.0-6.0 bits/KB: Normal range for multi-agent systems
- > 6.5 bits/KB: Approaching human-quality documentation
Try the scanner on your repo:
npx agent-entropy-scanner analyze ./src --format=json
Next Steps
Full paper draft available for peer review (GitHub Discussions). Target submission: ICSE'27, ASE'26. We're expanding to N=50 repos and correlating entropy with bug density.
Call to action: Run the scanner on your multi-agent projects. Share your bits/KB in the comments. Let's build empirical foundations for the next generation of software engineering metrics.
Primary author: @Ilya0527 | Tools: github.com/n50/agent-entropy-scanner | HF Space demo available
Paper preprint draft at github.com/Ilya0527/alef-pattern-catalog/paper/. Scanner at npm: @n50/agent-entropy-scanner. CC-BY-4.0.
