From Dev.to Community
Tech Articles
Curated developer articles, tutorials, and guides � auto-updated hourly
Alex Towell6d ago • 10 min read
Synthesis: Codecs as Structure
The series closes by restating the codes-as-priors thesis across all twelve instances and connecting...
#c#informationtheory#codingtheory#prefixfree
0 0
Alex Towell6d ago • 4 min read
All Induction Is the Same Induction
Solomonoff induction, MDL, speed priors, and neural networks are all special cases of one Bayesian f...
#machinelearning#solomonoffinduction#bayesianinference#informationtheory
0 0
Alex Towell6d ago • 9 min read
McMillan's Converse
Any length vector satisfying Kraft has a prefix-free code. Here is the construction.
#c#informationtheory#codingtheory#prefixfree
0 0
Alex Towell6d ago • 12 min read
Bits Follow Types
Codecs are not ad-hoc bit formats. They are constructions on the algebraic structure of types.
#c#genericprogramming#codecs#informationtheory
0 0
Alex Towell6d ago • 7 min read
VByte / Varint
VByte trades bit-level precision for byte-alignment, and that trade wins in practice. Most productio...
#c#informationtheory#codingtheory#prefixfree
0 0
Alex Towell6d ago • 7 min read
Unary and Elias Gamma
Unary and Elias gamma are the two simplest universal codes. Unary encodes n in n bits; gamma in 2 lo...
#c#informationtheory#codingtheory#prefixfree
0 0
Alex Towell6d ago • 9 min read
Huffman Coding
Given a finite distribution, Huffman's algorithm builds the prefix-free code with minimum expected l...
#c#informationtheory#codingtheory#prefixfree
0 0
Alex Towell6d ago • 7 min read
Rice / Golomb
Rice and Golomb codes are parametric: a single parameter k (or m) tunes the code to a specific geome...
#c#informationtheory#codingtheory#prefixfree
0 0
Alex Towell6d ago • 4 min read
The Beautiful Deception: How 256 Bits Pretend to be Infinity
Cryptographic theory assumes random oracles with infinite output. We have 256 bits. This paper explo...
#cryptography#hashfunctions#randomness#informationtheory
0 0
Alex Towell6d ago • 7 min read
Fibonacci Coding
Fibonacci coding uses Zeckendorf's representation to produce self-synchronizing codewords. Every cod...
#c#informationtheory#codingtheory#prefixfree
0 0
Alex Towell6d ago • 3 min read
Packed Containers: Zero-Waste Bit-Level Storage in C++
What if containers wasted zero bits? A C++ library for packing arbitrary value types at the bit leve...
#c#datastructures#informationtheory#genericprogramming
0 0
Alex Towell6d ago • 7 min read
Elias Delta and Omega
Elias delta and omega extend Elias gamma by recursively encoding the length prefix. Each step yields...
#c#informationtheory#codingtheory#prefixfree
0 0
Alex Towell6d ago • 10 min read
When Lists Become Bits
Prefix-freeness is the property that lifts the free-monoid construction into bit space.
#c#genericprogramming#codecs#informationtheory
0 0
Alex Towell6d ago • 8 min read
Universal Codes as Priors
Every prefix-free code is a hypothesis about the source. The codeword lengths determine an implicit ...
#c#informationtheory#codingtheory#prefixfree
0 0
Alex Towell6d ago • 4 min read
Introduction to Sequential Prediction
The problem of predicting what comes next, from compression to language models
#sequentialprediction#informationtheory#compression#prediction
0 0
Alex Towell6d ago • 9 min read
Kraft's Inequality
Which codeword-length vectors are achievable by prefix-free codes? Kraft's inequality is the answer.
#c#informationtheory#codingtheory#prefixfree
0 0
Alex Towell6d ago • 2 min read
Perfect Hashing: Space Bounds, Entropy, and Cryptographic Security
Space bounds, entropy requirements, and cryptographic security properties of perfect hash functions.
#cryptography#hashfunctions#informationtheory#spacecomplexity
0 0
Alex Towell6d ago • 6 min read
Succinct Bit Vectors and Rank/Select
A bit vector with O(1) rank and O(log n) select using only n + o(n) bits of space. The auxiliary ind...
#c#informationtheory#datastructures#succinct
0 0
Alex Towell6d ago • 7 min read
Arithmetic Coding
Arithmetic coding closes the gap between Huffman's per-symbol integer lengths and true entropy. A si...
#c#informationtheory#codingtheory#prefixfree
0 0