Epoll vs. io_uring in Linux

> But my students weren’t as happy as I was - they wanted to build something genuinely useful, and they were really disappointed that our “product” had strong architectural limits and couldn’t outperform titans like nginx and haproxy.

I took a (very brief) look at the github repo [1], it doesn't look like you're doing anything with cpu pinning.

You can probably eke (thanks) out a bit more performance if you cpu pin your threads and cpu pin your listen sockets (sockopt SO_INCOMING_CPU).

If you also cpu align your outgoing sockets, you should get a significant boost, but afaik, there's no great api for that. Linux does have an api for compatible NICs (traffic steering/flow steering) which can work, but if you know what hash your NIC uses (it's probably toeplitz) and you manage source port selection to your backend, you can pick ports that will hash properly.

The goal is for your proxy to be able to handle packets without any cross cpu communication.

[1] https://github.com/sibexico/TinyGate

Take a look at https://github.com/concurrencykit/ck and https://github.com/microsoft/mimalloc, it will fit well for a zero-copy and mem aligned reverse proxy. Also, if you want to add a DDoS protection and more advanced L4 stuff check out https://docs.ebpf.io/ebpf-library/libxdp/libxdp/

Yes, io_uring is significantly faster than epoll (I think I had like 20% faster req/s with io_uring.) The catch is that its kernel opt-in and disabled just about everywhere for security reasons. I think that it has direct memory sharing between the kernel and user-land which is kind of yikes. There's been multiple exploits that hit io_uring in recent times. It's because of this that even engineering projects that try to reach the highest performance possible (like Go) don't really bake io_uring in as a sane default. Though if you want to take the risk you can always run it yourself for your favourite language. It is faster but the cost is possible exploits.

Boost asio if you love C++ and asynchronous networking.

The author takes a very benchmark focus on this topic which only says part of the story particularly for complex systems. Noticed that there are a number of very similar interface that exist on other platform like windows long before io_uring, but that does make Linux’s I/O system worse or slow than these platforms. A fast server is likely fast in either multiplexing or async API if implemented correctly in almost all cases.