rzmq 0.5.24: crates.io | Github - Async, Pure Rust ZeroMQ with Leading Performance on Linux
After some experimenting, I found that io uring was maxed out on a single thread meaning multiple connections could saturate it, so it's been a long time coming, io_uring sharded workers is here! Yes, rzmq's io_uring is no longer one single thread, it's multiple threads now and meets or beats Tokio in throughput.
I knew there was always more throughput on the table because Tokio could do it and I always had a plan to have multiple workers. It's why rzmq is designed to have a worker than can be spawned with its own mechanics. Sharding it did not require any major reworking of anything, just needed to get routing and spawn multiple workers and boom. rzmq was always going to go this route, but the question was when.
17-18GB/s is the max for this TCP loopback on this efficiency laptop as far as it's going to go. It will be time to go to AWS, DigitalOcean, etc. and bench there to see what kinds of LAN/WAN numbers we can put out.
Inproc got a massive speed boost as well, reaching to 230GB/s PUSH/PULL. Personally, I don't use it. Not sure why anyone would use it. fibre is right there, use that instead. I guess if you wanted an abstraction to distribute data on local and other servers, you could technically have pull on the same process and other servers and then distribute data that way, sure. Yeah, I don't know it's a niche thing. It showed that I never really cared for it until I started using it for testing socket features for consistent behaviors. IPC? sidecar processes I guess.
Since rzmq 0.5.20, xsmb has gotten a speed boost I think, my deployment software deploys very quickly. Yes, it transfers data over DEALER/ROUTER using XSMB Streaming Upload rather than HTTP uploads now. Also making better use of XSMB's Push messages to push from server to client for bidirectional communication instead of using PUB/SUB like I was before.
All tests are passing, some were flaky because some were legit issues while some just needed to be split out and more focused.
rzmq 0.5.24 is more stable, more correct than 0.5.20. It offers flexible simultaneous tokio and io-uring (per socket opt in) with NULL/PLAIN/CURVE/NOISE-XX over both standard and io-uring.
rzmq is an ongoing effort to deliver the LEADING edge performance on Linux. rzmq exists only because XSMB exists, as long as XSMB exists, rzmq will exist.
Of course I always have to mention, fibre has been updated to 0.6.0. fibre exists because rzmq exists, it's primary purpose is for async-tokio to sync io-uring and vice-versa. It is also used in many other of my projects now.
fibre is one of the few full suite High Perf multimode sync/async channels in rust lang. It offers MPMC, MPSC, SPMC, SPMC Topic based, SPSC, and One-Shot channels.
Fibre's Bounded (Cap 128) Async MPSC throughput is probably the worlds most consistent and fastest. kanal can't beat it when it comes to contentious loads, even though beating kanal wasn't the purpose. Previous Fibre MPSC impls in the past had scaling issues that I couldn't figure out until now. I accept that fibre's 1 - 1 MPSC is bad against other channels, but that's not what an MPSC is for.
fibre also provides a Sync/Async Mutex and RW Lock now. I might add an MPMC Broadcast some day.