Fedify: an ActivityPub server framework
@fedify@hollo.social · Reply to Max's post
@PossiblyMax Great question about our queue implementation! Fedify doesn't actually create separate physical queues, but rather uses a single logical queue where each message contains its own destination information.
For resource management, we generally rely on the underlying queue implementation (Redis, PostgreSQL, etc.) to handle concurrent processing efficiently. Since version 1.0.0, we've introduced ParallelMessageQueue
which processes multiple messages concurrently with a configurable worker count—usually set close to your CPU core count for IO-bound operations.
We don't spin up new queues dynamically; instead, we focus on making the message processing scalable. You can control the parallelism level when using ParallelMessageQueue
, and for high-volume instances, you can horizontally scale by running multiple worker processes that connect to the same shared queue backend.
This approach keeps the architecture simpler while still allowing for good throughput and resource utilization that can scale with your instance size.