Jafka is a fast and lightweight message queue system that is implemented as a single 271KB JAR file. It uses Zookeeper for coordination and has dependencies on common Java libraries like Log4j and Jackson. Jafka aims to eventually become a full implementation of Apache Kafka with features like persistence, high throughput processing of millions of messages per second, load balancing and a simple message format. It currently focuses on providing basic queue functionality through a simple producer/consumer model.
5. Future
● A full kafka clone (http://incubator.apache.org/kafka)
● Persisten messaging with O(1)
● Constant time performance even with TB messages
● High-throughput (100k/s)
● Distributed messaging system
– producers/brokers/consumers
● Auto load balancing
● Simple message format
jafka mq 5
11. Offset length limit
● Long.MAX_VALUE
– 9223372036854775807 → 9*1018
● If one message size is 1k
– Messages: 9*1015
– If producing 1 billion messages per day
● Continuing days: 9*106
● Continuing years: 24657
jafka mq 11
12. About Filesystem
● 6 7200rpm SATA RAID-5 *
– 300MB/s linear writes (300k/s 1KB per message)
– 50k/s random writes
jafka mq 12
http://incubator.apache.org/kafka/design.html
13. sendfile() (1)
● Common data transfer
1. OS reads data from disk to pagecache
2. Application reads data from pagecache to buffer
3. Application writes data back into socket buffer
4. OS copies data from socket buffer to NIC buffer
● 4 copies + 2 system_call
jafka mq 13