Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

Useful Kafka tools

89 Aufrufe

Veröffentlicht am

Quick setup guide for kaf, kafkacat, and Node-RED

Veröffentlicht in: Technologie
  • Als Erste(r) kommentieren

  • Gehören Sie zu den Ersten, denen das gefällt!

Useful Kafka tools

  1. 1. Useful tools
  2. 2. kaf
  3. 3. Install it curl https://raw.githubusercontent.com/infinimesh/kaf/master/godownloader.sh | BINDIR=$HOME/bin bash
  4. 4. Configure it ~/.kaf/config current-cluster: dale clusters: - name: dale brokers: - bootstrap_host:port SASL: mechanism: PLAIN username: token password: API_KEY TLS: cafile: /Users/dalelane/dev/qp/auth/es-cert.pem insecure: false security-protocol: SASL_SSL
  5. 5. Configure it $ kaf config use-cluster dale Switched to cluster "dale".
  6. 6. Configure it $ kaf config use-cluster dale Switched to cluster "dale". $ kaf config select-cluster Use the arrow keys to navigate: ↓ ↑ → ← ? Select cluster: ▸ dale ed tim nic ↓ matt
  7. 7. Produce with it $ echo "Hello World" | kaf produce DEMO
  8. 8. Produce with it $ echo "Hello World" | kaf produce DEMO Sent record to partition 0 at offset 0.
  9. 9. Produce with it $ echo "Hello World" | kaf produce DEMO Sent record to partition 0 at offset 0. $ echo -n "Bonjour le monde" | kaf produce DEMO Sent record to partition 0 at offset 1.
  10. 10. Produce with it $ echo "Hello World" | kaf produce DEMO Sent record to partition 0 at offset 0. $ echo -n "Bonjour le monde" | kaf produce DEMO Sent record to partition 0 at offset 1. $ cat README.md | kaf produce DEMO Sent record to partition 0 at offset 2.
  11. 11. Consume with it $ kaf consume DEMO
  12. 12. Consume with it $ kaf consume DEMO Partition: 0 Offset: 0 Timestamp: 2019-07-11 19:41:10.178 +0100 BST Hello World Partition: 0 Offset: 1 Timestamp: 2019-07-11 19:42:36.479 +0100 BST Bonjour le monde Partition: 0 Offset: 2 Timestamp: 2019-07-11 19:43:29.248 +0100 BST # Ansible ICP Installer Ansible scripts to install IBM Cloud Private and IBM Event
  13. 13. Consume with it $ kaf consume DEMO -f
  14. 14. What else? $ kaf topic describe DEMO Name: DEMO Internal: false Compacted: false Partitions: Partition High Watermark Leader Replicas ISR --------- -------------- ------ -------- --- 0 3 2 [0 1 2] [0 1 2] Config: Name Value ReadOnly Sensitive ---- ----- -------- --------- min.insync.replicas 2 false false retention.ms 604800000 false false message.format.version 2.2-IV1 false false
  15. 15. What else? $ kaf topic create ANOTHER Created topic ANOTHER. $ kaf topic ls NAME PARTITIONS REPLICAS ANOTHER 1 1 DEMO 1 3 $ kaf topic delete ANOTHER Deleted topic ANOTHER.
  16. 16. kafkacat
  17. 17. Install it $ brew install kafkacat
  18. 18. Configure it ~/.config/kafkacat.conf bootstrap.servers=bootstrap_host:port ssl.ca.location=/Users/dalelane/dev/qp/auth/es-cert.pem security.protocol=sasl_ssl sasl.mechanisms=PLAIN sasl.username=token sasl.password=API_KEY
  19. 19. Produce with it $ kafkacat -P -t DEMO
  20. 20. Produce with it $ kafkacat -P -t DEMO Hello World
  21. 21. Produce with it $ kafkacat -P -t DEMO Hello World Bonjour le monde
  22. 22. Produce with it $ kafkacat -P -t DEMO Hello World Bonjour le monde $ kafkacat -P -t DEMO README.md
  23. 23. Produce with it $ kafkacat -P -t DEMO Hello World Bonjour le monde $ kafkacat -P -t DEMO README.md $ kafkacat -P -t DEMO -H mykey=myvalue README.md
  24. 24. Consume with it $ kafkacat -C -t DEMO
  25. 25. Consume with it $ kafkacat -C -t DEMO Hello World Bonjour le monde % Reached end of topic DEMO [0] at offset 2
  26. 26. Consume with it $ kafkacat -C -t DEMO -f 'NEXT MESSAGEn============nheaders:n%hnmessage:n%snn'
  27. 27. Consume with it $ kafkacat -C -t DEMO -f 'NEXT MESSAGEn============nheaders:n%hnmessage:n%snn' NEXT MESSAGE ============ headers: message: Hello World NEXT MESSAGE ============ headers: mykey=myvalue message: Bonjour le monde
  28. 28. What else? $ kafkacat -L Metadata for all topics (from broker -1: sasl_ssl://9.30.249.220:31250/bootstrap): 3 brokers: broker 0 at 9.30.249.220:31203 (controller) broker 2 at 9.30.249.220:32599 broker 1 at 9.30.249.220:30797 1 topics: topic "DEMO" with 1 partitions: partition 0, leader 1, replicas: 1,2,0, isrs: 1,2,0
  29. 29. Node-RED
  30. 30. Install it $ sudo npm install -g --unsafe-perm node-red
  31. 31. Run it $ export CPPFLAGS=-I/usr/local/opt/openssl/include $ export LDFLAGS=-L/usr/local/opt/openssl/lib $ node-red
  32. 32. Add Kafka nodes to it
  33. 33. Consume with it
  34. 34. Consume with it
  35. 35. Consume with it
  36. 36. Consume with it
  37. 37. Consume with it
  38. 38. Consume with it
  39. 39. Produce with it
  40. 40. Produce with it

×