Apache Kafka
Data Products, Data Contracts, and Change Data Capture
Change data capture is a popular method to connect database tables to data streams, but it comes with drawbacks. The next evolution of the CDC pattern, first-class data products, provide resilient pipelines that support both real-time and batch processing while isolating upstream systems...
Introducing Confluent Cloud Freight Clusters
Learn how the latest innovations in Kora enable us to introduce new Confluent Cloud Freight clusters, which can save you up to 90% at GBps+ scale. Confluent Cloud Freight clusters are now available in Early Access.
Contributing to Apache Kafka®: How to Write a KIP
Learn how to contribute to open source Apache Kafka by writing Kafka Improvement Proposals (KIPs) that solve problems and add features! Read on for real examples.
Diagnose and Debug Apache Kafka Issues: Understanding Increased Consumer Rebalance Time
Rebalancing comes into play in Apache Kafka® when consumers join or leave a consumer group. In either case, there is a different number of consumers over which to distribute the partitions from the topic(s), and, so, they must be redistributed and rebalanced....
Diagnose and Debug Apache Kafka Issues: Understanding Increased Request Rate, Response Time, and/or Broker Load
It can be easy to go about life without thinking about them, but requests are an important part of Apache Kafka; they form the basis of how clients interact with data as it moves into and out of Kafka topics, and, in certain cases, too many requests can have a negative impact on your brokers...
Learn How to Secure Your Apache Kafka Cluster Automatically
This blog post explores the need to implement security for your Apache Kafka® cluster, then briefly reviews the security features and advantages of using Confluent Cloud.
Diagnose and Debug Apache Kafka Issues: Understanding Reduced Message Throughput
When you encounter a problem with Apache Kafka®—for example, an exploding number of connections to your brokers or perhaps some wonky record batching—it’s easy to consider these issues as something to be solved in and of themselves...
Motion in Motion: Building an End-to-End Motion Detection and Alerting System with Apache Kafka and ksqlDB
Learn how to build an end-to-end motion detection and alerting system using Confluent Cloud, Kafka clusters, and ksqlDB.
Getting Started with Schemas and Schema Registries
Announcing the complete guide to Schema Registry. In our course, you’ll learn the basics, like how schema registry works, key concepts, how to manage schemas, and more!
Designing Events and Event Streams: Introduction and Best Practices
Properly designing your events and event streams is essential for any event-driven architecture. Precisely how you design and implement them will significantly affect not only what you can do today
What’s New in Apache Kafka 3.3
We are proud to announce the release of Apache Kafka® 3.3 on behalf of the Apache Kafka community. The 3.3 release contains many new features and improvements. This blog post […]
Event-Driven Microservices with Python and Apache Kafka
Our journey with microservices, so far… Back in the days before microservices, we used to build applications that were often hosted in application servers such as WebLogic or WebSphere (apologies […]
Keeping Multiple Databases in Sync Using Kafka Connect and CDC
Microservices architectures have now been widely adopted among developers, and with a great degree of success. However, drawbacks do exist. Data silos can arise where information processed by one microservice […]
Real-Time Gaming Infrastructure for Millions of Users with Apache Kafka, ksqlDB, and WebSockets
The global video game market is bigger than the music and film industry combined. It includes Triple-A, casual/mid-core, mobile, and multiplayer online games. Various business models exist, such as hardware […]
Getting Started with the KRaft Protocol
What is KRaft, and how does it work? Apache Kafka® Raft (KRaft) is the consensus protocol that was introduced to remove Apache Kafka’s dependency on ZooKeeper™ for metadata management. This […]
An Introduction to Apache Kafka Security: Securing Real-Time Data Streams
The largest companies in the world use Apache Kafka® for their real-time streaming data pipelines and applications. Kafka is the basis for the real-time fraud text alerts from your bank […]
Real-Time Wildlife Monitoring with Apache Kafka
Wildlife monitoring is critical for keeping track of population changes of vulnerable animals. As part of the Confluent Hackathon ʼ22, I was inspired to investigate if a streaming platform could […]