Writing Kafka applications without Kafka server access

Developers creating Apache Kafka client applications to be used with ""locked down"" (for example they're in a Cloud deployment) Kafka service endpoints have a unique problem: These developers do not have access to the Kafka server backends! Therefore the client applications have to be designed with this in mind. They have to tolerate brokers becoming temporarily unavailable while their Kafka provider automatically upgrades or rolls the brokers. They also have to instrument and monitor their Kafka client applications, so that they have a better understanding of their clients' interactions with the brokers. Zoltan's talk will introduce these challenges and will guide you through some important design principles for writing fault tolerant and cloud-ready Kafka client applications. He'll also discuss the instrumentation (logging, metrics) of these applications so that you can identify bottlenecks or issues without having access to broker specific backend debug data.

Zoltan Balogh
Software Engineer, IBM UK Ltd.