Developing Kafka Streams Applications with Upgradability in Mind
Date : April 26, 2022
Time : 4:00 PM - 4:45 PM

Does your organization struggle with updating of its Kafka Streams application? Releasing a new version of a Kafka Streams application can be challenging, especially if its state has to be preserved between releases. Consider these best-practices and architectural ideas to make this process smoother and improve your release process. Having experienced accidental removal of change-log topics and needing to expand partitions, it is much easier to handle with some planning. With the proper planning, you can achieve easier application upgrades. Key take-aways from the session include: * How do minimize the rebuilding of the state-stores. * How to change stream topologies without affecting the existing state stores. * What you can do when you absolutely need to increase the number of partitions within your application. * How to leveraging schemas for application releases. * Measures to prevent data corruption, especially if Kafka is not only your system of record but also your source of truth. * Techniques to support rolling back an application. * The advantages of splitting apart a Kafka Streams application into multiple applications.

Neil Buesing
Principal Solutions Architect, Rill Data