{"componentChunkName":"component---src-templates-session-details-js","path":"/sessions/trials-tribulations-triumphs","result":{"data":{"contentfulSessionDetails":{"title":"Trials, Tribulations, and Triumphs: Migrating from Self-Managed Kafka to Managed Kafka in the Cloud","slug":"trials-tribulations-triumphs","timing":"2:00 PM - 2:45 PM","details":{"details":"Compare the Market has long used Kafka as part of its event data streaming architecture. Whilst Kafka proved a reliable and scalable solution, hosting it in AWS and managing it ourselves came with its own challenges. These challenges included administration of the platform and security, not to mention handling incidents and needing highly skilled talent in the teams to do all of this - and were all contributing factors in driving the decision to migrate.\n\nAfter much discovery, we landed on ​​Confluent Cloud as our provider of choice to replace our existing Self-Managed Kafka infrastructure. Attend this session to learn about how we went about the migration and issues faced and how this will power our next generation data platform. We will discuss how we overcame the following challenges and more:\n\n- Identifying a method of replicating topics between platforms\n- Reproducing our existing infrastructure as code with the Confluent Platform and Terraform\n- Coordinating the migration of 100+ consumers and producers, used across the business\n"},"speakers":[{"name":"Adrian Sibilla","designation":"Senior Engineering Manager - Data Engineering","organisation":"Compare the Market","image":{"file":{"url":"//images.ctfassets.net/w1ipbbcgm10q/3VIsqjQ4i3yTnu1X5zGIGN/f9cbb8c3a8499899250adf7e65c59b8e/adrian-sibilla__1_.jpeg"}}},{"name":"Dewi Rees","designation":"Senior Engineering Manager - Data Engineering","organisation":"Compare the Market","image":{"file":{"url":"//images.ctfassets.net/w1ipbbcgm10q/5OjpHopPxaBagcD80cful2/72f3db7cb72c0d9512f698cf1c85542e/20211030_100733_2.jpg"}}}],"schedule_details":[{"schedule":[{"date":"April 26, 2022","agenda":[{"events":[{"displayName":"Kafka Summit London","slug":"kafka-summit-london-2022","startDate":"April 25","eventEndDate":"26, 2022","endDate":"April 26, 2022","eventStartMonth":"April","eventEndMonth":"April","registrationUrl":null,"registrationCtaText":"REGISTER","transparentHeader":true,"bannerImageLeft":{"file":{"url":"//images.ctfassets.net/w1ipbbcgm10q/5IjLXOZWDB5zdhNOdLIgdm/a2308de1471a9dee27d3febabb0894fe/banner_l.jpg"}},"bannerImageRight":{"file":{"url":"//images.ctfassets.net/w1ipbbcgm10q/5GPu7bBV5Pk5ojbyIqUdRC/b9445a0a19aeae3ae91bf07e7219cc53/banner_r.jpg"}},"bannerImageLeftMobile":{"file":{"url":"//images.ctfassets.net/w1ipbbcgm10q/5IjLXOZWDB5zdhNOdLIgdm/a2308de1471a9dee27d3febabb0894fe/banner_l.jpg"}},"bannerImageRightMobile":{"file":{"url":"//images.ctfassets.net/w1ipbbcgm10q/5GPu7bBV5Pk5ojbyIqUdRC/b9445a0a19aeae3ae91bf07e7219cc53/banner_r.jpg"}},"seoMetadata":{"title":"Kafka Summit London 2022 | April 25-26 | London, UK","description":{"description":"Kafka Summit London offers 2 days of technical sessions on Apache Kafka, data streaming, and real-time data technologies from leading technologists and industry experts."}}}]}]}]}]}},"pageContext":{"slug":"trials-tribulations-triumphs"}},"staticQueryHashes":["3974625060"]}