Natan Silnitsky is a backend-infra developer @Wix.com. He is on the Data streaming team in charge of building event driven libraries on top of Kafka. Before that he was part of a task force that was responsible for building the next generation CI system at Wix on top of Google's Bazel build tool. Has many years of experience as a developer of large scale web services - First in .Net, later in Scala. Natan's passions include clean code, dev velocity and great software design.
In today's environment of globally distributed large scale web applications, reactive, event-driven design is mandatory. In this talk I will showcase how at Wix.com we Implemented a reliable, scalable, event-driven, distributed Mega Microservices architecture on top of kafka but without kafka streams. We, at Wix Engineering, have built powerful soon-to-be-os libraries for data streaming and data storage using Kafka features like compacted logs and exactly once semantics. These libraries offer low latency, high throughput event processing and also high volume data-storage with very simple and easy to understand APIs that work seamlessly together with other technologies such as gRPC. I will present the challenges we faced while creating and fine-tuning such a system. An architecture that is comprised of a 1,000 micro-services that need to handle incoming traffic from multiple data centers while preserving eventual consistency and high throughput.
Scheduled on Friday from 13:10 to 14:00 in Room 6