Natan Silnitsky is a backend-infra engineer @Wix.com. He is on the Data streaming team in charge of building event driven libraries and tools on top of Kafka and ZIO. Before that he was part of a task force that was responsible for building the next generation CI system at Wix on top of Google's Bazel build tool. Has many years of experience as a developer of large scale web services - First in .Net, later in Scala. Natan's passions include clean and functional code, dev velocity and great software design.
At Wix, we have created a universal event-driven programming infrastructure on top of the Kafka message broker.
This infra makes sure messages are eventually successfully consumed and produced no matter what failure it encounters.
In this talk, you will learn about the features we introduced in order to make sure our distributed system can safely handle an ever growing message throughput in a fault tolerant manner.
You will be introduced to such techniques as retry topics, local persistent queues, and cooperative fibers that help make your flows more resilient and performant.
You will also learn how to make this infra work for all programming languages tech stacks with optimal resource manage using the power of Kubernetes and gRPC.
When to use a client library, and when to deploy an external pod (DaemonSet, StatefulSet) or even deploy a sidecar.
Scheduled on Friday from 11:15 to 12:05 in Stream 2
During the past couple of years I’ve implemented or have witnessed implementations of several key patterns of event-driven messaging designs on top of Kafka that have facilitated creating a robust distributed system at Wix that can easily handle increasing traffic and storage needs with many different use-cases.
In this talk I will share these patterns with you, including:
* Consume and Project (data decoupling)
* End-to-end Events (Kafka+websockets)
* In memory KV stores (consume and query with 0-latency)
* Events transactions (Exactly Once Delivery)
Scheduled on Friday from 13:30 to 14:20 in Stream 2