Tagged | apache-kafka

• Presentation: Building and Scaling a Control Plane for 1000s of Kafka Clusters

  (www.infoq.com)

  #performance #scaling #distributed-systems #apache-kafka

• Tiny Letter from Kafka

  (developers.soundcloud.com)

  #software-architecture #distributed-systems #apache-kafka #backend

• Enabling Seamless Kafka Async Queuing with Consumer Proxy

  (eng.uber.com)

  #data-pipeline #software-architecture #distributed-systems #apache-kafka

• Using kafka-merge-purge to Deal with Failure in an Event-Driven System at FLYERALARM

  (www.confluent.io)

  #infra #distributed-systems #apache-kafka

• Shallow Mirror

  (medium.com)

  #performance #apache-kafka #data-engineering

• Continuous learning in Project Thoth using Kafka and Argo

  (developers.redhat.com)

  #software-engineering #software-architecture #apache-kafka #python

• Stream Processing is Nothing Without Action

  (www.confluent.io)

  #data-pipeline #software-architecture #apache-kafka

• Uber Implements Disaster Recovery for Multi-Region Kafka

  (www.infoq.com)

  #software-architecture #infra #distributed-systems #apache-kafka

• Broadcom Modernizes Machine Learning and Anomaly Detection with ksqlDB

  (www.confluent.io)

  #data-pipeline #machine-learning #apache-kafka

• Real-Time Small Business Intelligence with ksqlDB

  (www.confluent.io)

  #DBMS #apache-kafka #SQL #analytics

• An investigation into Kafka Log Compaction

  (medium.com)

  #software-architecture #infra #logging #apache-kafka

• Real-Time Data Replication with ksqlDB

  (www.confluent.io)

  #DBMS #apache-kafka #data-engineering

• Using Kafka to Throttle QPS on MySQL Shards in Bulk Write APIs

  (stackshare.io)

  #software-architecture #DBMS #scaling #apache-kafka #data-engineering

• Presentation: Kafka Needs No Keeper

  (www.infoq.com)

  #software-architecture #distributed-systems #apache-kafka #backend

• 99th Percentile Latency at Scale with Apache Kafka

  (www.confluent.io)

  #performance #scaling #distributed-systems #apache-kafka

• Building an Adaptive, Multi-Tenant Stream Bus with Kafka and Golang

  (eng.lyft.com)

  #data-pipeline #distributed-systems #apache-kafka #GoLang

• Kafka on Ruby

  (medium.com)

  #apache-kafka #ruby #backend

• Article: The Kongo Problem: Building a Scalable IoT Application with Apache Kafka

  (www.infoq.com)

  #software-architecture #IoT #networking #apache-kafka

• Integrating Elasticsearch and ksqlDB for Powerful Data Enrichment and Analytics

  (www.confluent.io)

  #DBMS #apache-kafka #analytics #data-engineering

• Building a Materialized Cache with ksqlDB

  (www.confluent.io)

  #DBMS #apache-kafka #data-engineering

• Using graph algorithms to optimize Kafka operations, Part 2

  (medium.com)

  #algorithms #optimisation #apache-kafka #graph-processing

• Streaming Machine Learning with Tiered Storage and Without a Data Lake

  (www.confluent.io)

  #data-pipeline #machine-learning #distributed-systems #apache-kafka

• Infinite Storage in Confluent Platform

  (www.confluent.io)

  #distributed-systems #apache-kafka #big-data #data-engineering

• Streams and Monk – How Yelp is Approaching Kafka in 2020

  (engineeringblog.yelp.com)

  #data-pipeline #apache-kafka #backend #data-engineering

• Using graph algorithms to optimize Kafka operations, Part 1

  (medium.com)

  #algorithms #optimisation #apache-kafka #graph

• Streams and Tables in Apache Kafka: Processing Fundamentals

  (www.confluent.io)

  #data-pipeline #DBMS #distributed-systems #apache-kafka

• Streams and Tables in Apache Kafka: Topics, Partitions, and Storage Fundamentals

  (www.confluent.io)

  #DBMS #distributed-systems #apache-kafka #data-engineering

• Streams and Tables in Apache Kafka: A Primer

  (www.confluent.io)

  #data-pipeline #DBMS #apache-kafka #data-engineering

• Pipeline to the Cloud – Streaming On-Premises Data for Cloud Analytics

  (www.confluent.io)

  #data-pipeline #distributed-systems #apache-kafka #analytics

• Presentation: Practical Change Data Streaming Use Cases with Apache Kafka & Debezium

  (www.infoq.com)

  #scaling #apache-kafka #kubernetes #data-engineering

• Presentation: RabbitMQ and Kafka

  (www.infoq.com)

  #software-architecture #scaling #apache-kafka #rabbitMQ

• Apache Kafka Producer Improvements with the Sticky Partitioner

  (www.confluent.io)

  #software-architecture #performance #distributed-systems #apache-kafka

• Using Kafka to throttle QPS on MySQL shards in bulk write APIs

  (medium.com)

  #DBMS #apache-kafka #SQL #data-engineering

• Getting Started with Rust and Apache Kafka

  (www.confluent.io)

  #distributed-systems #apache-kafka #runtime #rust

• 🚂 On Track with Apache Kafka – Building a Streaming ETL Solution with Rail Data

  (www.confluent.io)

  #data-pipeline #scaling #distributed-systems #apache-kafka

• How LinkedIn customizes Apache Kafka for 7 trillion messages per day

  (engineering.linkedin.com)

  #performance #scaling #distributed-systems #apache-kafka #data-engineering

• Real-Time Analytics and Monitoring Dashboards with Apache Kafka and Rockset

  (www.confluent.io)

  #monitoring #apache-kafka #analytics #real-time

• Operating Apache Kafka Clusters 24/7 Without A Global Ops Team

  (eng.lyft.com)

  #infra #devops #scaling #apache-kafka

• Incremental Cooperative Rebalancing in Apache Kafka: Why Stop the World When You Can Change It?

  (www.confluent.io)

  #software-architecture #distributed-systems #apache-kafka

• 10 lessons learnt from the Ruby Refactoring Kata - Tennis Game

  (blog.arkency.com)

  #software-engineering #apache-kafka #ruby #refactor

• A look inside Kafka Mirrormaker 2

  (blog.cloudera.com)

  #data-pipeline #distributed-systems #apache-kafka

• Introducing Derivative Event Sourcing

  (www.confluent.io)

  #software-architecture #apache-kafka #backend #event-driven

• Using Graph Processing for Kafka Stream Visualizations

  (www.confluent.io)

  #data-pipeline #apache-kafka #graph-processing #visualisation

• Building Transactional Systems Using Apache Kafka

  (www.confluent.io)

  #software-architecture #distributed-systems #apache-kafka

• Building Shared State Microservices for Distributed Systems Using Kafka Streams

  (www.confluent.io)

  #software-architecture #distributed-systems #microservices #apache-kafka #backend

• Fault Tolerance in Distributed Systems: Tracing with Apache Kafka and Jaeger

  (www.confluent.io)

  #infra #distributed-systems #apache-kafka #tracing

• Presentation: Streaming Log Analytics with Kafka

  (www.infoq.com)

  #data-pipeline #distributed-systems #apache-kafka #backend

• Bust the Burglars – Machine Learning with TensorFlow and Apache Kafka

  (www.confluent.io)

  #software-design #image-processing #apache-kafka #tensor-flow

• How Heroku Operates its Multi-Tenant Apache Kafka Services

  (blog.heroku.com)

  #software-engineering #software-architecture #scaling #apache-kafka #backend

• Kafka Listeners – Explained

  (www.confluent.io)

  #software-architecture #distributed-systems #apache-kafka #internals #data-engineering

• Microservices, Apache Kafka, and Domain-Driven Design

  (www.confluent.io)

  #software-design #software-architecture #microservices #apache-kafka

• Building a Scalable Search Architecture

  (www.confluent.io)

  #search #scaling #apache-kafka #data-engineering

• High Availability Kafka Service in Microsoft Azure Cloud

  (medium.com)

  #scaling #distributed-systems #apache-kafka #cloud

• Streaming Data from the Universe with Apache Kafka

  (www.confluent.io)

  #software-architecture #distributed-systems #apache-kafka #data-engineering

• Reliable, Fast Access to On-Chain Data Insights

  (www.confluent.io)

  #block-chain #apache-kafka #real-time #ethereum

• Log Compacted Topics in Apache Kafka

  (towardsdatascience.com)

  #DBMS #apache-kafka #big-data #data-engineering

• Schemas, Contracts, and Compatibility

  (www.confluent.io)

  #scaling #distributed-systems #microservices #apache-kafka

• Kafka Replication: The case for MirrorMaker 2.0

  (blog.cloudera.com)

  #distributed-systems #apache-kafka #backend

• Apache Kafka Data Access Semantics: Consumers and Membership

  (www.confluent.io)

  #distributed-systems #apache-kafka #data-engineering

• Optimizing Kafka Streams Applications

  (www.confluent.io)

  #performance #optimisation #distributed-systems #apache-kafka #backend

• Understanding Kafka with Factorio

  (hackernoon.com)

  #distributed-systems #apache-kafka #event-queue

• Optimizing Kafka for the cloud

  (medium.com)

  #software-architecture #optimisation #apache-kafka #cloud

• EthVM: First Open-Source Block Explorer Powered by Kafka

  (hackernoon.com)

  #block-chain #apache-kafka #ethereum

• Putting Events in Their Place with Dynamic Routing

  (www.confluent.io)

  #software-architecture #microservices #apache-kafka #event-driven

• Consuming Messages Out of Apache Kafka in a Browser

  (www.confluent.io)

  #data-stream #apache-kafka #browsers #frontend

• Kafka Streams’ Take on Watermarks and Triggers

  (www.confluent.io)

  #data-pipeline #stream-processing #distributed-systems #apache-kafka

• Journey to Event Driven – Part 3: The Affinity Between Events, Streams and Serverless

  (www.confluent.io)

  #data-pipeline #serverless #apache-kafka #event-driven

• Rendezvous with Kafka : A simple guide to get started

  (medium.com)

  #distributed-systems #apache-kafka #backend

• Sysmon Security Event Processing in Real Time with KSQL and HELK

  (www.confluent.io)

  #data-pipeline #software-architecture #apache-kafka #real-time

• Presentation: Cloud-native Streaming Platform: Running Apache Kafka on PKS

  (www.infoq.com)

  #distributed-systems #apache-kafka #backend #cloud

• A Beginner’s Perspective on Kafka Streams: Building Real-Time Walkthrough Detection

  (www.confluent.io)

  #data-pipeline #stream-processing #distributed-systems #apache-kafka

• Building a Scalable Event Pipeline with Heroku and Salesforce

  (engineering.salesforce.com)

  #data-pipeline #scaling #apache-kafka #event-driven

• Journey to Event Driven – Part 1: Why Event-First Thinking Changes Everything

  (www.confluent.io)

  #software-design #software-architecture #apache-kafka #backend #event-driven

• Finding Kafka’s throughput limit in Dropbox infrastructure

  (blogs.dropbox.com)

  #performance #scaling #apache-kafka #backend

• Article: Increasing the Quality of Patient Care Through Stream Processing

  (www.infoq.com)

  #stream-processing #software-architecture #apache-kafka #backend

• Migrating Kafka's Zookeeper With No Downtime

  (engineeringblog.yelp.com)

  #distributed-systems #apache-kafka #migration #backend

• Building a Service-oriented Architecture with Rails and Kafka

  (blog.heroku.com)

  #software-architecture #RoR #apache-kafka #backend

• Achieving high availability with stateful Kafka Streams applications

  (tech.transferwise.com)

  #distributed-systems #apache-kafka #backend #availability

• Managing Real-time Event Streams and SQL Analytics with Apache Kafka on Heroku, Amazon Redshift, and Metabase

  (blog.heroku.com)

  #apache-kafka #SQL #real-time #event-queue

• Twitter’s Kafka adoption story

  (blog.twitter.com)

  #software-engineering #apache-kafka #backend

• How Pinterest runs Kafka at scale

  (medium.com)

  #data-pipeline #software-architecture #scaling #apache-kafka #backend

• Kafka Connect Deep Dive – Converters and Serialization Explained

  (www.confluent.io)

  #data-pipeline #distributed-systems #apache-kafka #internals #backend

• Running Apache Kafka on Kubernetes at Shopify

  (shopifyengineering.myshopify.com)

  #containers #distributed-systems #apache-kafka #kubernetes

• Apache Kafka’s Distributed System Firefighter — The Controller Broker

  (hackernoon.com)

  #software-architecture #distributed-systems #apache-kafka

• ATM Fraud Detection with Apache Kafka and KSQL

  (www.confluent.io)

  #data-pipeline #machine-learning #apache-kafka #SQL

• Noise Mapping with KSQL, a Raspberry Pi and a Software-Defined Radio

  (www.confluent.io)

  #hardware #data-stream #apache-kafka

• Open Sourcing Mirus

  (engineering.salesforce.com)

  #data-pipeline #software-architecture #distributed-systems #apache-kafka

• Real-Time Presence Detection at Scale with Apache Kafka on AWS

  (www.confluent.io)

  #stream-processing #AWS #apache-kafka #real-time

• The Changing Face of ETL

  (www.confluent.io)

  #stream-processing #software-architecture #apache-kafka

• Putting the Power of Kafka into the Hands of Data Scientists

  (multithreaded.stitchfix.com)

  #stream-processing #data-science #apache-kafka #analytics

• Keeping Counts In Sync

  (developers.soundcloud.com)

  #data-pipeline #software-architecture #distributed-systems #apache-kafka

• Experiences from Building an Event-Sourced System with Kafka Streams

  (www.infoq.com)

  #data-stream #apache-kafka #event-sourcing

• Analytics on Bare Metal: Xenon and Kafka® Connect

  (www.confluent.io)

  #distributed-systems #apache-kafka #analytics #systems

• Scalability of Kafka Messaging using Consumer Groups

  (blog.cloudera.com)

  #software-architecture #apache-kafka #event-queue #pubsub

• Confluent.io – Part 2: BUILD A STREAMING PIPELINE

  (blog.octo.com)

  #data-pipeline #stream-processing #apache-kafka

• We ❤️ syslogs: Real-time syslog Processing with Apache Kafka and KSQL—Part 1: Filtering

  (www.confluent.io)

  #logging #apache-kafka

• Why We Chose Kafka For The Trello Socket Architecture

  (tech.trello.com)

  #web-socket #apache-kafka #task-queue

• KSQL in Action: Enriching CSV Events with Data from RDBMS into AWS

  (www.confluent.io)

  #DBMS #data-stream #apache-kafka

• Tuning the Kafka Connect Cassandra Source (part 2)

  (medium.com)

  #data-pipeline #apache-kafka #cassandra

• Event Sourcing Using Apache Kafka

  (www.confluent.io)

  #distributed-systems #apache-kafka #backend #event-sourcing

• Squeezing the firehose: getting the most from Kafka compression

  (blog.cloudflare.com)

  #performance #CDN #apache-kafka #compression

• Event Sourcing with Kafka, RabbitMQ and JPA

  (engineering.pivotal.io)

  #apache-kafka #event-queue #event-sourcing

• Big, fast, easy data with KSQL

  (www.oreilly.com)

  #data-stream #apache-kafka #SQL

• Using Apache Kafka for Asynchronous Communication in Microservices

  (blog.codeship.com)

  #microservices #apache-kafka

• How Apache Kafka Inspired Our Platform Events Architecture

  (engineering.salesforce.com)

  #data-pipeline #software-architecture #apache-kafka #event-driven

• Why we wrote our Kafka Client in Pony

  (blog.wallaroolabs.com)

  #distributed-systems #apache-kafka #pony-lang

• Apache Mesos, Apache Kafka and Kafka Streams for Highly Scalable Microservices

  (www.confluent.io)

  #scaling #microservices #apache-kafka #mesos

• Rock Solid Kafka and ZooKeeper Ops on AWS

  (jobs.zalando.com)

  #devops #apache-kafka #apache-zookeeper

• Transactions in Apache Kafka

  (confluentio.wpengine.com)

  #distributed-systems #apache-kafka #backend

• A Blockchain Experiment With Apache Kafka

  (hackernoon.com)

  #distributed-systems #block-chain #apache-kafka #fun

• Surviving Data Loss

  (jobs.zalando.com)

  #data-pipeline #AWS #apache-kafka #apache-zookeeper

• The Blog Post on Monitoring an Apache Kafka Deployment to End Most Blog Posts

  (www.confluent.io)

  #monitoring #devops #apache-kafka #backend

• Enabling Exactly-Once in Kafka Streams

  (www.confluent.io)

  #software-architecture #distributed-systems #apache-kafka #task-queue

• Testing Kafka Streams topologies with Kafka Interceptors

  (blog.url.com)

  #software-design #apache-kafka #backend

• Scaling Slack’s Job Queue

  (slack.engineering)

  #scaling #apache-kafka #redis #task-queue

• Running Kafka Streams applications in AWS

  (jobs.zalando.com)

  #data-pipeline #stream-processing #distributed-systems #apache-kafka

• Real-time Ranking with Apache Kafka’s Streams API

  (jobs.zalando.com)

  #stream-processing #distributed-systems #apache-kafka #pubsub

• Transactions in Apache Kafka

  (www.confluent.io)

  #infra #distributed-systems #apache-kafka #backend

• Lambda computing with Minio and Kafka

  (blog.minio.io)

  #microservices #serverless #apache-kafka

• Building a Microservices Ecosystem with Kafka Streams and KSQL

  (www.confluent.io)

  #software-architecture #containers #microservices #apache-kafka

• Taking KSQL for a Spin Using Real-time Device Data

  (www.confluent.io)

  #data-pipeline #stream-processing #apache-kafka #SQL

• Rebuilding the Segment Leaderboards Infrastructure — Part 3: Design of the New System

  (medium.com)

  #stream-processing #apache-kafka #big-data #backend #cassandra

• Leveraging Kafka Streams to reduce DB Hits

  (engineering.wingify.com)

  #optimisation #apache-kafka #backend

• Ranking Websites in Real-time with Apache Kafka’s Streams API

  (www.confluent.io)

  #stream-processing #apache-kafka #ranking

• Publishing with Apache Kafka at The New York Times

  (open.nytimes.com)

  #data-pipeline #infra #apache-kafka #backend

• Predicting Flight Arrivals with the Apache Kafka Streams API

  (www.confluent.io)

  #stream-processing #machine-learning #apache-kafka

• Getting Started Analyzing Twitter Data in Apache Kafka through KSQL

  (www.confluent.io)

  #stream-processing #data-analytics #apache-kafka #SQL

• Event First Development - Moving Towards Kafka Pipeline Applications

  (jobs.zalando.com)

  #software-architecture #infra #apache-kafka #event-driven

• Using Kafka Streams API for predictive budgeting

  (medium.com)

  #data-pipeline #stream-processing #apache-kafka #big-data

• How to Build and Deploy Scalable Machine Learning in Production with Apache Kafka

  (www.confluent.io)

  #data-pipeline #machine-learning #apache-kafka

• Disaster Recovery for Multi-Datacenter Apache Kafka Deployments

  (www.confluent.io)

  #infra #distributed-systems #apache-kafka #availability

• Crossing the Streams – Joins in Apache Kafka

  (www.confluent.io)

  #stream-processing #distributed-systems #apache-kafka

• It’s Okay To Store Data In Apache Kafka

  (www.confluent.io)

  #noSQL #distributed-systems #apache-kafka

• How Apache Kafka is Tested

  (www.confluent.io)

  #testing #distributed-systems #apache-kafka

• Publishing with Apache Kafka at The New York Times

  (www.confluent.io)

  #distributed-systems #apache-kafka #backend

• Open Sourcing Kafka Cruise Control

  (engineering.linkedin.com)

  #dev-tools #automation #distributed-systems #apache-kafka

• Introducing KSQL: Open Source Streaming SQL for Apache Kafka

  (www.confluent.io)

  #data-stream #apache-kafka #SQL #real-time

• Open sourcing DoctorKafka: Kafka cluster healing and workload balancing

  (medium.com)

  #monitoring #logging #distributed-systems #apache-kafka

• The Simplest Useful Kafka Connect Data Pipeline In The World … or Thereabouts (Part 1)

  (www.confluent.io)

  #data-pipeline #distributed-systems #apache-kafka

• From Hadoop and Cassandra to Kafka Streams

  (tech.finn.no2017)

  #stream-processing #apache-kafka #hadoop #cassandra

• Chain Services with Exactly-Once Guarantees

  (www.confluent.io)

  #microservices #apache-kafka #state-machine

• Using Apache Kafka as a Scalable, Event Driven Backbone for Service Architectures

  (www.confluent.io)

  #software-architecture #apache-kafka #event-queue

• Exactly-once Semantics are Possible: Here’s How Kafka Does it

  (www.confluent.io)

  #stream-processing #infra #web-backend #apache-kafka