Azure Event Hubs ➔ Amazon Kinesis Data Streams & Firehose
From Azure Event Hubs Throughput Units & Capture to Amazon Kinesis Shards & Firehose.
Both Event Hubs and Kinesis share the same append-only partitioned log architecture. In Azure, you size namespaces using Throughput Units (1 TU = 1 MB/s in, 2 MB/s out) or dedicated Processing Units (PUs). In AWS Kinesis, you size streams by Shards (1 Shard = 1 MB/s in, 2 MB/s out) or select On-Demand mode. For Event Hubs Capture, AWS provides Amazon Kinesis Data Firehose, which can also convert incoming JSON to Parquet on the fly!
1. Architectural Mechanism Comparison
Azure Event Hubs
Partitioned append-only commit logs. Throughput is provisioned in Throughput Units (TUs) with Auto-Inflate, Kafka 1.0+ protocol on port 9093, and zero-code Event Hubs Capture into ADLS Gen2.
- Native Apache Kafka multi-protocol endpoint on port 9093 with zero broker management.
- Auto-Inflate prevents 429 throttling by dynamically scaling Throughput Units during spikes.
- Event Hubs Capture automatically lands stream data into Avro/Parquet in ADLS Gen2.
Amazon Kinesis Data Streams & Firehose
Kinesis Data Streams partitions data into Shards (1 MB/s Ingress, 2 MB/s Egress per shard) or On-Demand mode. Kinesis Data Firehose provides zero-code delivery to S3, Redshift, and Snowflake.
- On-Demand Mode: Automatically scales shard capacity up and down based on observed traffic.
- Enhanced Fan-Out (EFO): Dedicated 2 MB/s HTTP/2 push pipe per consumer group with sub-70ms latency.
- Kinesis Data Firehose: Zero-code ingestion with in-flight dynamic partitioning and Parquet conversion.
2. Interactive Terminology & Concept Bridge
Interactive Concept Bridge: Terminology & Architectural Mapping
Click any concept below to see how your AZURE knowledge directly maps into AWS.
Partition
Append-only commit log segment maintaining FIFO ordering per partition key.
Shard
Base throughput unit in Kinesis providing 1 MB/sec ingress and 2 MB/sec egress.
Both represent the horizontal scaling unit and enforce strict FIFO message ordering per key.
3. Visual Architecture Pipeline (Amazon Kinesis Data Streams & Firehose)
Amazon Kinesis 3-Stage Architecture: Producers ➔ Shards & Logs ➔ EFO & Firehose
Click any section below or run the simulation to see how Kinesis handles partitioned real-time streams.
2. Shard Commit Log & Retention Engine
Each Shard operates as an independent, append-only, ordered commit log. Events are stored immutably for up to 365 days. On-Demand mode automatically splits and merges shards based on live traffic volume.
“Like a row of high-speed tape recorders writing continuously, allowing multiple listeners to rewind and replay independently.”
- Sequence Numbers: Monotonically increasing 128-bit integer IDs assigned to every record.
- On-Demand Mode: Auto-scales shard count from 0 to thousands with zero capacity planning.
- Replay Retention: Up to 365 days of historical stream re-processing.
4. Side-by-Side Code, CLI & Terraform Translator
Side-by-Side Code & Syntax Translator
# Azure CLI: Create Event Hubs Namespace & Hub
az eventhubs namespace create \
--name "eh-telemetry-prod" \
--resource-group "rg-analytics-prod" \
--sku Standard \
--capacity 4 \
--enable-auto-inflate true
az eventhubs eventhub create \
--name "iot-sensor-stream" \
--namespace-name "eh-telemetry-prod" \
--resource-group "rg-analytics-prod" \
--partition-count 8 \
--message-retention 7# AWS CLI: Create Kinesis Stream (On-Demand Mode)
aws kinesis create-stream \
--stream-name "iot-sensor-stream" \
--stream-mode-details "StreamMode=ON_DEMAND"
# Or create Provisioned Stream with 8 Shards
aws kinesis create-stream \
--stream-name "iot-sensor-stream-provisioned" \
--shard-count 8
# Set retention period to 7 days (168 hours)
aws kinesis increase-stream-retention-period \
--stream-name "iot-sensor-stream" \
--retention-period-hours 1685. Paradigm Shift Gotchas: Traps to Avoid in AWS
Standard Kinesis Consumer Throughput Contention
In Azure Event Hubs, each consumer group gets its own independent egress allocation. In standard AWS Kinesis, if 3 consumer applications read from the same shard simultaneously, they share the single 2 MB/s egress limit and throttle each other.
Register critical consumers as Enhanced Fan-Out (EFO) consumers. EFO provides dedicated 2 MB/s HTTP/2 push pipelines to each consumer application.
25 KB Payload Billing Chunk Size in Kinesis
Kinesis charges PUT Payload Units in 25 KB chunks (compared to Event Hubs 64 KB event units). Publishing 100 KB records incurs 4 billing units per event.
Compress payloads using gzip/snappy or pack multiple micro-events into aggregated buffers using the Kinesis Producer Library (KPL).
6. Test Your Mental Model
Quick Knowledge Check: Test Your AWS Mental Model
Solidify your cross-cloud understanding with instant feedback.