SAA-C03
Deep Dive
Amazon Kinesis is the answer when the scenario involves processing millions of events in real time (IoT, click-streams, application logs, financial transactions). SQS FIFO guarantees order and exact deduplication when sequential processing is critical (SQS FIFO). SAA-C03 tests when to use streaming vs traditional messaging.
Contents
Kinesis Data Streams lets you ingest and process real-time data streams at massive scale. Data is split across shards: each shard accepts 1 MB/s in and 2 MB/s out. Total stream capacity = number of shards × capacity per shard.
Shards
Unit of capacity. Each shard = 1 MB/s in, 2 MB/s out. Scales horizontally by adding shards.
Retention
24h by default, up to 365 days with Extended Retention. Allows replaying historical data.
Partition Key
Determines which shard each record goes to. The same key always goes to the same shard — a hot partition if not well distributed.
Producers and Consumers
Producers
AWS SDK, Kinesis Producer Library (KPL), Kinesis Agent. KPL aggregates records and retries automatically.
Consumers
KCL, Lambda, Managed Service for Apache Flink, Firehose. With Enhanced Fan-Out, each consumer gets a dedicated 2 MB/s per shard.
Available consumers
Kinesis Consumer Library (KCL)
SDK for custom consumers. Manages checkpointing and shard balancing automatically.
AWS Lambda
Native trigger. Processes records in batches. Ideal for serverless pipelines.
Managed Service for Apache Flink
Real-time processing with Apache Flink (Java, Python, or Flink SQL). Formerly Kinesis Data Analytics.
Amazon Data Firehose
Consumes the stream and delivers to managed destinations (S3, Redshift, OpenSearch).
Provisioned Mode
Manual shard management. Billed per shard-hour. Ideal when throughput is predictable.
On-Demand Mode
Scales automatically based on traffic. Billed per GB processed. Ideal for variable or unpredictable loads.
Exam trap
Data in Kinesis Streams is immutable — it cannot be deleted manually. It only expires according to the configured retention (24h by default, up to 365 days). Don't confuse it with SQS, where messages are removed after processing.
Firehose is a fully serverless service for loading data into managed destinations. It requires no shard or consumer management. In exchange, it introduceslatency due to buffering: a minimum of 60 seconds before delivering the data.
Managed destinations
S3, Redshift, OpenSearch Service, Splunk, custom HTTP endpoints.
In-flight transformations
Lambda to transform/filter records. Automatic conversion to Parquet or ORC before loading.
Buffering
Accumulates data before delivering. Buffer size: 1–128 MB. Buffer interval: 60–900 seconds.
Key difference vs Kinesis Data Streams
Real-time processing. Multiple consumers read the same stream. Data persists (configurable retention). Full control over the consumer.
Managed delivery to a destination. No custom consumers. Minimum 60s latency. Serverless with no capacity management.
Exam trap
Firehose is NOT pure real time. If the scenario requires processing with latency under 60 seconds, the answer is Kinesis Data Streams, not Firehose. Firehose is for loading/ETL, not for instant reaction to events.
Runs Apache Flink applications over streams in real time using Kinesis Data Streams, Amazon MSK, or other sources. Fully managed — no servers to provision. The older SQL-only variant (Kinesis Data Analytics for SQL) was shut down in January 2026.
Flink SQL and Studio notebooks
Queries with time windows (tumbling windows, sliding windows). Ideal for real-time aggregate metrics: average, sum, count per time window.
Managed Apache Flink
For more complex processing with Java, Python or Scala. Allows advanced business logic that SQL can't express easily.
Main use cases
Real-time anomaly detection, alerts on business metrics, enriching streams with reference data.
Supported sources
Kinesis Data Streams, Amazon MSK, and other sources as source. Output can go to Streams, Firehose, Lambda or other destinations.
SQS FIFO guarantees two properties that SQS Standard cannot offer: exact First-In-First-Out order and exactly-once processing via deduplication. The trade-off is a lower maximum throughput.
Throughput
300 msg/s by default. Up to 3,000 msg/s with batching (groups of 10 messages).
Deduplication
Message Deduplication ID. If the same ID arrives within the 5-minute window, the duplicate is dropped.
Message Group ID
Groups messages for ordered processing per group. Different groups are processed in parallel by different consumers.
SQS Standard vs SQS FIFO
SQS Standard
Nearly unlimited throughput. At-least-once delivery (can duplicate). Best-effort ordering (not guaranteed). Ideal for massive scale where order doesn't matter.
SQS FIFO
300 msg/s (3,000 with batching). Exactly-once processing. Order guaranteed per Message Group. Queue name must end in .fifo.
Exam trap
A FIFO queue name must end in .fifo — without that suffix, creation fails. Also, SQS Standard cannot be converted to FIFO after creation: you must create a new queue.
Kinesis Data Streams
Multiple consumers read the same data, retention for replay, strict real time, big data / analytics.
SQS Standard
Decoupling, high scale, workers compete for messages, order doesn't matter.
SQS FIFO
Guaranteed order, exact deduplication, financial transactions, cases with ≤300 msg/s.
Amazon Data Firehose
ETL to S3/Redshift, no custom processing needed, fully managed delivery.
Exam keyword patterns
"Millions of IoT events in real time" → Kinesis Data Streams
"Process in exact order / deduplication" → SQS FIFO with Message Group ID
"Load into S3 / Redshift with no infrastructure management" → Amazon Data Firehose
"Multiple applications read the same events" → Kinesis Streams (fan-out)
"Firehose is real time" — false. Minimum latency of 60s due to buffering
Hot partition = same Partition Key for too many records
Distribute with a high-cardinality key (e.g. user_id, device_id). A saturated shard drops records.
Enhanced Fan-Out when there are multiple low-latency consumers
Each registered consumer gets a dedicated 2 MB/s per shard. Without Enhanced Fan-Out, all consumers share the shard's total 2 MB/s.
Firehose CANNOT be a source for Kinesis Streams
The direction is: Streams → Firehose. Firehose can consume from Streams, not the other way around.
SQS FIFO: the queue name MUST end in .fifo
Without that suffix, creation fails. Also remember there is no ordering in SQS Standard — it cannot be enabled "after the fact".
Records in Kinesis Streams are immutable
They cannot be deleted manually. They only expire according to the configured retention (24h – 365 days).
Visualizes the flow from producers to consumers through the stream's shards.
Did you get this topic?
Test what you just learned
A company processes payment transactions. They need to guarantee that each transaction is processed exactly once and in the order it was received, per customer account. The volume is 100 transactions/second. Which service is most appropriate?
Put what you learned to the test with original SAA-C03 exam questions.
Start SAA-C03 mock examUnlimited mock exams with the Pro plan
Log in to track your progress.