Skip to content

Build the topology

The adapter binds to what you create and never creates a topic, queue, subscription or redrive policy for a durable subscription. Why. This page is what to create. The names come from Topology and naming.

For each subject you publish or subscribe to, and for each durable subscription, you need the following.

1. A topic per subject

Variety Topic name Attributes
Standard the subject with each . replaced by _, for example orders_created none
FIFO the same, plus .fifo, for example orders_created.fifo FifoTopic=true; ContentBasedDeduplication=false as the adapter's suite creates it

orders.created and orders_created derive the same name. Do not use both as subjects.

2. A queue per durable subscription

A subscription under AtLeastOnce consumes the queue named by its DurableName. A competing AtMostOnce subscription (FIFO only) consumes the queue named by its Group. A FIFO name gets .fifo appended, and a name over 80 characters with the suffix included is refused.

Variety Queue name Attributes
Standard the DurableName a RedrivePolicy
FIFO the DurableName plus .fifo FifoQueue=true, a RedrivePolicy

A Standard backend cannot be given an AtMostOnce subscription at all, so every queue it consumes is a durable one.

3. A dead-letter queue and a redrive policy

Under AtLeastOnce, the queue's RedrivePolicy must exist, and its maxReceiveCount must equal the subscription's MaxDeliver. The adapter reads both at Subscribe and refuses a queue with no policy, and one whose count differs. It does not look at deadLetterTargetArn, so it does not check that the dead-letter queue exists. The adapter's suite creates a dead-letter queue of the same variety as the queue it serves, which is what AWS requires of a FIFO queue.

maxReceiveCount may be a number or a quoted number, as AWS returns it.

4. A subscription from each topic to each queue

Subscribe the queue to the topic with raw message delivery on:

$ aws sns subscribe --topic-arn $TOPIC --protocol sqs \
    --notification-endpoint $QUEUE_ARN --attributes RawMessageDelivery=true

The adapter reads the SQS message's body and message attributes as the event. With raw delivery they are what the publisher sent. Without it SNS wraps both in a JSON envelope, and nothing the adapter reads is where it expects. The adapter does not check the flag, and a wrapped message is simply an unreadable one.

The adapter does not check that the queue is subscribed to the topic either. A queue with no subscription receives nothing, and nothing says so.

5. A queue policy that lets the topic send

On AWS the queue needs an access policy admitting the topic. The adapter writes this one on the queues it creates:

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": {"Service": "sns.amazonaws.com"},
    "Action": "sqs:SendMessage",
    "Resource": "<the queue's ARN>",
    "Condition": {"ArnEquals": {"aws:SourceArn": "<the topic's ARN>"}}
  }]
}

LocalStack does not enforce queue policies, so a topology that works there can still fail on AWS without one.

The visibility timeout

AckWait zero means the adapter uses the queue's VisibilityTimeout, which must then be above zero. With AckWait set, every receive asks for that many whole seconds and the queue's own setting is unused. See Settlement, visibility and redrive.

At-most-once fan-out needs nothing

A Fanout subscription under AtMostOnce on FIFO gets a queue the adapter creates, subscribes raw to the subject's topic, and deletes when the source retires. You create only the topic. That queue holds one of the topic's subscriptions while it lives; see Limits.

Check it

With the topology made, the quickest check is the adapter's own refusals. A missing topic is sqs.ErrNoTopic at Subscribe and Publish, a missing queue sqs.ErrNoQueue at Subscribe, and a wrong maxReceiveCount an error naming both numbers. Errors and refusals lists them.