Skip to content

Wire format and SNS limits

The formats themselves are go/envelope's and, for CloudEvents, go/envelope-cloudevents's. This page is how the adapter carries what a format produces. To choose one, see Choose a wire format.

Writing

The adapter asks Settings.Format to encode the event into a wire form: headers, a content type and a body. It sends:

Wire part SNS
body the Message
each header a message attribute of data type String, carrying the header's first value
the content type a message attribute named content-type, String, because neither SNS nor SQS has a property for it

No CloudEvents binding covers SNS or SQS, so binary mode uses the HTTP binding's ce- prefix.

What is refused at Publish

It is refused before SNS is called, so LocalStack and AWS agree.

Refused Because Error text starts
a format that fails to encode the format's own rule messaging-sqs: encoding the event
an empty body SNS refuses an empty message messaging-sqs: SNS refuses an empty message
a body that is not valid UTF-8 SNS takes a message only as UTF-8 text. LocalStack does not enforce this and AWS does messaging-sqs: SNS takes a message only as UTF-8 text
more than 10 attributes, the content type included SNS carries at most ten message attributes messaging-sqs: the event has N headers and SNS carries at most 10 attributes
a message over 262,144 bytes (256 KiB) SNS's cap on a message, attributes included messaging-sqs: the encoded event is N bytes and SNS carries at most 262144

The size counted is the body, plus for each attribute the length of its name, of its data type name String and of its value.

Binary and structured against the caps

Binary (ce-) Structured
attributes one per CloudEvents attribute set, one per extension, and the content type the content type only
the ten-attribute cap the event's four required attributes take four, and its content type, when it has one, a fifth; each optional attribute (time, subject, dataschema) and each extension takes another the content type is the only attribute
a payload that is not UTF-8 refused carried base64-encoded as data_base64
a filter policy on an attribute works on message attributes has to use FilterPolicyScope: MessageBody

MassTransit, like structured, puts the whole event in the body and spends one attribute, its content type. It refuses at publish an ID that is not a UUID and a payload that is not JSON. NServiceBus writes its own headers as attributes, which count toward the ten.

Reading

The adapter builds a header map from the SQS message's message attributes, taking each attribute's String value, and reads it with the body through envelope.Read over Settings.Reads. One format must claim it.

Result What happens
one format claims and decodes it a Delivery is made, with the event and what no format took as Foreign
none claims it, two claim it, or the claiming format cannot decode it unreadable

The receive asks for every message attribute and for ApproximateReceiveCount, which becomes Delivery.Attempt.

Attributes no format takes reach a handler through Delivery.Foreign() and are never sent on.

Raw delivery

The adapter reads the SQS message exactly as SNS delivered it. That means subscriptions from topic to queue must have RawMessageDelivery=true. Without it SNS puts its own JSON wrapper in the body and the attributes inside it, and the adapter, which reads the SQS message's own body and attributes, finds nothing to claim. See Build the topology.