Simple Queue Service is a queuing product much like the Tuxedo product now owned by Oracle. Together with AWS Simple Notification Service, SQS enables distributed, fault tolerate applications to be easily created.
SQS guarantees each message will be delivered at least once; the only issues is that the message might be delivered more than once. Message delivery order is not guaranteed either. SQS is NOT FIFO. The max time a SQS message can be retained (MessageRetentionPeriod) is 14 days.
Each 64kb of SQS message is billed as a message for a max of 256kb per message total. If the message payload is larger than 256kb it is best practice to store the message in ElasticCache, DynamoDB or on S3.
Polling
By default SQS uses short polling where the listening process hits the queue and gets a message (or not) and disconnects. Using long polling, the ReceiveMessage call issued from the worker of the queue will wait and listen to the queue for as long as 20 seconds before it times out or retrieves as a message. To enable long polling increase the ReceiveMessageWaitTimeSeconds attribute of the queue to a value greater then 0; a ReceiveMessageWaitTimeSeconds attribute of 0 enables short polling.
Visibility Timeout
VisibilityTimeout defines how long a message is INVISIBLE to other workers after being ReceiveMessage‘d by a worker. It is invisible so the worker who retrieved the message has the opportunity to process the message and remove it from the queue. If the worker is not successfully in processing the message, the VisibilityTimout then expires and the message is again available to be accessed by another worker. This ensures that if part of your application fails the message is not lost. Default visibility timeout is 30 seconds and the max is 12 hours and can be changed by the ChangeMessageVisibility action.
SQS API Reference Points
Messaging
| SQS API (Messaging) | Notes |
|---|---|
| SendMessage | Delivers a message to the specified queue. |
| ReceiveMessage | Retrieves one or more messages, with a maximum limit of 10 messages, from the specified queue. You can set WaitTimeSeconds to a number greater than 1 and enable long poll support per request. |
| DeleteMessage | Deletes the specified message from the specified queue. |
| ChangeMessageVisibility | Changes the visibility timeout of a specified message in a queue to a new value. This call restarts visibility in queue but not past 12 hours. |
Queue
| SQS API (Queue) | Notes |
|---|---|
| CreateQueue | Creates a new queue, or returns the URL of an existing one. |
| DeleteQueue | Deletes the queue specified by the queue URL, regardless of whether the queue is empty. |
| GetQueueAttributes | Gets attributes for the specified queue. |
| SetQueueAttributes | Sets the value of one or more queue attributes. |
SQS CLI
ChangeMessageVisability add-permission
Use Cases
Message priority? Multiple queues; more resources on high priority queue; Add DelaySeconds (Delivery Delay in the Console) to the lower priority queue
Bigger than 64kb? More than one message chunk
bigger than 256kb? Message with pointer to data in DynamoDB or S3
Workflow longer than 14 days? SWF
Decrease SQS cost? long poll
Lots of Visibility timeout? up timeout
Enable long poll? ReceiveMessageWaitTimeSeconds > 0