SubscriptionsBlogContact

Scalable webhook listener with Lambda and SQS

Published in Code
July 09, 2020
1 min read

Listening for webhooks isn’t exactly difficult but there are a number of things which might trip you up:

  • Sudden spikes in traffic
  • Publicly exposing endpoints for internal services
  • The application taking too long to return, usually five seconds, causing multiple webhooks to fire (adding to additional server load)
  • Duplicate webhooks firing before the first has completed (caused by the above)

By building a simple Serverless application using both Lambda and SQS, you can solve a lot of these issues then consume the webhooks from the queue in your own time.

If anything fails whilst being consumed, it will get pushed straight back to the queue and can be retried again later on.

Create the application

Assuming Serverless is setup, first we’ll start a new project.

Setup your configuration

Replace the contents serverless.yml with the below then update your profile name and region.

This will define the NodeJS version, memory allocation, AWS profile, region, along with the lambda function name and SQS queue. The name of the queue is passed into the environment variable AWS_SQS_QUEUE_URL.

Webhook receiver to SQS function

Replace the contents of handler.js with the below.

The function will:

  1. Log the event to CloudWatch.
  2. Create a list of HTTP headers we can safely ignore.
  3. Add the remaining HTTP headers to the SQS message attributes. Only ten attributes are supported so adjust the ignore list if nessecary.
  4. Push the webhook event to SQS.
  5. Log the result to CloudWatch.
  6. Return a successful HTTP 200 response

The SQS message will now contain the HTTP headers and body from the original request.

Deploy to Lambda

The output will show your webhook URL, the SQS queue name can be found in the AWS console.

Sending a test webhook

Send a test webhook from the console.

Check the response in the SQS console.

SQS message details
SQS message details
SQS message body
SQS message body
SQS message attributes
SQS message attributes

Uninstall

To remove the function simply run:

Get the code

You can find the code on Github.


Tags

nodejslambdasqs
Next Article
Loading environment variables from AWS Secrets Manager

Topics

Code

Motorhome Controller

Subscriptions

Related Posts

Loading environment variables from AWS Secrets Manager
July 13, 2020
2 min

Get in touch

Do you want to work with us? Get in touch today.
Contact Us
© 2021, Viadog

Subscriptions

UpsellFree Gifts