Cloudflare + SQS = increased reliability

We've added a Cloudflare worker proxy and SQS queue to improve the performance and reliability of our API. It's not just for our Cloud product, you can optionally use it if you're self-hosting as all the code is in our open source repository.

To begin with we're using this when events are sent to us. That happens when you use client.sendEvent and client.sendEvents. More API routes will be supported in the future.

How does it work?

Requests to the API are proxied through a Cloudflare worker. That worker intercepts certain API routes and sends the data to an SQS queue. The main API server then polls the queue for new data and processes it.

Why is this better?

If there is any downtime on the main API servers, we won't lose events. The Cloudflare worker will queue them up and the main API server will process them when it's back online.

Also, it allows us to deal with more load than before. The Cloudflare worker can handle a lot of requests and the main API server can process them at its own pace.

Event processing can be scaled horizontally with more workers that poll the queue.

How to update

The Trigger.dev Cloud is now running v2.2.9. If you are self-hosting you can upgrade by pinning to the v2.2.9 tag.