Reactor Events

Reactor Events allow you to configure rules that react to messages being published or presence events emitted (such as members entering or leaving) on channels. These rules can notify HTTP endpoints, serverless functions or other services for each event as they arise, or in batches.

Reactor Events are rate limited and are suitable for low to medium volumes of updates. If you expect a high volume of events and messages (averaging more than 25 per second), then you should consider using our message queues or firehose as they are more suitable for higher volumes.

Subscribing to events and messages on-demand is often best done using our realtime client libraries or by subscribing to Ably using any of the realtime protocols we support. However, when a persistent subscription is required to push data into third party systems, the Reactor is designed for this use case and is available as Reactor Events (for HTTP requests, serverless functions, etc), Reactor Queues (data is pushed into our own hosted message queues that you can subscribe to), or Reactor Firehose (stream events into third party systems like Kafka and AWS Kinesis).

If you want to be notified as events arise, trigger serverless functions, or invoke an HTTP request to an endpoint, then Reactor Events is the right choice. For example, if you want to send a welcome message to someone when they become present on a chat channel, you can use Reactor Events to trigger a serverless function immediately after they enter with using channel lifecycles, which in turn can publish a welcome message back to that user on the chat channel.

In addition, various existing systems, such as Azure Functions, Google Functions, and AWS Lambda rely on HTTP events. Reactor Events will allow for simple integration with said systems.


Ably Reactor Events diagram

You can configure events from the Reactor tab in your app on a per app basis which can apply to one or more channels in that app. Reactor Events can be filtered by channel naming using a regular expression, for example ^click_.*_mouse$. This would match the string click_ followed by a string followed by _mouse, for example, click_left_mouse.

Available integrations

At present, in addition to support for any custom HTTP endpoint, we have ready-made integrations with the following services:

Configuring a webhook

Webhooks are configured from the Reactor tab in your app dashboard. The following fields are shared between each webhook:

URL
The URL of the endpoint where messages will be sent
Custom headers
Optionally allows you to provide a set of headers that will be included in all HTTP POST requests. You must use format name:value for each header you add, for example, X-Custom-Header:foo
Source
Choose which of Message, Presence, or Channel Lifecycle events on channels should activate this Reactor Event Rule. Channel Lifecycle events are only available in Batch Request mode
Request Mode
This will either be in Single Request mode or Batch Request mode. Single Request will send each event as separately to the endpoint specified by the Rule. Batch Request will roll up multiple events in the same request
Channel filter
An optional filter, which allows the Rule to be applied to a restricted set of channels. This can be specified as a regular expression, allowing for swathes of channels to be used
Encoding
The encoding to be used by this Rule. This can be either JSON or MsgPack. Encoding only applies to enveloped and batched messages


If the Rule is in the Single Request mode, it will also have the following options:

Enveloped
If the rule has the Enveloped option set, then data delivered by this Rule will be wrapped in an Ably envelope. Otherwise, the Rule will send the raw payload


If the Rule is in the Batch Request mode, it will have the following additional options:

Sign with key
Ably will optionally sign the data with the specified private key. This will be included as an HTTP header X-Ably-Signature in every HTTP post request issued to your server. See webhook security for more details.

Note that various integrations have restrictions on them which will mean some of these base options are either changed or absent. You can check specific details in each integration’s page.

Sources

Ably currently supports three types of Events:

  • Messages – messages trigger function calls as soon as they are published on a channel
  • Presence events – presence events trigger function calls when clients enter, update their data, or leave channels
  • Channel lifecycle events – get notified when a channel is created (following the first client attaching to this channel) or discarded (when there are no more clients attached to the channel). Currently this is only available for batched messages

Single vs Batched requests

If Single request is chosen for a Reactor Event Rule, then a POST request will be sent to your specified endpoint/service each time an event occurs. Although this can be useful for some use-cases where the endpoint can only process one message per request, or needs the event as soon as it’s available, it can result in the endpoint being overloaded with requests. To avoid this, it’s possible to instead make use of Batch request instead, which will batch messages sent within a set timeframe together.

Single request details

Single request is best suited for scenarios where you’re wanting a 1-to-1 relationship between sent messages and events being called. If you are making use of a serverless system which is expecting a single piece of data each time, and then intends to perform some transformation/event following that, then Single request will likely work well for you. If you’re using a single server, which has the potential to be overloaded by requests, and can process multiple events per payload sent, Batch request will be a better choice.

Rate limits
  • Free accounts are limited to 15 Reactor invocations per second on single requests, whilst paid are limited to 30.
  • Webhook requests are made with a default timeout of 15s. If the request fails or times out, Ably retries the request with exponential backoff (base delay 1s, backoff factor sqrt(2), up to a max of 60s)
  • Multiple requests can be in-flight at once, up to the max concurrency limit. If the number of in-flight requests exceeds the max concurrency limit, new messages coming in are placed in a short queue (length 10); if that queue length is exceeded, further messages are rejected
Failures and back off
  • If a request is rejected with 5xx or times out, it will be retried twice more, once after 4s, then if that fails, again after 20s

Batch request details

Batch requests are useful for endpoints which have the potential to be overloaded by requests, or simply have no preference requirement for processing messages sent one-by-one. If you are using an endpoint which has either of these requirements (for example IFTTT requires one event per request), you should use Single request.

Webhook batched requests are typically published at most once per second per configured webhook.

Rate limits
  • For each configured webhook, up to one request per second will be made to the configured endpoint URL
  • The first event that matches a configured webhook will trigger a webhook request immediately. Therefore, if you have a low volume of events you are listening to, in most cases your request should arrive in under a second from the time the event was generated
  • webhook requests are made with a default timeout of 15s. If the request fails or times out, Ably retries the request with exponential backoff (base delay 1s, backoff factor sqrt(2), up to a max of 60s)
  • Once a webhook request is triggered, all other events will be queued so that they can be delivered in a batch in the next request. The next webhook request will be issued within one second with the following caveats:
    • Only a limited number of http requests are in-flight at one time for each configured webhook. Therefore, if you want to be notified quickly, we recommend you accept requests quickly and defer any work to be done asynchronously
    • If there are more than 1,000 events queued for the next webhook, the oldest 1,000 events will be bundled into the next webhook and the remaining events will be delivered in the next webhook. Therefore, if your sustained rate of events is expected to be more than 1,000 per second or your servers are slow to respond, then it is possible a backlog will build up and you will not receive all events. Get in touch if you need a higher sustained rate.
Failures and back off
  • If the endpoint for any of the webhook requests respond with an HTTP status code that does not indicate success i.e. 200 - 209, then Ably will retry that failed request
  • Every retry is performed with an incrementing back off that is calculated as delay = delay * sqrt(2) where delay is initially 1. For example, if the initial webhook request fails, and subsequent for retries fail, the back off delays for each request would look as follows: initial request > wait 1.4s > 1st retry > wait 2s > 2nd retry > wait 2.8s > 3rd retry > wait 4s > 4th retry > wait 5.6s > successful request
  • The back off for consecutively failing requests will increase until it reaches 60s. All subsequent retries for failed requests will then be made every 60s until a request is successful
  • The queue of events is retained for 5 minutes. If an event cannot be delivered within 5 minutes, then the events are discarded to prevent the queue from growing indefinitely

Envelopes

When you configure a Reactor Event rule using single requests, you are given the option to envelope messages, which is enabled by default. In most cases, we believe an enveloped message provides more flexibility as it contains additional metadata in a portable format that can be useful such as the clientId of the publisher, or the channel name the message originated from.

However, if you don’t need anything besides the payload of each message, or the endpoint expects a very restricted data structure, you may choose not to envelope messages and instead have only the message payload (data element) published. This has the advantage of requiring one less parsing step, however decoding of the raw payload in the published message will be your responsibility.

Check out examples of enveloped and non-enveloped examples down below.

Payload Encoding

The encoding of payloads sent is defined when setting up a Reactor Rule in the Reactor tab of your app. This only applies to enveloped messages and their structure, non-enveloped messages will remain their original format. You can have the message sent in JSON format, or as a MessagePack payload.

  • JSON (JavaScript Object Notation): An efficient data-interchange format which is fairly standard and provides simple text based encoding.
  • MessagePack: An efficient binary serialization format that is similar to JSON, but smaller. This is notably useful for binary payloads, as a JSON envelope with a binary payload would need to have the payload base64-encoded

Webhook security

We encourage customers to use a secure HTTPS URL when configuring their Reactor Events. This will ensure that requests cannot be intercepted and all communication with your servers is secured with TLS.

However, in addition, we optionally support a signature included as an HTTP header X-Ably-Signature in batched requests. The endpoint can use the chosen private API key to verify the authenticity of the webhook data.

In order to verify the signature, you need to do the following:

  • start with the webhook request body. This will be a JSON string encoded with content-encoding utf-8;
  • identify the key based on the keyId indicated in the X-Ably-Key header;
  • calculate the HMAC of that request body with algorithm SHA-256 and the key being the corresponding keyValue (the secret part of the key after the “:”);
  • encode the resulting HMAC using RFC 3548 base 64;
  • compare that result with the signature value indicated in the X-Ably-Signature header

Webhook HMAC SHA-256 signature verification example

If you choose to sign your webhook requests, we recommend you try the following first:

  1. Set up a free RequestBin HTTP endpoint test URL
  2. Configure a webhook with the URL set to the RequestBin endpoint, and ensure you have chosen to batch messages and are using a key to sign each webhook request
  3. Trigger an event using the Dev Console in your app dashboard which will generate a webhook. You should then confirm that the webhook has been received in your RequestBin
  4. Check that the X-Ably-Signature header in your webhook request matches the HMAC SHA-256 you create using our Javascript HMAC SHA-256 demo

Examples

Given the various potential combinations of enveloped, batched and message sources, it can be good to know what to expect given certain combinations of rules.

Batched event payloads

Batched events will have the following headers:

content-type
the type of the payload. This can be either application/json, text/plain, or application/octet-stream, depending on if it’s JSON, text, or binary respectively
x-ably-envelope-appid
the app ID which the message came from
content-type
the type of the payload. This will be application/json or application/x-msgpack
x-ably-version
the version of Reactor Event. At present this should be 1.0, though older Events will be 0.8

Each batched message will have the following fields:

name
the event type, aka “presence.message”, “channel.message”, “channel.closed”, etc
webhookId
an internal unique ID for the configured webhook
source
the source for the webhook, namely “channel.message”, “channel.presence”, “channel.lifecycle
timestamp
a timestamp represented as milliseconds since the epoch for the presence event
data
an object containing the data of the event defined below in JSONPath format

Batched message events

For message events, data will contain:

data.channelId
name of the channel that the presence event belongs to
data.site
an internal site identifier indicating which primary datacenter the member is present in
data.messages
an Array of Message
{
  "items": [{
    "webhookId": "ABcDEf",
    "source": "channel.lifecycle",
    "timestamp": 1562124922426,
    "serial": "a7bcdEFghIjklm123456789:4",
    "name": "channel.message",
    "data": {
      "channelId": "channelName",
      "site": "eu-west-1-A",
      "messages": [{
        "id": "ABcDefgHIj:1:0",
        "connectionId": "ABcDefgHIj",
        "timestamp": 1123145678900,
        "data": "some message data",
        "name": "my message name"
      }]
    }
  }]
}

Batched presence events

For presence events, data will contain:

data.channelId
name of the channel that the presence event belongs to
data.site
an internal site identifier indicating which primary datacenter the member is present in
data.presence
an Array of Presence events
{
  "items": [{
    "webhookId": "ABcDEf",
    "source": "channel.lifecycle",
    "timestamp": 1562124922426,
    "serial": "a7bcdEFghIjklm123456789:4",
    "name": "presence.message",
    "data": {
      "channelId": "education",
      "site": "eu-west-1-A",
      "presence": [{
        "id": "ABcDefgHIj:1:0",
        "connectionId": "ABcDefgHIj",
        "timestamp": 1123145678900,
        "clientId": "bob",
        "data": "some message data",
        "action": 4
      }]
    }
  }]
}

Batched channel lifecycle events

For channel lifecycle events, data will contain:

data.channelId
name of the channel that the presence event belongs to
data.status
a ChannelStatus object
{
  "items": [{
    "webhookId": "ABcDEf",
    "source": "channel.lifecycle",
    "timestamp": 1562124922426,
    "serial": "a7bcdEFghIjklm123456789:4",
    "name": "channel.opened",
    "data": {
      "channelId": "channelName",
      "name": "channelName",
      "status": {
        "isActive": true,
        "occupancy": {
          "metrics": {
            "connections": 1,
            "publishers": 1,
            "subscribers": 1,
            "presenceConnections": 1,
            "presenceMembers": 0,
            "presenceSubscribers": 1
          }
        }
      }
    }
  }]
}

Enveloped event payloads

Enveloped events will have the following headers:

content-type
the type of the payload. This can be either application/json, text/plain, or application/octet-stream, depending on if it’s JSON, text, or binary respectively
x-ably-version
the version of Reactor Event. At present this should be 1.0, though older Events will be 0.8
x-ably-envelope-appid
the app ID which the message came from
content-type
the type of the payload. This will be application/json or application/x-msgpack for enveloped messages

Each enveloped message will have the following fields:

source
the source for the webhook, namely “channel.message” or “channel.presence
appId
the Ably app this message came from
channel
the Ably channel where the event occurred
site
the Ably datacenter which sent the message
timestamp
a timestamp represented as milliseconds since the epoch for the presence event

In addition, it will contain another field which will contain the actual message, which is named according to the message type.

Enveloped message events

For message events, there will be a messages field, which will contain the Message event.

{
  "source": "channel.message",
  "appId": "aBCdEf",
  "channel": "channel-name",
  "site": "eu-central-1-A",
  "ruleId": "1-a2Bc",
  "messages": [{
    "id": "ABcDefgHIj:1:0",
    "connectionId": "ABcDefgHIj",
    "timestamp": 1123145678900,
    "data": "some message data",
    "name": "my message name"
  }]
}

Enveloped presence events

For presence events, there will be a presence field, which will contain the Presence event.

{
  "source": "channel.message",
  "appId": "aBCdEf",
  "channel": "channel-name",
  "site": "eu-central-1-A",
  "ruleId": "1-a2Bc",
  "presence": [{
    "id": "abCdEFgHIJ:1:0",
    "clientId": "bob",
    "connectionId": "Ab1CDE2FGh",
    "timestamp": 1582270137276,
    "data": "some data in the presence object",
    "action": 4
  }]
}

Non-enveloped event payloads

Non-enveloped events have quite a few headers, in order to provide context to the data sent in the payload. These are:

content-type
the type of the payload. This can be either application/json, text/plain, or application/octet-stream, depending on if it’s JSON, text, or binary respectively
x-ably-version
the version of Reactor Event. At present this should be 1.0, though older Events will be 0.8
x-ably-envelope-appid
the app ID which the message came from
x-ably-envelope-channel
the Ably channel which the message came from
x-ably-envelope-rule-id
the Ably Reactor Rule ID which was activated to send this message
x-ably-envelope-site
the Ably datacenter which sent the message
x-ably-envelope-source
the source for the webhook, namely “channel.message” or “channel.presence
x-ably-message-client-id
the client ID of the connection which sent the event
x-ably-message-connection-id
the connection ID responsible for the initial event
x-ably-message-id
the message’s unique ID
x-ably-message-timestamp
the time the message was originally sent

Non-enveloped message events

For message events, there will be the additional headers:

x-ably-message-name
The name of the Message

The payload will contain the data of the Message.

For example, if you sent the following curl message, which sends a JSON message to the channel my_channel:

curl -X POST https://rest.ably.io/channels/my_channel/messages \
          -u "xVLyHw.Z2lYqg:8h8qiva3GB5o9lrf" \
          -H "Content-Type: application/json" \
          --data '{ "name": "publish", "data": "example" }'

The x-ably-message-name header would be publish, and the payload would be example.

Non-enveloped presence events

For Presence events, there will be the additional headers:

x-ably-message-action
the action performed by the event (update, enter, leave)

The payload will contain the data of the Presence message.

For example, if a client enters a channel’s presence with the following code:

realtime = new Ably.Realtime({ key: 'xVLyHw.Z2lYqg:8h8qiva3GB5o9lrf', clientId: 'bob' });
channel = realtime.channels.get('some_channel');
channel.presence.enter('some data');
realtime = new Ably.Realtime({ key: 'xVLyHw.Z2lYqg:8h8qiva3GB5o9lrf', clientId: 'bob' });
channel = realtime.channels.get('some_channel');
channel.presence.enter('some data');

Then the x-ably-message-action would be enter, the x-ably-message-client-id would be “bob”, and the payload would be “some data”.


API reference
Documentation