Skip to content
This repository was archived by the owner on Aug 10, 2026. It is now read-only.

Feature deltas#873

Closed
paddybyers wants to merge 2 commits into from
Closed

Feature deltas #873

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
d779f19
Deltas: first draft of documentation
paddybyers Apr 21, 2020
eab0ce1
Channel params: restructure information on rewind to reflect the new …
paddybyers Apr 21, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion content/partials/types/_channel_options.textile
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
Currently the supported channel options are only used for "configuring encryption":/realtime/encryption.
Channel options are used for specifying "channel params":/realtime/channel-params and for "configuring encryption":/realtime/encryption.

blang[jsall].
@ChannelOptions@, a plain Javascript object, may optionally be specified when instancing a "@Channel@":/realtime/channels, and this may be used to specify channel-specific options. The following attributes can be defined on the object:
Expand Down
211 changes: 30 additions & 181 deletions content/realtime/channel-params.textile
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -5,11 +5,9 @@ index: 32
jump_to:
Help with:
- Overview#overview
- Using params with v1.2 or later Ably libraries#using-params-with-lib-v12
- Using params with v1.1 or earlier Ably libraries#using-params-with-lib-v11
- Using params with non-Ably transports#using-params-with-other-transports
- Examples#examples
Parameters:
- rewind#rewind
---

h2(#overview). Overview
Expand All@@ -20,12 +18,38 @@ The methods provided for specifying channel parameters, and the currently availa

h2(#supported-params). Currently supported channel params

- rewind := Used to request that an attachment start from some number of messages or point in time in the past. See "rewind":#rewind for more information
- delta := **In an experimental state**. Used to request that data payloads should be sent as deltas to the previous payload. "Contact us":https://www.ably.io/contact for more information and supported values
- rewind := Used to request that an attachment start from some number of messages or point in time in the past. See "rewind":#./rewind for more information
- delta := Used to request that data payloads should be sent as deltas to the previous payload. See "delta":./delta for more information

h2(#using-params-with-lib-v12). Using channel params with v1.2 or later Ably client libraries

Channel params may be specified in the @ChannelOptions@ when obtaining a @Channel@. A collection of channel params is expressed as a map of string key/value pairs. The @ChannelOptions@ associated with a channel may also be updated by calling "setOptions":./channel#setOptions. The params associated with a channel take effect when the channel is first attached; if the params are subsequently modified via a call to "setOptions":./channel#setOptions, then that call triggers attach operation that applies the updated params, if successful.

@QuintinWillisonQuintinWillisonApr 24, 2020

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clicking the 'setOptions' link I get:

File not found: /realtime/channel-params/channel

when viewing a local build (via bundle exec nanoc compile && bundle exec nanoc view -p 4000) at http://localhost:4000/realtime/channel-params/channel#setOptions

The Heroku preview presents that as:

404 Not Found : nginx


h3. Example

For example, to specify the @rewind@ channel param with the value @"1"@:

```[javascript]
const realtime = new Ably.Realtime('{{API_KEY}}');
const channelOpts = {params: {rewind: '1'}}
const channel = realtime.channels.get('{{RANDOM_CHANNEL_NAME}}', channelOpts);
```

To modify the @rewind@ channel param with the value @"15s"@:

```[javascript]
const realtime = new Ably.Realtime('{{API_KEY}}');
const channelOpts = {params: {rewind: '15s'}}
channel.setOptions(channelOpts, (err) => {
if(!err) {
console.log('channel params updated')
}
});
```

h2(#using-params-with-lib-v11). Using channel params with v1.1 or earlier Ably client libraries

The current Ably libraries, at version 1.1, do not expose an API for expressing channel parameters. This means that it is necessary to specify parameters in a way that is opaque to the library.
The Ably libraries at version 1.1 and earlier, do not expose a the API for expressing channel parameters programmatically. This means that it is necessary to specify parameters in a way that is opaque to the library.

@SimonWoolfSimonWoolfApr 21, 2020

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and earlier, do not expose

superflous comma

a the


A set of params is expressed by including a query string, using standard URL query syntax and encoding, within the qualifier part of a channel name. The qualifier part is in square brackets at the start of the channel name.

Expand DownExpand Up@@ -70,178 +94,3 @@ Or to specify the same parameter but only applying to one channel of two, using
var querystring = 'v=1.2&key={{API_KEY}}&channels=' + channels';
var eventSource = new EventSource('https://realtime.ably.io/event-stream?' + querystring);
```

h2(#rewind). Rewind parameter

The @rewind@ channel parameter relates to the initial attachment of a connection to a channel, and expresses the intent to attach to the channel at a position, or a point in time, in the past (that is, effectively "rewinding" the channel for the purposes of the present attachment).

A @rewind@ parameter can express a channel position in terms of a number of messages, or a time interval.

A @rewind@ value that is simply a number @n@ (eg @rewind=1@) is a request to attach to the channel at a position @n@ messages before the present position. If that attachment is successful, and one or more messages exist on the channel prior to the present position, then those messages will be delivered to the subscriber immediately after the attachment has completed, and before any subsequent messages that arise in real time.

If fewer than the requested number of messages exists on the channel (including the case that there are no prior messages), then the available messages are sent; this does not constitute an error.

A @rewind@ value can also be a string that is a time interval specifier. Supported specifier values express an integral number of seconds (eg @15s@) or minutes (eg @2m@). If that attachment is successful, and one or more messages exist on the channel in the given time interval prior to the present time, then those messages will be delivered to the subscriber immediately after the attachment has completed, and before any subsequent messages that arise in real time.

If you wish to use a time interval rewind but additionally specify a limit on the number of messages to be returned, you can use the @rewindLimit@ channel param. For example, to request up to 10 messages in a window 5m before the present time, specify a channel parameter string of @rewind=5m&rewindLimit=10@. If fewer than the requested number of messages exists on the channel in that interval (including the case that there are no messages), then the available messages are sent; this does not constitute an error.

At most 100 messages will be sent in a rewind request. If the number of messages within the specified interval is greater than that limit, then only the most recent messages up to that limit are sent. The attachment succeeds, but truncation of the message backlog is indicated as a non-fatal error in the attachment response.

By default, a maximum of two minutes of channel history is available when attaching. This means that a rewind time specifier of greater than two minutes will only be able to rewind by two minutes. If a channel has persistence enabled, then it is possible to rewind back in time by up to the persistence TTL on the channel.

The channel position expressed by a @rewind@ parameter has an effect only on an initial channel attachment. Any subsequent reattachment of the same channel on the same connection, in order to resume the connection, will attempt to resume with continuity from the point at which the connection dropped. (There are a few exceptions to this: in particular, client libraries earlier than v1.2 that have been disconnected for over two minutes, and all clients when using "@recover@ mode":/realtime/connection#connection-state-recovery ; in both cases the previous attachment state is not preserved).

Any @rewind@ parameter value that cannot be parsed either as a number or a time specifier represents an error, and any attachment request will fail with an error.

h3(#rewind-example-ably). Rewind example with an Ably client library

To subscribe to a channel, getting the most recent message if available:

```[jsall]
// only with ably-js v1.2 or later
const realtime = new Ably.Realtime('{{API_KEY}}');
realtime.channels.get('{{RANDOM_CHANNEL_NAME}}', {
rewind: '1'
}).subscribe(msg => console.log("Received message: ", msg));
```

```[jsall]
// with ably-js v1.1 or below
const realtime = new Ably.Realtime('{{API_KEY}}');
const channel = realtime.channels.get('[?rewind=1]{{RANDOM_CHANNEL_NAME}}');
channel.subscribe(msg => console.log("Received message: ", msg));
```

h3(#rewind-example-sse). Rewind example with SSE

To subscribe to a channel, getting the most recent message if available:

```[javascript]
var querystring = 'v=1.2&channels={{RANDOM_CHANNEL_NAME}}&rewind=1&key={{API_KEY}}';
var eventSource = new EventSource('https://realtime.ably.io/event-stream?' + querystring);
```

h3(#rewind-examples-mqtt). Rewind example with MQTT

```[nodejs]
var mqtt = require('mqtt');
var options = {
keepalive: 30,
username: 'FIRST_HALF_OF_API_KEY',
password: 'SECOND_HALF_OF_API_KEY',
port: 8883
};
var client = mqtt.connect('mqtts:mqtt.ably.io', options);
client.on('connect', () => {
client.subscribe('[?rewind=1]{{RANDOM_CHANNEL_NAME}}');
});
client.on('message', (topic, message) => {
...
});
```

h2(#delta). Delta parameter

The @delta@ channel parameter allows subscribers to expresse their desire to receive deltas (diffs) between the previous and current message instead of the current message in full on a given channel. The effects of this parameter are invisible to message senders and only affect subscribers that specify it - i.e. the sender sends messages in full and Ably computes and sends the message deltas to any subscriber which subscribed using this parameter.

Note that requesting deltas does not guarantee that every message received will be a delta, as the server may chose to send some messages in full depending on system load and other factors. Therefore a subscriber using the @delta@ parameter should be prepared to handle a random mix of delta and full messages.

The @delta@ parameter can take only the value @vcdiff@. VCDIFF is the format of the deltas. It is an open format specified in "RFC 3284":https://tools.ietf.org/html/rfc3284.

Ably provides a JavaScript "delta codec library":https://github.com/ably/delta-codec-js to help you avoid writing boilerplate VCDIFF handling code. Some examples follow.

h3(#delta-example-sse). Delta example with enveloped SSE

```[jsall]
const key = '{{API_KEY}}';
const channel = 'sample-app-sse';
const url = `https://realtime.ably.io/event-stream?channels=${channel}&v=1.2&key=${key}&delta=vcdiff`;
const eventSource = new EventSource(url);
const channelDecoder = new DeltaCodec.CheckedVcdiffDecoder();

eventSource.onmessage = (event) => {
/* event.data is JSON-encoded Ably Message (see https://www.ably.io/documentation/realtime/types#message) */
const message = JSON.parse(event.data);
const { id, extras } = message;
let { data } = message;

try {
if (extras && extras.delta) {
data = channelDecoder.applyBase64Delta(data, id, extras.delta.from).asUtf8String();
} else {
channelDecoder.setBase(data, id);
}
} catch(e) {
/* Delta decoder error */
console.log(e);
}

/* Process decoded data */
console.log(data);
};
```

h3(#delta-example-unenv-sse). Delta example with unenveloped SSE

```[jsall]
const key = '{{API_KEY}}';
const channel = 'sample-app-sse';
const url = `https://realtime.ably.io/event-stream?channels=${channel}&v=1.2&key=${key}&delta=vcdiff&enveloped=false`;
const eventSource = new EventSource(url);
const channelDecoder = new DeltaCodec.VcdiffDecoder();

eventSource.onmessage = (event) => {
let data = event.data;

try {
if (DeltaCodec.VcdiffDecoder.isBase64Delta(data)) {
data = channelDecoder.applyBase64Delta(data).asUtf8String();
} else {
channelDecoder.setBase(data);
}
} catch(e) {
/* Delta decoder error */
console.log(e);
}

/* Process decoded data */
console.log(data);
};
```

h3(#delta-example-mqtt). Delta example with MQTT

```[jsall]
const mqtt = require('mqtt');
const { VcdiffDecoder } = require('./lib');

const options = {
keepalive: 30,
username: 'FIRST_HALF_OF_API_KEY',
password: 'SECOND_HALF_OF_API_KEY',
port: 8883
};
const client = mqtt.connect('mqtts:mqtt.ably.io', options);
const channelName = 'sample-app-mqtt';
const channelDecoder = new VcdiffDecoder();

client.on('message', (_, payload) => {
let data = payload;

try {
if (VcdiffDecoder.isDelta(data)) {
data = channelDecoder.applyDelta(data).asUint8Array();
} else {
channelDecoder.setBase(data);
}
} catch(e) {
/* Delta decoder error */
console.log(e);
}

/* Process decoded data */
console.log(data);
});

client.subscribe(`[?delta=vcdiff]${channelName}`);
```
21 changes: 19 additions & 2 deletions content/realtime/channels.textile
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -140,9 +140,22 @@ bc[objc]. ARTRealtimeChannel *channel = [realtime.channels get:@"channelName"];

bc[swift]. let channel = realtime.channels.get("channelName")

h4(#setting-channel). Setting channel options and encryption
h4(#setting-channel). Setting channel options

A set of "channel options":#channel-options may also be passed to configure a channel for encryption. Find out more about "symmetric message encryption":/realtime/encryption.
A set of "channel options":#channel-options also be passed to specify options a channel when the channel is first obtained via "channels.get":#obtaining-channel. The options associated with a given channel may also be updated at any time after creation via "channel.setOptions":#setOptions

@SimonWoolfSimonWoolfApr 21, 2020

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also be passed

to specify options a channel

@QuintinWillisonQuintinWillisonApr 24, 2020

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, the setOptions anchor doesn't appear on this page from what I can see.


h5(#setting-channel-params). Setting channel params

Channel parameters are a general mechanism by which a client can express properties of a channel, or of its attachment to a channel. The currently supported params are "rewind":./channel-params#rewind or "delta generation":./delta.

@QuintinWillisonQuintinWillisonApr 24, 2020

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neither of these links work. Both 404.

Out of interest why doesn't the build fail on this? Surely something in the docs build chain should be able to detect self-referential issues? cc @tomczoink@Ugbot


bc[jsall]. const realtime = new Ably.Realtime('{{API_KEY}}');
const channelOpts = {params: {rewind: '1'}}
const channel = realtime.channels.get('{{RANDOM_CHANNEL_NAME}}', channelOpts);

h5(#setting-channel-params). Setting channel encryption options

It is possible to enable encryption on a channel via the channel options.
Channel options include channel parameters - such as to specify "rewind":./channel-params#rewind or "delta generation":./delta - and options that enable for encryption. Find out more about "symmetric message encryption":/realtime/encryption.

bc[jsall]. Ably.Realtime.Crypto.generateRandomKey(function(err, key) {
var options = { cipher: { key: key } };
Expand DownExpand Up@@ -515,6 +528,10 @@ channel.publish("action", data: "boom!")

Normally, errors in attaching to a channel are communicated through the attach callback. For implicit attaches (and other cases where a channel is attached or reattached automatically, e.g. following the library reconnecting after a period in the @suspended@ state), there is no callback, so if you want to know what happens, you'll need to listen for channel state changes.

h3(#set-options). Modifying channel options

It is possible to modify the @ChannelOptions@ associated with a given channel instance by calling @setOptions@ and passing a new @ChannelOptions@. The modified options take effect at the time of attachment (if an attach for that channel has not yet been initiated), or the @setOptions@ call will trigger an immediate attach operation to apply the modified options. Success or failure of any triggered attach operation triggered is indicated in the result of the @setOptions@ call.

h3(#multi-publish). Publishing to multiple channels

Often it is necessary to publish a single message in multiple channels at the same time. In the realtime API, this is achieved simply by making multiple separate "publish":#publish requests. If a separate publish is made in each of the channels in question, the realtime protocol will allow for those concurrent requests to be in-flight simultaneously. This ensures that a publish on a channel is not delayed waiting for completion of operations in other channels.
Expand Down
Loading