diff --git a/content/partials/types/_channel_options.textile b/content/partials/types/_channel_options.textile index 2ff18322ef..c64c92dd76 100644 --- a/content/partials/types/_channel_options.textile +++ b/content/partials/types/_channel_options.textile @@ -1,4 +1,4 @@ -Currently the supported channel options are only used for "configuring encryption":/realtime/encryption. +Channel options are used for specifying "channel params":/realtime/channel-params and for "configuring encryption":/realtime/encryption. blang[jsall]. @ChannelOptions@, a plain Javascript object, may optionally be specified when instancing a "@Channel@":/realtime/channels, and this may be used to specify channel-specific options. The following attributes can be defined on the object: diff --git a/content/realtime/channel-params.textile b/content/realtime/channel-params.textile index 19e473054a..bf13e1cd78 100644 --- a/content/realtime/channel-params.textile +++ b/content/realtime/channel-params.textile @@ -5,11 +5,9 @@ index: 32 jump_to: Help with: - Overview#overview + - Using params with v1.2 or later Ably libraries#using-params-with-lib-v12 - Using params with v1.1 or earlier Ably libraries#using-params-with-lib-v11 - Using params with non-Ably transports#using-params-with-other-transports - - Examples#examples - Parameters: - - rewind#rewind --- h2(#overview). Overview @@ -20,12 +18,38 @@ The methods provided for specifying channel parameters, and the currently availa h2(#supported-params). Currently supported channel params -- rewind := Used to request that an attachment start from some number of messages or point in time in the past. See "rewind":#rewind for more information -- delta := **In an experimental state**. Used to request that data payloads should be sent as deltas to the previous payload. "Contact us":https://www.ably.io/contact for more information and supported values +- rewind := Used to request that an attachment start from some number of messages or point in time in the past. See "rewind":#./rewind for more information +- delta := Used to request that data payloads should be sent as deltas to the previous payload. See "delta":./delta for more information + +h2(#using-params-with-lib-v12). Using channel params with v1.2 or later Ably client libraries + +Channel params may be specified in the @ChannelOptions@ when obtaining a @Channel@. A collection of channel params is expressed as a map of string key/value pairs. The @ChannelOptions@ associated with a channel may also be updated by calling "setOptions":./channel#setOptions. The params associated with a channel take effect when the channel is first attached; if the params are subsequently modified via a call to "setOptions":./channel#setOptions, then that call triggers attach operation that applies the updated params, if successful. + +h3. Example + +For example, to specify the @rewind@ channel param with the value @"1"@: + +```[javascript] + const realtime = new Ably.Realtime('{{API_KEY}}'); + const channelOpts = {params: {rewind: '1'}} + const channel = realtime.channels.get('{{RANDOM_CHANNEL_NAME}}', channelOpts); +``` + +To modify the @rewind@ channel param with the value @"15s"@: + +```[javascript] + const realtime = new Ably.Realtime('{{API_KEY}}'); + const channelOpts = {params: {rewind: '15s'}} + channel.setOptions(channelOpts, (err) => { + if(!err) { + console.log('channel params updated') + } + }); +``` h2(#using-params-with-lib-v11). Using channel params with v1.1 or earlier Ably client libraries -The current Ably libraries, at version 1.1, do not expose an API for expressing channel parameters. This means that it is necessary to specify parameters in a way that is opaque to the library. +The Ably libraries at version 1.1 and earlier, do not expose a the API for expressing channel parameters programmatically. This means that it is necessary to specify parameters in a way that is opaque to the library. A set of params is expressed by including a query string, using standard URL query syntax and encoding, within the qualifier part of a channel name. The qualifier part is in square brackets at the start of the channel name. @@ -70,178 +94,3 @@ Or to specify the same parameter but only applying to one channel of two, using var querystring = 'v=1.2&key={{API_KEY}}&channels=' + channels'; var eventSource = new EventSource('https://realtime.ably.io/event-stream?' + querystring); ``` - -h2(#rewind). Rewind parameter - -The @rewind@ channel parameter relates to the initial attachment of a connection to a channel, and expresses the intent to attach to the channel at a position, or a point in time, in the past (that is, effectively "rewinding" the channel for the purposes of the present attachment). - -A @rewind@ parameter can express a channel position in terms of a number of messages, or a time interval. - -A @rewind@ value that is simply a number @n@ (eg @rewind=1@) is a request to attach to the channel at a position @n@ messages before the present position. If that attachment is successful, and one or more messages exist on the channel prior to the present position, then those messages will be delivered to the subscriber immediately after the attachment has completed, and before any subsequent messages that arise in real time. - -If fewer than the requested number of messages exists on the channel (including the case that there are no prior messages), then the available messages are sent; this does not constitute an error. - -A @rewind@ value can also be a string that is a time interval specifier. Supported specifier values express an integral number of seconds (eg @15s@) or minutes (eg @2m@). If that attachment is successful, and one or more messages exist on the channel in the given time interval prior to the present time, then those messages will be delivered to the subscriber immediately after the attachment has completed, and before any subsequent messages that arise in real time. - -If you wish to use a time interval rewind but additionally specify a limit on the number of messages to be returned, you can use the @rewindLimit@ channel param. For example, to request up to 10 messages in a window 5m before the present time, specify a channel parameter string of @rewind=5m&rewindLimit=10@. If fewer than the requested number of messages exists on the channel in that interval (including the case that there are no messages), then the available messages are sent; this does not constitute an error. - -At most 100 messages will be sent in a rewind request. If the number of messages within the specified interval is greater than that limit, then only the most recent messages up to that limit are sent. The attachment succeeds, but truncation of the message backlog is indicated as a non-fatal error in the attachment response. - -By default, a maximum of two minutes of channel history is available when attaching. This means that a rewind time specifier of greater than two minutes will only be able to rewind by two minutes. If a channel has persistence enabled, then it is possible to rewind back in time by up to the persistence TTL on the channel. - -The channel position expressed by a @rewind@ parameter has an effect only on an initial channel attachment. Any subsequent reattachment of the same channel on the same connection, in order to resume the connection, will attempt to resume with continuity from the point at which the connection dropped. (There are a few exceptions to this: in particular, client libraries earlier than v1.2 that have been disconnected for over two minutes, and all clients when using "@recover@ mode":/realtime/connection#connection-state-recovery ; in both cases the previous attachment state is not preserved). - -Any @rewind@ parameter value that cannot be parsed either as a number or a time specifier represents an error, and any attachment request will fail with an error. - -h3(#rewind-example-ably). Rewind example with an Ably client library - -To subscribe to a channel, getting the most recent message if available: - -```[jsall] - // only with ably-js v1.2 or later - const realtime = new Ably.Realtime('{{API_KEY}}'); - realtime.channels.get('{{RANDOM_CHANNEL_NAME}}', { - rewind: '1' - }).subscribe(msg => console.log("Received message: ", msg)); -``` - -```[jsall] - // with ably-js v1.1 or below - const realtime = new Ably.Realtime('{{API_KEY}}'); - const channel = realtime.channels.get('[?rewind=1]{{RANDOM_CHANNEL_NAME}}'); - channel.subscribe(msg => console.log("Received message: ", msg)); -``` - -h3(#rewind-example-sse). Rewind example with SSE - -To subscribe to a channel, getting the most recent message if available: - -```[javascript] - var querystring = 'v=1.2&channels={{RANDOM_CHANNEL_NAME}}&rewind=1&key={{API_KEY}}'; - var eventSource = new EventSource('https://realtime.ably.io/event-stream?' + querystring); -``` - -h3(#rewind-examples-mqtt). Rewind example with MQTT - -```[nodejs] - var mqtt = require('mqtt'); - var options = { - keepalive: 30, - username: 'FIRST_HALF_OF_API_KEY', - password: 'SECOND_HALF_OF_API_KEY', - port: 8883 - }; - var client = mqtt.connect('mqtts:mqtt.ably.io', options); - client.on('connect', () => { - client.subscribe('[?rewind=1]{{RANDOM_CHANNEL_NAME}}'); - }); - client.on('message', (topic, message) => { - ... - }); -``` - -h2(#delta). Delta parameter - -The @delta@ channel parameter allows subscribers to expresse their desire to receive deltas (diffs) between the previous and current message instead of the current message in full on a given channel. The effects of this parameter are invisible to message senders and only affect subscribers that specify it - i.e. the sender sends messages in full and Ably computes and sends the message deltas to any subscriber which subscribed using this parameter. - -Note that requesting deltas does not guarantee that every message received will be a delta, as the server may chose to send some messages in full depending on system load and other factors. Therefore a subscriber using the @delta@ parameter should be prepared to handle a random mix of delta and full messages. - -The @delta@ parameter can take only the value @vcdiff@. VCDIFF is the format of the deltas. It is an open format specified in "RFC 3284":https://tools.ietf.org/html/rfc3284. - -Ably provides a JavaScript "delta codec library":https://github.com/ably/delta-codec-js to help you avoid writing boilerplate VCDIFF handling code. Some examples follow. - -h3(#delta-example-sse). Delta example with enveloped SSE - -```[jsall] -const key = '{{API_KEY}}'; -const channel = 'sample-app-sse'; -const url = `https://realtime.ably.io/event-stream?channels=${channel}&v=1.2&key=${key}&delta=vcdiff`; -const eventSource = new EventSource(url); -const channelDecoder = new DeltaCodec.CheckedVcdiffDecoder(); - -eventSource.onmessage = (event) => { - /* event.data is JSON-encoded Ably Message (see https://www.ably.io/documentation/realtime/types#message) */ - const message = JSON.parse(event.data); - const { id, extras } = message; - let { data } = message; - - try { - if (extras && extras.delta) { - data = channelDecoder.applyBase64Delta(data, id, extras.delta.from).asUtf8String(); - } else { - channelDecoder.setBase(data, id); - } - } catch(e) { - /* Delta decoder error */ - console.log(e); - } - - /* Process decoded data */ - console.log(data); -}; -``` - -h3(#delta-example-unenv-sse). Delta example with unenveloped SSE - -```[jsall] -const key = '{{API_KEY}}'; -const channel = 'sample-app-sse'; -const url = `https://realtime.ably.io/event-stream?channels=${channel}&v=1.2&key=${key}&delta=vcdiff&enveloped=false`; -const eventSource = new EventSource(url); -const channelDecoder = new DeltaCodec.VcdiffDecoder(); - -eventSource.onmessage = (event) => { - let data = event.data; - - try { - if (DeltaCodec.VcdiffDecoder.isBase64Delta(data)) { - data = channelDecoder.applyBase64Delta(data).asUtf8String(); - } else { - channelDecoder.setBase(data); - } - } catch(e) { - /* Delta decoder error */ - console.log(e); - } - - /* Process decoded data */ - console.log(data); -}; -``` - -h3(#delta-example-mqtt). Delta example with MQTT - -```[jsall] -const mqtt = require('mqtt'); -const { VcdiffDecoder } = require('./lib'); - -const options = { - keepalive: 30, - username: 'FIRST_HALF_OF_API_KEY', - password: 'SECOND_HALF_OF_API_KEY', - port: 8883 -}; -const client = mqtt.connect('mqtts:mqtt.ably.io', options); -const channelName = 'sample-app-mqtt'; -const channelDecoder = new VcdiffDecoder(); - -client.on('message', (_, payload) => { - let data = payload; - - try { - if (VcdiffDecoder.isDelta(data)) { - data = channelDecoder.applyDelta(data).asUint8Array(); - } else { - channelDecoder.setBase(data); - } - } catch(e) { - /* Delta decoder error */ - console.log(e); - } - - /* Process decoded data */ - console.log(data); -}); - -client.subscribe(`[?delta=vcdiff]${channelName}`); -``` diff --git a/content/realtime/channels.textile b/content/realtime/channels.textile index 0332e35ed4..fcf6929abc 100644 --- a/content/realtime/channels.textile +++ b/content/realtime/channels.textile @@ -140,9 +140,22 @@ bc[objc]. ARTRealtimeChannel *channel = [realtime.channels get:@"channelName"]; bc[swift]. let channel = realtime.channels.get("channelName") -h4(#setting-channel). Setting channel options and encryption +h4(#setting-channel). Setting channel options -A set of "channel options":#channel-options may also be passed to configure a channel for encryption. Find out more about "symmetric message encryption":/realtime/encryption. +A set of "channel options":#channel-options also be passed to specify options a channel when the channel is first obtained via "channels.get":#obtaining-channel. The options associated with a given channel may also be updated at any time after creation via "channel.setOptions":#setOptions + +h5(#setting-channel-params). Setting channel params + +Channel parameters are a general mechanism by which a client can express properties of a channel, or of its attachment to a channel. The currently supported params are "rewind":./channel-params#rewind or "delta generation":./delta. + +bc[jsall]. const realtime = new Ably.Realtime('{{API_KEY}}'); +const channelOpts = {params: {rewind: '1'}} +const channel = realtime.channels.get('{{RANDOM_CHANNEL_NAME}}', channelOpts); + +h5(#setting-channel-params). Setting channel encryption options + +It is possible to enable encryption on a channel via the channel options. +Channel options include channel parameters - such as to specify "rewind":./channel-params#rewind or "delta generation":./delta - and options that enable for encryption. Find out more about "symmetric message encryption":/realtime/encryption. bc[jsall]. Ably.Realtime.Crypto.generateRandomKey(function(err, key) { var options = { cipher: { key: key } }; @@ -515,6 +528,10 @@ channel.publish("action", data: "boom!") Normally, errors in attaching to a channel are communicated through the attach callback. For implicit attaches (and other cases where a channel is attached or reattached automatically, e.g. following the library reconnecting after a period in the @suspended@ state), there is no callback, so if you want to know what happens, you'll need to listen for channel state changes. +h3(#set-options). Modifying channel options + +It is possible to modify the @ChannelOptions@ associated with a given channel instance by calling @setOptions@ and passing a new @ChannelOptions@. The modified options take effect at the time of attachment (if an attach for that channel has not yet been initiated), or the @setOptions@ call will trigger an immediate attach operation to apply the modified options. Success or failure of any triggered attach operation triggered is indicated in the result of the @setOptions@ call. + h3(#multi-publish). Publishing to multiple channels Often it is necessary to publish a single message in multiple channels at the same time. In the realtime API, this is achieved simply by making multiple separate "publish":#publish requests. If a separate publish is made in each of the channels in question, the realtime protocol will allow for those concurrent requests to be in-flight simultaneously. This ensures that a publish on a channel is not delayed waiting for completion of operations in other channels. diff --git a/content/realtime/delta.textile b/content/realtime/delta.textile new file mode 100644 index 0000000000..84e4008670 --- /dev/null +++ b/content/realtime/delta.textile @@ -0,0 +1,222 @@ +--- +title: Delta mode subscription +section: realtime +index: 32 +jump_to: + Help with: + - Overview#overview + - Examples#examples + Parameters: + - delta#delta +--- + +h2(#overview). Overview + +Often a channel carries messages that represent a series updates to a particular object or document and, as such, there is a significant degree of similarily between successive messages. Delta mode is a way for a client to subscribe to a channel so that message payloads sent over the wire contain only the difference (ie the delta) between the present message and the previous message on the channel. The client can then apply the delta to the previous message to obtain the full payload. Using delta mode can signficantly reduce the encoded size of each message in the case that message payloads change by differences that are small relative to the size of the value. This reduction in size can reduce bandwidth costs, reduce transit latencies, and enable greater message throughput on a connection. + +The present delta mode implementation supports a single representation of a delta, [VCDIFF](https://tools.ietf.org/html/rfc3284). However, the protocol and API are designed to allow other representations to be used in the future. + +Since version 1.2, Ably libraries support delta subscriptions; subscribing in delta mode is enabled for a given channel by specifying a [delta channel param](./channel-params) with the value @vcdiff@. This will cause delta messages to be generated by the server and sent to the client, and the library reconstitutes the original message payload. The end result is that messages on the channel are delivered to the subscriber's listener in just the same way as happens with a normal subscription. + +h2(#delta-processing). Delta processing + +Deltas apply to the principal payload of a @Message@ published via Ably, which is the @data@ member. Other elements of a message, such as @clientId@, @name@, or @extras@ are unchanged by use of deltas and are not compressed. + +Deltas are supported for realtime subscriptions only. Messages retrieved via the history API, and messages delivered to Reactor endpoints, are not compressed. Support for delta-compressed messages via Reactor is under consideration for the future. + +Delta compression via @vcdiff@ is supported for all payloads, whether string or binary, or JSON-encoded. The delta algorithm processes message payloads as opaque binaries and has no dependency on the stucture of the payload - it does not process line-oriented diffs, for example. In principle, @vcdiff@ deltas can be applied to encrypted message payloads, but in practice this provides no benefit because there is no similarity between successive encrypted payloads even on identical or near-identical plaintext message payloads. + +Delta compression is a subscriber-specified option only - the publisher has no control over whether or not deltas are generated for any given message; the processing is performed if there is at least one subscriber on a channel that has requested a delta-mode subscription. + +There is no constraint on how many publishers or subscribers there are. If there are multiple publishers, then deltas can still be generated, and they will be determined based on the order of messages in the channel in question, for the region in question. When there are multiple publishers in multiple regions, publishing messages nearly simultaneously, the ordering of messages delievred to subscribers can be different in different regions, depending on actual region-to-region transit latencies; in this case, deltas are generated based on the actual message order in each region, and subscribers are delivered a sequence of delta messages that reflects that regional order. + +Delta processing, when activated on a channel, is performed for all messages on a channel, and deltas are calculated strictly based on the message ordering in that channel. Effectiveness of delta processing - that is, whether or not there is a material saving in payload size - is dependent on the level of similarity between successive payloads. Therefore, if a channel carries messages from multiple sources or streams that are dissimilar, then delta processing might not result in a useful size reduction, even if the messages in each individual stream are similar; it depends on the specific sequence of messages that occurs in the channel, in the region in question. + +If a delta is generated and it results in a difference that is not appreciably smaller than the original message, or is even larger than the original message (which can happen if successive messages are completely different), then the delta will not be sent - clients will receive the original, unprocessed message. Therefore, in general, the sequence of messages that will be delivered to a client for any given channel will be a combination of regular messages and delta-compressed messages, with the delta messages only being present when they achieve a payload size reduction in comparison with the unmodified message. + +On some occasions a channel subscriber can experience a discontinuity in the sequence of messages it receives on any given channel. There are several possible reasons for this: the connection can drop, and there will be a discontinuity of the client is unable to reconnect within the two-minute window it is allowed to preserve connection continuity; the outbound connection might have been rate-limited, which causes some messages to be dropped; or there might have been some internal error in the Ably system which leads to the server being unable to preserve continuity on the channel. In these cases, the service indicates the discontinuity to the client, together with the reason, and this is usually visible to the subscriber in a channel @UPDATE@ event. If a subscriber has a delta-mode subscription and the channel in question experiences a discontunity, then a non-delta message will be delivered to the client as the first message after the discontinuity, so that lost messages do not prevent the client from reconstituting messages from deltas. + +h2(#using-deltas). Using deltas + +h3(#using-deltas-12). Via an Ably library from v1.2 (via the delta channel parameter) + +The most common way to subscribe to Ably channels is via a realtime connection, using an Ably realtime library. + +From version 1.2, Ably libraries support the ability to subscribe to a channel in delta mode. For many libraries this requires no change on the part of the caller except to specify the [delta channel param](./channel-params) when subscribing to the channel. In some libraries, the @vcdiff@ delta decoding library is excluded from the default library distribution in order to avoid bloating the library; in these cases, it is also necessary to supply the delta decoder plugin when instancing the Ably library. + + +```[jsall] + const vcdiffPlugin = require('{{vcdiff-plugin}}') + const realtime = new Ably.Realtime({key: '{{API_KEY}}', plugins: {vcdiffDecoder: vcdiffDecoder}}); + realtime.channels.get('{{RANDOM_CHANNEL_NAME}}', { + delta: 'vcdiff' + }).subscribe(msg => console.log("Received message: ", msg)); +``` + +```[java] + AblyRealtime ably = new AblyRealtime("{{API_KEY}}"") + Channel channel = ably.channels.get("{{RANDOM_CHANNEL_NAME}}", new ChannelOptions{{params = Map.of("delta", "vcdiff")}}); + channel.subscribe(new MessageListener() { + @Override + public void onMessage(Message message) { + System.out.println("Received `" + message.name + "` message with data: " + message.data); + } + }); +``` + +```[obj-c] +TBD +``` + +```[dotnet] +TBD +``` + +h3(#using-deltas-11). Via an Ably library before v1.2 (via a qualified channel name) + +```[jsall] + const realtime = new Ably.Realtime({key: '{{API_KEY}}'}); + const channel = realtime.channels.get('[?delta=vcdiff]{{RANDOM_CHANNEL_NAME}}'); + channel.subscribe(msg => console.log("Received message: ", msg)); +``` + +```[java] + AblyRealtime ably = new AblyRealtime("{{API_KEY}}"") + Channel channel = ably.channels.get("[?delta=vcdiff]{{RANDOM_CHANNEL_NAME}}"); + channel.subscribe(new MessageListener() { + @Override + public void onMessage(Message message) { + System.out.println("Received `" + message.name + "` message with data: " + message.data); + } + }); +``` + +```[obj-c] +TBD +``` + +```[dotnet] +TBD +``` + +h3(#using-deltas-non-ably). Via a subscription that does not use an Ably library + +If subscribing to a channel in delta mode using [SSE](https://www.ably.io/documentation/sse) or one of the protocol adaptors such as [MQTT](https://www.ably.io/documentation/mqtt), then you will need to decode any received delta messages yourself. There are decoder libraries available to do this for several platforms; see the [download](https://www.ably.io/download) section for details. If you need to decode @vcdiff@-formatted delta messages in languages for which there is no decoder available from Ably, then any compliant open-source implementation of the standard will work. + +Messages that contain a @vcdiff@ delta have that delta payload in their @data@ attribute; in the case of @vcdiff@ deltas this will be a binary value, even if the @data@ in the original meessage was text. in addition, there is metadata in the message @extras@ attribute that indicate that the payload is a delta, and which message the delta is relative to. The format of the `extras` attribute is as follows: + +```[jsall] + +extras: { + format: 'vcdiff', + from: '{{previous message id}}' +} +``` + +In order to reconstruct the original message, the @vcdiff@ decoder algorithm needs to be applied to the given @data@ @vcdiff@ value, together with the @data@ for the previous message (on the assumption that that message had already beein decoded if it itself was also a delta). If the original form of the present message was text, then this is indicated in the `encoding` attribute of the message, so the original text can be reconstructed by @utf-8@ decoding, once the @vcdiff@ decoding is complete. The decoder libraries that are available to [download]() simplify this process, and more detailed information can be found in the @README@ of each of those libraries. + +When subscribing without an Ably library, then the channel @delta@ param must be specified using a [qualified channel name](./channel-params). In the case of [SSE](https://www.ably.io/documentation/sse), it is also possible to specify channel params as regular query params on the connection URL. + +Some transports provide raw message payloads - that is, the content of the @data@ attribute of a @Message@ - without the accompanying metadata. That means that the recipient of the message does not have access to the @extras@ or @encoding@ attributes of the message that would ordinarily be used to decode delta message payloads. Examples of such transports are [MQTT](https://www.ably.io/documentation/mqtt), and [SSE](https://www.ably.io/documentation/sse) in non-enveloped mode. In order to assist applications that use these transports, the @vcdiff@ decoder libraries can check for the @vcdiff@ magic number at the start of the message payload as an inexact method of determining whether or not the message is a regular message or a delta. Note that, in order to rely on that check, you need to know that that magic number will not be present in any valid (uncompressed) message in your app. No valid JSON value, for example, will match the @vcdiff@ header check, so it is safe to perform this sniffing on JSON message payloads. + +h4(#delta-example-sse). Delta example with SSE + +You can subscribe to messages in delta mode, using the [SSE](https://www.ably.io/documentation/sse) transport, as follows. + + +``` +(() => { + const key = '{{API_KEY}}'; + const channel = 'sample-app-sse'; + const url = `https://realtime.ably.io/event-stream?channels=${channel}&v=1.1&key=${key}&delta=vcdiff`; + const eventSource = new EventSource(url); + const channelDecoder = new DeltaCodec.CheckedVcdiffDecoder(); + + eventSource.onmessage = (event) => { + /* event.data is JSON-encoded Ably Message (see https://www.ably.io/documentation/realtime/types#message) */ + const message = JSON.parse(event.data); + const { id, extras } = message; + let { data } = message; + + try { + if (extras && extras.delta) { + data = channelDecoder.applyBase64Delta(data, id, extras.delta.from).asUtf8String(); + } else { + channelDecoder.setBase(data, id); + } + } catch(e) { + /* Delta decoder error */ + console.log(e); + } + + /* Process decoded data */ + console.log(data); + }; +})(); +``` + +h4(#delta-example-unenv-sse). Delta example with unenveloped SSE + +```(() => { + const key = '{{API_KEY}}'; + const channel = 'sample-app-sse'; + const url = `https://realtime.ably.io/event-stream?channels=${channel}&v=1.1&key=${key}&delta=vcdiff&enveloped=false`; + const eventSource = new EventSource(url); + const channelDecoder = new DeltaCodec.VcdiffDecoder(); + + eventSource.onmessage = (event) => { + let data = event.data; + + try { + if (DeltaCodec.VcdiffDecoder.isBase64Delta(data)) { + data = channelDecoder.applyBase64Delta(data).asUtf8String(); + } else { + channelDecoder.setBase(data); + } + } catch(e) { + /* Delta decoder error */ + console.log(e); + } + + /* Process decoded data */ + console.log(data); + }; +})(); +``` + +h4(#delta-example-mqtt). Delta example with MQTT + +``` +const mqtt = require('mqtt'); +const { VcdiffDecoder } = require('./lib'); + +const options = { + keepalive: 30, + username: 'FIRST_HALF_OF_API_KEY', + password: 'SECOND_HALF_OF_API_KEY', + port: 8883 +}; +const client = mqtt.connect('mqtts:mqtt.ably.io', options); +const channelName = 'sample-app-mqtt'; +const channelDecoder = new VcdiffDecoder(); + +client.on('message', (_, payload) => { + let data = payload; + + try { + if (VcdiffDecoder.isDelta(data)) { + data = channelDecoder.applyDelta(data).asUint8Array(); + } else { + channelDecoder.setBase(data); + } + } catch(e) { + /* Delta decoder error */ + console.log(e); + } + + /* Process decoded data */ + console.log(data); +}); + +client.subscribe(`[?delta=vcdiff]${channelName}`); +``` diff --git a/content/realtime/rewind.textile b/content/realtime/rewind.textile new file mode 100644 index 0000000000..cd8c7221c2 --- /dev/null +++ b/content/realtime/rewind.textile @@ -0,0 +1,79 @@ +--- +title: Rewind +section: realtime +index: 32 +jump_to: + Help with: + - Overview#overview +--- + +h2(#overview). Overview + +Channels support a parameter that applies at the time of attachment that requests that an attachment start from some number of messages or point in time in the past. The @rewind@ param is specified via the "channel params":./channel-params mechanism. + +The @rewind@ channel parameter relates to the initial attachment of a connection to a channel, and expresses the intent to attach to the channel at a position, or a point in time, in the past (that is, effectively "rewinding" the channel for the purposes of the present attachment). + +A @rewind@ parameter can express a channel position in terms of a number of messages, or a time interval. + +A @rewind@ value that is simply a number @n@ (eg @rewind=1@) is a request to attach to the channel at a position @n@ messages before the present position. If that attachment is successful, and one or more messages exist on the channel prior to the present position, then those messages will be delivered to the subscriber immediately after the attachment has completed, and before any subsequent messages that arise in real time. + +If fewer than the requested number of messages exists on the channel (including the case that there are no prior messages), then the available messages are sent; this does not constitute an error. + +A @rewind@ value can also be a string that is a time interval specifier. Supported specifier values express an integral number of seconds (eg @15s@) or minutes (eg @2m@). If that attachment is successful, and one or more messages exist on the channel in the given time interval prior to the present time, then those messages will be delivered to the subscriber immediately after the attachment has completed, and before any subsequent messages that arise in real time. + +If you wish to use a time interval rewind but additionally specify a limit on the number of messages to be returned, you can use the @rewindLimit@ channel param. For example, to request up to 10 messages in a window 5m before the present time, specify a channel parameter string of @rewind=5m&rewindLimit=10@. If fewer than the requested number of messages exists on the channel in that interval (including the case that there are no messages), then the available messages are sent; this does not constitute an error. + +At most 100 messages will be sent in a rewind request. If the number of messages within the specified interval is greater than that limit, then only the most recent messages up to that limit are sent. The attachment succeeds, but truncation of the message backlog is indicated as a non-fatal error in the attachment response. + +By default, a maximum of two minutes of channel history is available when attaching. This means that a rewind time specifier of greater than two minutes will only be able to rewind by two minutes. If a channel has persistence enabled, then it is possible to rewind back in time by up to the persistence TTL on the channel. + +The channel position expressed by a @rewind@ parameter has an effect only on an initial channel attachment. Any subsequent reattachment of the same channel on the same connection, in order to resume the connection, will attempt to resume with continuity from the point at which the connection dropped. (There are a few exceptions to this: in particular, client libraries earlier than v1.2 that have been disconnected for over two minutes, and all clients when using "@recover@ mode":/realtime/connection#connection-state-recovery ; in both cases the previous attachment state is not preserved). + +Any @rewind@ parameter value that cannot be parsed either as a number or a time specifier represents an error, and any attachment request will fail with an error. + +h3(#rewind-example-ably). Rewind example with an Ably client library + +To subscribe to a channel, getting the most recent message if available: + +```[jsall] + // only with ably-js v1.2 or later + const realtime = new Ably.Realtime('{{API_KEY}}'); + realtime.channels.get('{{RANDOM_CHANNEL_NAME}}', { + rewind: '1' + }).subscribe(msg => console.log("Received message: ", msg)); +``` + +```[jsall] + // with ably-js v1.1 or below + const realtime = new Ably.Realtime('{{API_KEY}}'); + const channel = realtime.channels.get('[?rewind=1]{{RANDOM_CHANNEL_NAME}}'); + channel.subscribe(msg => console.log("Received message: ", msg)); +``` + +h3(#rewind-example-sse). Rewind example with SSE + +To subscribe to a channel, getting the most recent message if available: + +```[javascript] + var querystring = 'v=1.2&channels={{RANDOM_CHANNEL_NAME}}&rewind=1&key={{API_KEY}}'; + var eventSource = new EventSource('https://realtime.ably.io/event-stream?' + querystring); +``` + +h3(#rewind-examples-mqtt). Rewind example with MQTT + +```[nodejs] + var mqtt = require('mqtt'); + var options = { + keepalive: 30, + username: 'FIRST_HALF_OF_API_KEY', + password: 'SECOND_HALF_OF_API_KEY', + port: 8883 + }; + var client = mqtt.connect('mqtts:mqtt.ably.io', options); + client.on('connect', () => { + client.subscribe('[?rewind=1]{{RANDOM_CHANNEL_NAME}}'); + }); + client.on('message', (topic, message) => { + ... + }); +```