Broadcaster

Broadcaster helps you develop realtime streaming functionality by providing a simple broadcast API onto a number of different backend services.

It currently supports Redis PUB/SUB, Redis Streams, Apache Kafka, and Postgres LISTEN/NOTIFY, plus a simple in-memory backend, that you can use for local development or during testing.

Here's a complete example of the backend code for a simple websocket chat app:

app.py

# Requires: `starlette`, `uvicorn`, `jinja2`# Run with `uvicorn example:app`importanyiofrombroadcasterimportBroadcastfromstarlette.applicationsimportStarlettefromstarlette.routingimportRoute, WebSocketRoutefromstarlette.templatingimportJinja2Templatesbroadcast=Broadcast("redis://localhost:6379")
templates=Jinja2Templates("templates")
asyncdefhomepage(request):
template="index.html"context= {"request": request}
returntemplates.TemplateResponse(template, context)
asyncdefchatroom_ws(websocket):
awaitwebsocket.accept()
asyncwithanyio.create_task_group() astask_group:
# run until first is completeasyncdefrun_chatroom_ws_receiver() ->None:
awaitchatroom_ws_receiver(websocket=websocket)
task_group.cancel_scope.cancel()
task_group.start_soon(run_chatroom_ws_receiver)
awaitchatroom_ws_sender(websocket)
asyncdefchatroom_ws_receiver(websocket):
asyncformessageinwebsocket.iter_text():
awaitbroadcast.publish(channel="chatroom", message=message)
asyncdefchatroom_ws_sender(websocket):
asyncwithbroadcast.subscribe(channel="chatroom") assubscriber:
asyncforeventinsubscriber:
awaitwebsocket.send_text(event.message)
routes= [
Route("/", homepage),
WebSocketRoute("/", chatroom_ws, name='chatroom_ws'),
]
app=Starlette(
routes=routes, on_startup=[broadcast.connect], on_shutdown=[broadcast.disconnect],
)

The HTML template for the front end is available here, and is adapted from Pieter Noordhuis's PUB/SUB demo.

Requirements

Python 3.8+

Installation

• pip install broadcaster
• pip install broadcaster[redis]
• pip install broadcaster[postgres]
• pip install broadcaster[kafka]

Available backends

• Broadcast('memory://')
• Broadcast("redis://localhost:6379")
• Broadcast("redis-stream://localhost:6379")
• Broadcast("postgres://localhost:5432/broadcaster")
• Broadcast("kafka://localhost:9092")

Using custom backends

You can create your own backend and use it with broadcaster. To do that you need to create a class which extends from BroadcastBackend and pass it to the broadcaster via backend argument.

frombroadcasterimportBroadcaster, BroadcastBackendclassMyBackend(BroadcastBackend):
broadcaster=Broadcaster(backend=MyBackend())

Where next?

At the moment broadcaster is in Alpha, and should be considered a working design document.

The API should be considered subject to change. If you do want to use Broadcaster in its current state, make sure to strictly pin your requirements to broadcaster==0.2.0.

To be more capable we'd really want to add some additional backends, provide API support for reading recent event history from persistent stores, and provide a serialization/deserialization API...

• Serialization / deserialization to support broadcasting structured data.
• A backend for RabbitMQ.
• Add support for subscribe('chatroom', history=100) for backends which provide persistence. (Redis Streams, Apache Kafka) This will allow applications to subscribe to channel updates, while also being given an initial window onto the most recent events. We might also want to support some basic paging operations, to allow applications to scan back in the event history.
• Support for pattern subscribes in backends that support it.

Third Party Packages

MQTT backend

Gist

Integrates MQTT with Broadcaster