Note

The code in this repository has been made public as-is for informational purposes. The repository may use private resources for the building and execution of the code. For example, private registries may be used for dependency resolution.

The documentation may refer to restricted URLs.

---

Quick Guide

Simple queue client which provides a generic interface for interacting with multiple queuing systems, currently supports:

• Depot
• RabbitMQ
• A simple in memory queue

In Memory Queues

implementation based on pythons double ended queue

Sample usage:

fromqueueclientimportInMemoryQueueClientq=InMemoryQueueClient(queue_id='uuid')
# msg can be any python objectq.enqueue(msg="message")
# from workermsg=q.dequeue()
assertmsg=="message"

Depot

Sample usage:

fromqueueclientimportDepotQueueClientq=DepotQueueClient(host="depot.service.consul", queue_id="uuid")
assertq.status() isTrue# msg can be any json objectresponse=q.enqueue(msg=dict(did="AAAAA", size=123))

fromqueueclientimportDepotQueueClientq=DepotQueueClient(host="depot.service.consul", queue_id="uuid")
# retrieve previous messagemsg=q.dequeue()
assertmsgassertmsg["did"] =="AAAAA"assertmsg["size"] ==123

RabbitMQ

Sample usage:

importjsonfromqueueclientimportRabbitMQClient# requires a custom call back function, and assumes body is a json objectdefconsumer_callback(body):
msg=json.loads(body)
assertmsgassertmsg["did"] =="AAAAA"assertmsg["size"] ==123q=RabbitMQClient(host="localhost", port=5672, vhost="/", queue_id="qid", username="guest", password="guest")
response=q.enqueue(msg=dict(did="AAAAA", size=123))
assertresponseisTrue# this line blocks, call q.close() to cancelq.consume(consumer_callback)

Setup pre-commit hook to check for secrets

We use pre-commit to setup pre-commit hooks for this repo. We use detect-secrets to search for secrets being committed into the repo.

To install the pre-commit hook, run

pre-commit install

To update the .secrets.baseline file run

detect-secrets scan --update .secrets.baseline

.secrets.baseline contains all the string that were caught by detect-secrets but are not stored in plain text. Audit the baseline to view the secrets .

detect-secrets audit .secrets.baseline

Running Test

Tests uses pifpaf to spin up daemon rabbitmq-server locally which requires RabbitMQ installed, to install on ubuntu

$ sudo apt install rabbitmq-server

run tests via tox

$ tox

or manually

$ python -m pip install .
$ python -m pip install -r dev-requirements.txt
$ pifpaf run rabbitmq -- python -m pytest