Skip to main content

SphereQueue (RabbitMQ)

SphereQueue is a managed RabbitMQ message broker in your environment. Any AMQP 0-9-1 client works, for example Celery, pika, amqplib, Spring AMQP or MassTransit.

Use it for background jobs, work queues, and passing events between services.

Regions: Sweden and US East.

Sizes and prices​

You pay a fixed monthly price for each broker. Each size includes a disk for messages waiting in queues. High availability runs three copies and costs three times the price.

SizevCPUMemoryDiskPer monthWith high availability
Queue 112 GB20 GB$49$147
Queue 224 GB40 GB$95$285
Queue Plus 228 GB60 GB$179$537
Queue 4416 GB80 GB$279$837
Queue 8832 GB120 GB$499$1,497
Queue 161664 GB200 GB$999$2,997
Queue Freeshared——Coming soon—
Queue Sharedshared——Coming soon—
  • Queue Plus 2 has extra memory for large backlogs.
  • Queue 1 and Queue 2 can use their full vCPU when it's free, and always have at least half of it reserved.
  • Regions: prices are for most regions. Southeast Asia, South Africa North and South India cost 25% more. The pricing page has every price.
  • Coming soon: Queue Free and Queue Shared. You can't create them yet.

Create a broker​

Console: in a project, choose New, then SphereQueue ("Message queue"). Pick a name and a size.

A broker of your own (Queue 1 and up) runs RabbitMQ 4.3, with one user (app) and one virtual host (/).

Connect from a service​

A broker has a private address that only services in the same environment can reach. There's no public endpoint, and services in other environments can't reach it.

Every connection uses TLS (AMQPS, port 5671). Copy the connection string from the broker's page in the console, and add it to your service as a secret, for example AMQP_URL or CELERY_BROKER_URL:

amqps://app:<password>@<host>:5671/

The password is hidden until you choose Reveal, which needs write access to the project. Every reveal is recorded in the audit log.

No management interface. SphereQueue doesn't expose the RabbitMQ management UI or API. Apps connect over AMQP only. If you use Celery, Flower's Broker tab (queue lengths) stays empty: it needs the management API.

Change the password: choose Rotate password. The old password keeps working until the broker has taken the new one; then reconnect your apps with the new password.

Using Celery

Celery works with SphereQueue once a few settings are in place. Our brokers don't allow transient non-exclusive queues, a RabbitMQ feature deprecated since 2021. Celery's default rpc:// result backend and its remote control use that feature. Tested with Celery 5.5.

app.conf.task_default_queue_type = "quorum"
app.conf.result_backend = os.environ["CELERY_RESULT_BACKEND"] # a SphereDB or SphereCache URL, not rpc://
app.conf.worker_enable_remote_control = False
app.conf.worker_send_task_events = False # and don't start workers with -E
app.conf.task_send_sent_event = False

For a SphereCache result backend, also point Celery at your environment's CA file: app.conf.redis_backend_use_ssl = {"ssl_cert_reqs": ssl.CERT_REQUIRED, "ssl_ca_certs": "/etc/computesphere/db/<key>.crt"}.

Start workers with:

celery -A your_app worker --without-gossip --without-mingle --without-heartbeat

Flower needs events and remote control, so it can't show your workers. Celery 5.5's delayed delivery (countdown/ETA) creates 28 queues on the broker; Queue 1 allows 500.

Verify TLS with your environment's CA​

Your app checks the connection against your environment's own certificate authority. Every service in the environment has its CA file at /etc/computesphere/db/<key>.crt, where <key> is the broker's host without db-: for the host db-3f9c…, the file is /etc/computesphere/db/3f9c….crt. Point your client's TLS CA setting at that file, and use the host as the TLS server name.

The CA file arrives about 80 seconds after the broker is ready. Most apps read the CA when they start, so restart (redeploy) any service that started before then. The examples below read the file when they connect.

Node.js (amqplib):

Using Node.js amqplib? Version 0.10.5 and older ask for a smaller frame size than SphereQueue accepts, so SphereQueue closes the connection right after login. Upgrade to amqplib 0.10.6 or later, or add ?frameMax=131072 to the connection string.

import fs from "node:fs";
import amqp from "amqplib";

const url = new URL(process.env.AMQP_URL);
url.searchParams.set("frameMax", "131072");
const ca = fs.readFileSync(`/etc/computesphere/db/${url.hostname.replace(/^db-/, "")}.crt`);

const conn = await amqp.connect(url.toString(), { ca: [ca], servername: url.hostname });

Python (pika):

import os, ssl
from urllib.parse import urlparse
import pika

url = os.environ["AMQP_URL"]
host = urlparse(url).hostname
ctx = ssl.create_default_context(cafile=f"/etc/computesphere/db/{host.removeprefix('db-')}.crt")

params = pika.URLParameters(url)
params.ssl_options = pika.SSLOptions(ctx, server_hostname=host)
conn = pika.BlockingConnection(params)

Go (amqp091-go):

import (
"crypto/tls"
"crypto/x509"
"net/url"
"os"
"strings"

amqp "github.com/rabbitmq/amqp091-go"
)

func connect() (*amqp.Connection, error) {
raw := os.Getenv("AMQP_URL")
u, err := url.Parse(raw)
if err != nil {
return nil, err
}
pem, err := os.ReadFile("/etc/computesphere/db/" + strings.TrimPrefix(u.Hostname(), "db-") + ".crt")
if err != nil {
return nil, err
}
pool := x509.NewCertPool()
pool.AppendCertsFromPEM(pem)
return amqp.DialTLS(raw, &tls.Config{RootCAs: pool, ServerName: u.Hostname()})
}

High availability​

A broker with high availability runs three copies in three availability zones. Queues are quorum queues by default: a queue you declare without x-queue-type is kept on all three copies. If one copy fails, the other two keep serving, and messages the broker has confirmed to your app aren't lost.

You can still ask for a classic queue or a stream with x-queue-type. Exclusive queues are always classic (a RabbitMQ rule), so they live on one copy only.

Limits​

LimitValue
Connections250 per GB of memory (Queue 1: 500)
Queues250 per GB of memory (Queue 1: 500)
Size of one queueA quarter of the disk (Queue 1: 5 GB). When a queue is full, new messages to it are refused, so the disk never fills.
Changing the sizeNot yet available. To grow, create a larger broker and move your apps to it.

Backups​

Every broker's setup is backed up once a day: its queues, exchanges, bindings and policies. Users and passwords aren't in the backup: a broker restored from one has its own password. The last 7 daily copies are kept.

Messages aren't backed up. Messages waiting in queues are on the broker's disk, and with high availability on three copies. Don't treat a queue as long-term storage.

To restore, contact support. Name the broker and the day you want back. Support restores that setup into a new broker. You can't yet list or restore backups yourself from the console.