SCOP d'ingénieurs experts du logiciel libre depuis 2004
+33 1 70 61 60 16

High available MQTT service

[MQTT]( is a pubsub protocol targetting small devices and loosy network, with a centralized server. Mosquitto is a clean, simple and stable MQTT server

Notre prochain webinar

MQTT is a pubsub protocol targetting small devices and loosy network, with a centralized server. Mosquitto is a clean, simple and stable MQTT server. For now high availability is not part of the MQTT protocol and Mosquitto doesn’t handle it. But all needed parts for building a resilient high availability service are available.

I want to build a ping system : agents ping servers in order to monitor them. Agents use different internet provider or data center, ping a dynamic list of servers. Agents are usually behind a NAT, and should remain polite.

Let’s bring the classic pubsub pattern. Nodes publish ping statistics on a channel : ping/{domain}, publish their will on rip/agent/{hostname} and subscribe to watch which provides the list for targets. Indexer subscribes to ping/+ and rip/# (+ means one slug, # a lot of slugs).

The high availabity model is naive. Pingers have a list of mqtt servers, shuffle it, try to connect to one of them. If nothing happens on the socket, it uses the next server, in a round robin fashion. Pingers ping ONE server, randomly chosen. Indexers connect to ALL the servers, and send messages to all the servers. Indexers say a few things but listen a lot. Data are pushed to ElasticSearch, à la Logstash, for Kibana drilling.

Supporting high availability server side, with a loosy connection can be painful. I don’t care if message are duplicated or even loss (it’s just ping, not financials transactions). Mosquitto is a nice MQTT server, though it’s not hackable enough to implement replication, and it’s a bad idea. Mosquitto handles low level contracts (passing message, handling mailboxes and dead peoples), not business logic. I can use RabbitMQ as a mqtt server , using its clustering features, but no, I want something dead simple.

With this pattern, I can use an unconfigured mosquitto server (apt-get install mosquitto-server) and vanilla clients. I can stop a server for a short period of time, deploy a new version, and restart it, without breaking the service.

Later, I will add authentication (with SSL certificates), client side trouble detection (it’s just me, or the target?), thresold levels (it’s slow or broken?), speed measurements.

Demo time

A simple demo, full localhost, on a debian linux. It works with any Linux or os X, just translate the instructions.

Get source of the swarming project :

 git clone cd swarming 

Virtualenv dance :

 virtualenv . ./bin/pip install -r requirements.txt 

Install and launch mosquitto :

 sudo apt-get install mosquitto-server sudo /etc/init.d/mosquitto start 

Launch another mosquitto instance, on another port :

 mosquitto -p 1884 

Launch two snitchs in two terminals :

 mosquitto_sub -t "ping/+" -t "rip/#" -v mosquitto_sub -t "ping/+" -t "rip/#" -v -p 1884 

Launch one agent in another terminal :

 ./bin/python localhost localhost:1884 

Install elasticsearch :

 wget sudo dpkg -i elasticsearch-0.90.10.deb sudo /etc/init.d/elasticsearch start 

Launch the indexer :

 ./bin/python localhost localhost:1884 

You can now stop, wait, restart one or other mosquitto server, add swarm agent.

Install Kibana. I’m cheating, I’m using the ugly one liner python http server.

 wget tar -xvzf kibana-3.0.0milestone4.tar.gz cd kibana-3.0.0milestone4 python -m SimpleHTTPServer 

Open the default web page : http://localhost:8000/index.html#/dashboard/file/logstash.json


Lets explore kibana. Compare target performance, compare agent performance, find something useful and meaningful from all this points.

Mathieu Lecarme

Inscrivez-vous à notre newsletter

Mieux comprendre le monde du DevOps et de l'administration système.

Abonnez-vous à notre newsletter

Hébergement & Infogérance

  • ✓ Service Astreinte 24h/7j/365
  • ✓ Supervision, monitoring & Alertes
  • ✓ Mises à jour en continu
  • ✓ Certificat SSL letsencrypt
  • ✓ Hébergement dédié sécurisé en France
  • ✓ Backup vers datacenter distant
Découvrir notre offre

Expertise Technologique

Notre équipe possède une vaste expertise technologique.