What is MQTT over WebSockets and how is it implemented?

Many of today’s Internet-of-Things (IoT) devices require a web front-end. These could be real-time dashboards for monitoring sensor readings or web-based user interfaces to control devices or machines. IoT devices differ from the web in that they rely on low-bandwidth, low-speed data communication protocols to communicate data with one another over the Internet. Since tiny microcontrollers power most IoT and embedded devices, they can’t handle large packets or high-speed data. Therefore, IoT devices depend on internet protocols like MQTT to communicate efficiently and without much overhead.

Browsers lack support for popular IoT protocols like MQTT, relying on HTTP protocol for all internet data communication. So, how do web front-ends for IoT devices interact with these devices? The answer is MQTT over WebSockets. Browsers understand the WebSocket protocol, allowing IoT devices to communicate with web clients by tunneling MQTT messages through WebSockets.

This article explores MQTT over WebSockets, why it’s required, and how it’s implemented on IoT devices and browsers to design web interfaces for IoT solutions.

What is MQTT?

Message Queuing Telemetry Transport (MQTT) is a lightweight messaging protocol designed for the IoT. MQTT facilitates communication between devices with limited resources using a publish-subscribe model. Devices can send (publish) and receive (subscribe) data as MQTT messages. Instead of directly communicating between devices, data is sent from the publisher to the subscriber via an MQTT broker.

This protocol is efficient for low-bandwidth networks and supports security features like Transport Layer Security (TLS) and Secure Sockets Layer (SSL) for encryption. MQTT is often used for collecting sensor data, remote controlling devices, home, and industrial automation.

MQTT is an application-layer protocol that uses the same underlying network infrastructure (e.g., physical cables, routers, etc.) as an HTTP protocol. At the transport layer, MQTT relies on TCP or UDP connections to carry messages, while at the network layer, these messages are routed using IP and ICMP protocols.

What are WebSockets?

WebSockets are a network layer protocol that establishes a two-way communication channel between a web client and a web server. Unlike the HTTP protocol, which follows a request-response model, WebSockets enable real-time, bidirectional data communication, allowing both the client and the server to send data simultaneously without polling or refreshing webpages.

The process begins with a web client sending an HTTP request to the server, indicating an intention to upgrade to a WebSockets connection. If the server supports WebSockets, it responds with an upgrade header to establish the connection. Once connected, the server and the client can exchange data in text format using the WebSocket protocol. The WebSocket connection remains open, allowing continuous real-time data exchange until the client or server closes the connection.

The protocol is commonly used in applications like real-time chats, streaming audio and video, real-time multiplayer games, and live collaborative tools where instantaneous data exchange is essential.

What is MQTT over WebSockets?

MQTT and HTTP are both application layer protocols, but they operate differently. HTTP operates over high-bandwidth networks, enabling real-time data communication between servers and clients. On the other hand, MQTT operates over low-bandwidth networks, facilitating the exchange of short messages between publishers and subscribers through a broker or server.

While microcontrollers have limited capabilities and struggle with handling the HTTP protocol and its data, browsers and similar web clients lack the built-in support for MQTT. However, both protocols operate over a TCP connection, presenting an opportunity to bridge their distinct functionalities.

MQTT over WebSockets, a feature implemented and managed by the MQTT broker or server, allows for the transmission of MQTT messages between IoT devices and web browsers or similar web clients. Built-in support for MQTT over WebSockets is now available for almost all popular MQTT brokers, including HiveMQ, Eclipse Mosquitto, EMQX, RabbitMQ, and CloudMQTT.

Why choose MQTT over WebSockets?

Plain MQTT offers optimal performance for device-to-device communication with the IoT. MQTT over WebSockets becomes relevant when IoT devices need to communicate with a browser or similar web client. This setup facilitates the creation of real-time dashboards and user interfaces for direct interaction with IoT devices. At the same time, its universal access allows end-users to manage devices without technical expertise.

Web applications on browsers have capabilities that extend beyond microcontrollers, such as visualizing data, deriving insights, performing data analytics, or using the data for machine learning. Additionally, MQTT over WebSockets can bypass firewall restrictions. While many firewalls permit only HTTP traffic (through ports 80 and 443), MQTT brokers can circumvent these restrictions and freely transfer MQTT data anywhere and at anytime.

The applications

MQTT over WebSockets is essential for creating real-time dashboards and user interfaces and displaying live sensor data from IoT devices. These dashboards allow web applications in browsers to subscribe to specific topics and showcase real-time updates from devices, enabling end-users to control and interact with devices in real time. This allows users to control machines through a web app or browser and get immediate updates for items such as predictive maintenance.

Single-page applications (SPAs) that rely on or interact with IoT data can also benefit from MQTT over WebSockets. SPAs can effectively communicate and exchange data with IoT devices using WebSockets. However, this network must be designed to minimize IoT devices’ reliance on browser interactions, with critical device-to-device communication tasks managed exclusively by MQTT.

How MQTT over WebSockets work

IoT devices and browsers function as web clients in data communication for MQTT over WebSockets. The browser establishes a WebSocket connection with the MQTT broker, which supports native WebSockets, maintaining an open bidirectional channel for real-time communication. Subsequently, the browser operates as a typical MQTT client, subscribing to topics to receive and publish MQTT messages to the broker.

Behind the scenes, when IoT devices publish data intended for the browser as a subscriber, the MQTT messages are encapsulated into WebSocket packets. These encapsulated messages are then transmitted to the browser over a single TCP connection which unpacks WebSocket packets and processes MQTT messages as standard MQTT messages.

The requirements

Certain conditions are required when transporting MQTT data over WebSockets, as specified in the OASIS standard for MQTT 3.1.1. WebSocket binary data frames are necessary to transmit MQTT Control Packets. If any other type of data frame is received, the recipient must terminate the network connection.

Multiple or partial MQTT Control Packets can be contained within a single WebSocket data frame. However, the receiver should not assume that MQTT Control Packets align with WebSocket frame boundaries. Also, the WebSocket Sub Protocols the client offers must include the text “mqtt,” the server must select and return “mqtt” as the WebSocket Sub Protocol name.

The MQTT protocol should remain unaffected by the WebSocket URI used to establish the client-server connection.

Preparing the MQTT broker

To properly establish data communication between IoT devices and the browser, here’s what’s required:

An MQTT broker with a native WebSocket support
An MQTT over WebSockets client-side library for the web front-end

An MQTT broker can be set up by deploying a private MQTT broker on your server, subscribing to a fully managed cloud service that supports MQTT over WebSockets, or using a free public MQTT broker. While a free public MQTT broker can be used for testing purposes, it’s best to avoid using them in production due to security issues and the potential for significant downtime. Cloud-based services offer the most affordable option, but a private deployment is optimal for obtaining full control.

Setting up MQTT WebSocket client

On the front end, a client-side library is needed to enable MQTT over WebSockets in the browser. Since many real-time dashboards and user interfaces are single-page applications built with Node.js, JavaScript is often the front-end language of choice. The Paho JavaScript MQTT Client is a popular library for implementing MQTT in JavaScript clients. It includes built-in support for MQTT over WebSockets, allowing direct connection to a broker using the “wss://” protocol scheme. Other libraries like MQTT.js and Tay MQTT are also available for MQTT implementation in JavaScript.

For PHP web applications, the combination of Paho MQTT for PHP with ReactPHP Ratchet serves a similar purpose. For Ruby on Rails web apps, MQTT for Rails can be used. This library uses the emqttd-client library and provides functionalities for connecting to MQTT brokers, subscribing to topics, and publishing messages. This library employs puma_websocket to establish the WebSocket connection on the server side.

For high-performance Java web applications, the Eclipse Paho Java Client is a dependable solution for implementing various MQTT features, including connection management, publishing/subscribing to topics, and message delivery options. Python web applications can use the websockets-client library for WebSocket functionality and the Paho MQTT library for MQTT operations.

Using WebSocket over SSL/TLS

Transport Layer Security (TLS) can be used to secure WebSockets with encryption, providing an additional layer of security. Most MQTT brokers offer built-in support for security features like TLS and SSL. On the front-end side, these security measures can be implemented using methods provided by the client-side library, ensuring secure communication between the browser and the MQTT broker.

MQTT over WebSocket – with Arduino

MQTT over WebSockets can also be implemented using Arduino. While Arduino lacks the required built-in support or native libraries, the MQTTPubSubClient library, in conjunction with the WebSocketStreamClient library, can transport MQTT messages over WebSockets. Furthermore, the MQTTPubSubClient library from hideakitai is a single library that offers both MQTT and MQTT over WebSockets.