MQTT Simplified: A Beginner's Guide to Seamless Communication

Vaibhav Sharma

Jul 18, 2023 • 12 min read

The role of communication protocols in IoT is crucial for enabling effective and reliable data exchange among interconnected devices. These protocols act as the foundation for seamless communication, ensuring compatibility, security, and scalability within the IoT ecosystem. While there are several communication protocols available, MQTT (Message Queuing Telemetry Transport) has emerged as the frontrunner for IoT communication. According to an IoT developer survey conducted in 2022 by the Eclipse Foundation, MQTT continues to be the most widely used IIoT communication protocol.

The widespread adoption of MQTT in IoT systems encouraged us to compile a beginner’s guide on MQTT. In this article, we aim to provide you with an understanding of the MQTT protocol, its features and future advancements.

Understanding MQTT and its significance in the world of IoT

Message Queuing Telemetry Transport aka MQTT is a lightweight messaging protocol specifically designed for efficient communication in the Internet of Things (IoT) ecosystem.

MQTT was developed in the late 1990s at IBM's headquarters by Dr. Andy Stanford-Clark and Arlen Nipper to address the shortcomings of existing protocols. It was initially designed for supervisory control and data acquisition (SCADA) systems, aiming to collect data from remote locations and communicate it to central servers and later it became a popular name in the IoT sector.

How MQTT became the frontrunner for IoT communication? Firstly, MQTT is known for its lightweight nature, making it highly suitable for resource-constrained IoT devices. Because of this many GSM Modems come built-in with MQTT functionality. This makes getting started with MQTT very easy, particularly on microcontroller devices.

Secondly, MQTT follows a publish-subscribe messaging model. This model allows multiple devices to publish messages on specific topics and other devices to subscribe to those topics of interest. As a result, it optimizes network bandwidth, enhances scalability, and supports one-to-many and many-to-many communication patterns.

Furthermore, MQTT offers various Quality of Service (QoS) levels, ensuring reliable message delivery in different scenarios. These features play a crucial role in facilitating seamless data exchange among interconnected devices in the IoT ecosystem. We will be discussing these features in detail in a later section.

Understanding MQTT Architecture

At the core of MQTT's architecture lies the concept of a publish-subscribe model. In this model, there are two main components: MQTT clients and the MQTT broker. The clients are the devices or applications that produce and consume data. They can be sensors, actuators, smartphones, or any IoT device. The broker acts as an intermediary, responsible for receiving messages from publishers and delivering them to subscribed clients. Let's have a look at the components of MQTT architecture in detail.

In MQTT, the Publish-Subscribe model is a messaging pattern where publishers send messages to a broker by publishing them to a topic, and subscribers receive messages by subscribing to specific topics or matching filters. This decoupled communication allows for efficient and scalable data exchange in MQTT-based systems, such as IoT applications.

In the context of home automation, imagine you have various smart devices in your home, such as thermostats, lights, and sensors. Each of these devices can be a publisher, sending data or status updates to specific topics on the MQTT broker. For instance, a temperature sensor can publish temperature readings to the "home/temperature" topic.

On the other hand, you can have subscribers that are interested in receiving data from specific topics. For example, a mobile app or a control panel can subscribe to the "home/temperature" topic to display the current temperature.

MQTT Client

In MQTT, a client is a device or application that connects to an MQTT broker to publish messages or subscribe to topics. MQTT clients can have two roles

Publisher: MQTT Client that publishes messages on the topic
Subscriber: MQTT client that subscribes to the topic to receive messages

For example, a home automation system can have a client, such as a temperature sensor, acting as a publisher, and a lighting control system as a subscriber, receiving temperature updates from the sensor via the broker.

A single MQTT client can have multiple roles. That is a client can be a publisher or
subscriber simultaneously. For example, A smartphone application for a coffee maker machine can act as a publisher to send on/off commands to the coffee machine at the same time it can be a subscriber to get the current state coffee machine.

MQTT Broker

An MQTT broker is a central hub in the MQTT protocol that receives messages from publishers and distributes them to subscribed subscribers. It acts as a message broker, facilitating efficient and reliable communication between devices. Some of the well-known MQTT brokers are

Rumqttd : Rumqttd is an open-source MQTT broker written in Rust. It’s lightweight robust and performant. It is embeddable, meaning you can use it as a library in your code and extend functionality.
Mosquitto : Mosquitto is an open-source MQTT broker. Mosquitto is lightweight and suitable for low-power IoT devices and servers.
HiveMQ : HiveMQ is a scalable and feature-rich MQTT broker designed for enterprise-level IoT deployments, providing reliable and secure messaging communication between devices.
EMQ X: EMQ X is a high-performance and scalable MQTT broker that supports massive connections and advanced features, making it ideal for large-scale IoT deployments.

Topics

In MQTT, a topic is a hierarchical identifier that acts as a label for messages. Publisher clients use topics to publish their messages, while subscriber clients subscribe to specific topics to receive relevant messages. For example, "home/temperature/living-room" can be used to publish temperature readings from the living room. Whereas subscribers can subscribe to this topic to receive temperature values.

MQTT Topics key considerations

MQTT topic should be a UTF-8 string with at least one character.
MQTT topic is case sensitive which means “home/light/living-room” and “home/Light/Living-room” are different topics.
MQTT topics can contain wildcard characters like “#”, “+”.

MQTT Subscribe topics with wildcard characters

Firstly, wildcard characters can only be used for subscriptions, not while publishing. There are two types of wildcard characters depending on the hierarchy

# (hash sign): To subscribe to all levels of the hierarchy
- For Example: using “living-room/#” topic we can subscribe to multiple topics at once like “living-room/sensor/temperature”, “living-room/sensor/light” but can’t subscribe to “kitchen/sensor/chimney”

living-room/#
# matches following topics
living-room/sensor/light
living-room/sensor/chimney
#but does not match 
kitchen/sensor/chimney

+ (plus sign): To subscribe to single levels of the hierarchy
- For Example: “+/sensor/temperature” can subscribe to multiple topics at once like living-room/sensor/temperature, kitchen/sensor/temperature but can’t subscribe to kitchen/status/chimney.

+/sensor/temperature
#matches 
living-room/sensor/temperature
kitchen/sensor/temperature
#does not match
kitchen/status/chimney

Quality of Service (QoS)

Quality of Service (QoS) levels in MQTT determine the reliability and guarantee of message delivery between publishers and subscribers. MQTT offers three levels of QoS:

QoS 0 (At most once)

Messages are delivered with the best effort, without guarantees.

Practical use case of QoS 0: imagine a smart home system with various motion sensors installed throughout the house. These sensors constantly detect motion and send updates to the central control unit. By employing QoS 0 in MQTT, the motion sensor data can be transmitted without waiting for acknowledgements, ensuring real-time responsiveness. Occasional loss of a motion detection update, wouldn't significantly affect the overall functionality of the home automation system.

QoS 1 (At least once)

Messages are guaranteed to be delivered at least once to the subscriber. If a message is lost or fails to reach its destination, it will be resent until it is received. When a publisher sends a message, it stores the message in its memory until it receives an acknowledgement (PUBACK) from the broker. If the broker doesn't receive the acknowledgement within a specified time frame, it resends the message.

Practical use case of QoS 1:For example, let's consider a scenario where an individual wants to remotely control their smart lights using a smartphone app. In this case, utilizing QoS Level 1 ensures at least once delivery, guaranteeing that the command to turn on or off the lights is received by the smart lighting system. Even if there is a momentary loss of connectivity or a network hiccup, the QoS 1 level would ensure that the command is retransmitted until successful delivery is achieved.

QoS 2 (Exactly once)

Messages are ensured to be delivered exactly once to the subscriber. This level provides the highest reliability but incurs additional overhead. In the case of QoS2, When the client publishes a message, the broker acknowledges with PUBREC (publish record) after the publisher client gets PUBREC, It sends PUBREL (publish release) to the broker. Once the broker gets PUBREL, it sends the final acknowledgement PUBCOMP(publish complete) to the client. If any part of the handshake fails, the broker or the device resends the necessary messages to ensure exactly-once delivery.

Practical use case of QoS 2: For Example, an individual is monitoring their home security system through a live video feed on their tablet. In this situation, opting for QoS Level 2 would be ideal. QoS 2 offers exactly once delivery, ensuring that each video frame is received accurately, without any duplication or loss. This level of reliability is crucial for maintaining the integrity of video surveillance in a home automation setup.

Retained Messages in MQTT

Retained messages allow publishers to store the last message published on a specific topic in the MQTT broker. This feature ensures that new subscribers instantly receive the last known value upon joining.

Setting Retained Flag

When a publisher sends a message with the retained flag set to true, the broker stores that message as the last known state for that topic.

Topic: home/living-room/temperature
Payload: 25.5
Retain: true

Overwriting Retained Messages

If a publisher sends a message with the retained flag set to true, it overwrites any existing retained message on that topic. This means the last retained message is replaced with the new one.

Clearing Retained Messages

To remove a retained message for a specific topic, a publisher sends an empty message with the retained flag set to true. When the broker receives this empty retained message, it will clear the retained message for that topic, and new subscribers will not receive any previous state.

Topic: home/living-room/temperature
Payload:
Retain: true

Non-Retained Messages

If a publisher sends a message with the retained flag set to false (or does not set the retained flag), the broker treats the message as a regular non-retained message. It is delivered to current subscribers but is not saved for future subscribers.

Topic: home/living-room/temperature
Payload: 25.5
Retain: false

For Example, a smart lighting system could publish a retained message indicating the current brightness level. When a new subscriber, such as a mobile app or another device, connects to the MQTT network, it will immediately receive the most recent brightness status

Last Will and Testament (LWT) in MQTT

LWT is a mechanism that enables MQTT clients to define a message that the broker will publish on their behalf in the event of unexpected disconnection. In-home automation, this feature can be valuable for notifying other devices or systems about the status of a disconnected device. To use the LWT feature, a client sets up a "last will message" and associates it with a specific "last will topic" when connecting to the broker. If the client disconnects abnormally, the broker automatically publishes the last will message to the last will topic on behalf of the disconnected client.

Broker Address: mqtt.example.com
Broker Port: 1883
Client ID: client123
Last Will Topic: home/living-room/presence
Last Will Message: Client ABC disconnected unexpectedly

For Example, if a temperature sensor unexpectedly loses connection to the MQTT network, it can set its LWT message to indicate an "Offline" status. Subscribed devices, such as thermostats or control panels, will receive the LWT message and take appropriate actions, such as adjusting the temperature setpoint or triggering an alert.

MQTT Security Best Practices

In this section, we will delve into MQTT security best practices, focusing on two key areas: securing MQTT communication with authentication and encryption, and implementing access control and user permissions.

Securing MQTT communication with authentication and encryption

Authentication is essential to verify the identity of MQTT clients. Implementing strong authentication mechanisms, such as client certificates, username/ password authentication, ensures that only authorized devices can connect to the MQTT broker. Encryption, achieved through protocols like TLS/SSL, adds an additional layer of security by encrypting the communication between MQTT clients and the broker. Thus, preventing unauthorized access.

Implementing access control and user permissions in MQTT

Access control is crucial in MQTT environments to regulate which clients can publish or subscribe to specific topics. By implementing access control lists (ACLs) and defining policies administrators can define granular rules that restrict or permit access based on client identifiers and topic patterns. User permissions allow fine-grained control over the actions a client can perform, such as publishing, subscribing, or modifying topics. By carefully configuring access control and user permissions, MQTT brokers can ensure that data is shared only with authorized clients, preventing unauthorized actions and maintaining the integrity of the system.

MQTT Future Trends and Advancements

MQTT has been a continuously evolving field. With the recent advancement in MQTT architecture and wide adoption of MQTT 5.0, its future continues to be promising. Here are the key features and advancements introduced in MQTT 5.0

Improved Session Management

Clients can now request to resume previous sessions, allowing for more efficient reconnections and reducing the need for redundant messages and state reinitialization. Another feature of MQTT 5.0 is Session Expiry, It allows us to wait for a certain time before ending the session, this helps in case of short disconnects

Topic Alias

Topic alias is a new feature introduced in MQTT 5.0 to improve efficiency and reduce the overhead of message communication between MQTT clients and brokers. In traditional MQTT, every message sent from a client to the broker includes the full topic string, which can be relatively long and repetitive. Topic aliasing provides a way to represent the topic string with a numeric identifier (alias) during the initial communication.

For Example, In the case of MQTT v3.1

#In MQTT 3 Publish packets look like this
Packet ID: 100
Topic Name: home/living-room/temperature
Qos: 1
Payload: 25.3

#next Packet
Packet ID: 101
Topic Name: home/living-room/temperature
Qos: 1
Payload: 25.3

In the case of MQTT 5

#In MQTT5 publish packets look like
Packet ID: 100
Topic Name: home/living-room/temperature
Topic Alias: 421
Qos: 1
Payload: 25.3

#next packet
Packet ID: 101
Topic Name:""
Topic Alias: 421
Qos: 1
Payload: 25.3

Shared Subscriptions

MQTT 5.0 introduces shared subscriptions, which allow multiple clients to share the message load from a single topic subscription facilitating load balancing and more dynamic messaging patterns. Shared subscription uses the following topic structure:
$share/{group_name}/{topic_name}

$share: prefix indicates this will be a shared subscription
group_name: group_name will act as the group identifier for all the clients
topic_name: actual topic for subscription

For example, There are three clients C1, C2 and C3 subscribed individually to a topic let’s say “state/processing”. Instead of all three clients subscribing individually to the topic "state/processing" they can form a shared subscription group called "processing_group". So the shared topic, in this case, would be “$share/processing_group/state/processing”

User Properties

This feature allows users to add custom key-value pairs to MQTT packets, enhancing extensibility and enabling the inclusion of additional metadata or application-specific information.

Message Expiry

Publishers can set an expiry time on messages, ensuring that outdated or stale data does not persist in the system, leading to more accurate and up-to-date information.

Payload Format Indicators

This feature allows clients to interpret message payloads correctly, making it easier to handle data in different formats and reducing ambiguity.

Reason Codes

Response messages now include reason codes, providing clearer indications of the outcome or status of the requested operation. In MQTT v3 there were reason codes only for CONNACK and SUBACK. Whereas, in MQTT v5 the number of reason codes has been extended to 43 earlier. There are reason codes for the following:

PUBREL
PUBREC
PUBCOMP
UNSUBACK
AUTH
PUBACK
CONNACK
SUBACK
DISCONNECT.

Bidirectional Communication

MQTT 5.0 enables clients to send messages to both subscribers and publishers, opening up possibilities for advanced request-response patterns.

Custom Response Topics

Publishers can now define custom response topics, allowing for more organized and targeted message handling in complex applications.

Subscription Identifiers

MQTT 5.0 introduces subscription identifiers, enabling more efficient management of subscriptions and reducing network overhead.

MQTT-SN (MQTT for Sensor Networks)

MQTT-SN (Message Queuing Telemetry Transport for Sensor Networks) is a variant of MQTT designed for resource-constrained devices and sensor networks. It uses UDP for efficient communication in wireless or unreliable networks. MQTT-SN can work without a centralized broker, using gateways for message routing. It supports short topic names to save bandwidth and provides both asynchronous and synchronous messaging patterns. Devices can enter sleep mode and wake up periodically for energy conservation. While not directly compatible with standard MQTT, MQTT-SN is valuable for connecting resource-constrained devices to IoT systems, serving applications like wireless sensor networks and industrial automation.

Conclusion

In this article, we explained MQTT concepts in detail. We got an understanding of MQTT architecture and its different components. We also discussed recent advancements in MQTT 5.0 and its future prospects. I hope you gained valuable insights into leveraging MQTT for IoT applications.

I hope you gained valuable insights from this comparison summary. As we continue to come up with more interesting tutorials and maker content, we invite you to explore Bytebeam further and see how our platform can empower your projects.

Stay tuned!