Choosing the Right IoT Protocol: A Comprehensive Guide on MQTT, CoAP, and HTTP

In the ever-evolving world of the Internet of Things (IoT), the debate over selecting the most suitable communication protocol for IoT systems continues to prolong.  

Which protocol to use?
Do I need to use different protocols at different levels of my IoT architecture?

To answer these questions we have compiled a multipart guide. In this series we will go through the core aspects of these protocols, providing insights into their fundamentals, performance, and security features that would help you to choose the right protocol out of MQTT, CoAP and HTTP for your IoT project

What to expect:  In part 1 of this series, we will explore some of the fundamental concepts associated with three major IoT protocols - MQTT, HTTP, and CoAP. Additionally, we will provide some use cases that illustrate when and how to leverage these protocols effectively in IoT applications. In Part2 and Part 3 of this series we will look at performance and security aspects of   coap vs mqtt vs http

So let's get started

An Overview: MQTT, HTTP and CoAP

In this section, we will provide you with an overview of MQTT, HTTP and COAP. We will walk through fundamental concepts, data flow and use-cases of respective protocols.  

MQTT Overview

In this section, we will go through the fundamental concepts of MQTT, gaining insights into its data flow and operational principles. Additionally, we will examine a real-world scenario where MQTT proves to be beneficial.

MQTT Fundamentals

MQTT is an application layer protocol. It uses TCP in the transport layer and IP in the network and data link layer. At the core of MQTT's application layer is a publish-subscribe model. In this model, there are two main components: MQTT clients and the MQTT broker. The clients are the devices or applications that produce and consume data. They can be sensors, actuators, smartphones, or any IoT device. The broker acts as an intermediary, responsible for receiving messages from publishers and delivering them to subscribed clients.

To get a detailed walkthrough of MQTT concepts, Check out our beginner's guide to MQTT. Let's look at some of the fundamental concepts of MQTT in detail.

Publish-Subscribe Model

MQTT's Publish-Subscribe model contains a client. The client can either be Publisher or Subscriber. Publishers send messages to a broker by publishing them to topics, and subscribers receive messages by subscribing to relevant topics or matching filters. This decoupled approach ensures efficient and scalable data exchange, allowing real-time updates and event-driven interactions. Embracing the Publish-Subscribe model, MQTT empowers IoT ecosystems to operate seamlessly, enabling reliable and dynamic interactions between devices and applications.

Example: In the context of IIoT, imagine you have various systems in a smart industry, such as HVAC systems, lighting systems, and sensors. Each of these devices can be a publisher, sending data or status updates to specific topics on the MQTT broker. For instance, an HVAC system can publish temperature readings to the "industry/temperature" topic. On the other hand, you can have subscribers that are interested in receiving data from specific topics. For example, a mobile app or a control panel can subscribe to the "industry/temperature" topic to display the current temperature.

Lightweight Messaging Protocol

MQTT (Message Queuing Telemetry Transport) is a lightweight and efficient messaging protocol designed for low-power, low-bandwidth IoT devices. Additionally, the protocol's binary payload format further significantly reduces the data size

Broker Based Communication

MQTT uses a central message broker that acts as an intermediary between publishers and subscribers, facilitating efficient message routing and delivery. When a device publishes a message on a specific topic, the broker receives the message and efficiently forwards it to all the subscribers interested in that topic. Some of the well-known MQTT brokers are

• Rumqttd: Rumqttd is an open-source MQTT broker written in Rust. It’s lightweight robust and performant. It is embeddable, meaning you can use it as a library in your code and extend functionality.
• Mosquitto: Mosquitto is an open-source MQTT broker. Mosquitto is lightweight and suitable for low-power IoT devices and servers.
• HiveMQ: HiveMQ is a scalable and feature-rich MQTT broker designed for enterprise-level IoT deployments, providing reliable and secure messaging communication between devices.
• EMQ X: EMQ X is a high-performance and scalable MQTT broker that supports massive connections and advanced features, making it ideal for large-scale IoT deployments.

• Quality of Service (QoS) Levels: MQTT offers three QoS levels (0, 1, 2) that determine the message delivery guarantees. QoS 0 ensures "at most once" delivery, QoS 1 provides "at least once" delivery, and QoS 2 guarantees "exactly once" delivery.
• Persistent Sessions: MQTT supports persistent sessions, enabling devices to maintain their subscription status and queued messages across disconnections, ensuring message reliability.

MQTT Handshake and Data Flow

The MQTT handshake process involves several steps, including TCP connection establishment, TLS handshake (if using TLS/SSL for encryption), MQTT connects handshake, publishing handshake, and TCP termination. Let's break down each step:

TCP Connection Establishment

The MQTT client initiates a TCP connection to the MQTT broker by sending a SYN (Synchronize) packet to the broker. The Broker responds with an SYN-ACK packet, acknowledging the connection request. Finally, the Publisher sends an ACK packet to the Broker, confirming the connection establishment.

TLS Handshake

It involves the client sending a "Client Hello" message to the broker, followed by the broker responding with a "Server Hello" message along with a certificate signed by Certificate Authority (CA). Then, they exchange cryptographic information to establish encryption keys for secure communication. After this broker request for client certificate and the client provides its client certificate. Once the handshake is completed, the MQTT data exchange occurs over the encrypted channel.

MQTT Connect Handshake

After the TLS handshake is successfully done. The client sends a CONNECT message containing its identification information, such as client ID, username, password (if required), and MQTT session details. Upon receiving the CONNECT message, the broker responds with a CONNACK message, indicating whether the connection is successful.

Data publishing

Once the MQTT handshake is completed, the Publisher can publish data to the Broker. The Publisher sends a PUBLISH packet containing the data payload and the topic to which the data should be published.

TCP Termination

Now to terminate the TCP connection, the Client sends a FIN packet to the broker. The broker then sends an ACK-FIN packet to terminate the session. Broker on receiving ACK-FIN gives the final acknowledgement to a broker using ACK packet

When to use MQTT?

Let’s take up a scenario to answer this question. Consider a use case of Environmental sensors deployed in a remote forest to predict and minimize the chances of a forest fire. These sensors monitor temperature, and humidity and detect gas, and fire. So, Why use MQTT in this scenario? There are many reasons to choose MQTT for this use case, Some of them are the following

MQTT can be used when:

• Efficient data transfer is required: In a remote area with limited connectivity and constrained resources, efficient data transfer is crucial to conserve power and optimize bandwidth. MQTT's lightweight and low overhead message format makes it ideal for transmitting sensor data efficiently.
• The low Data rate is required: Since the focus is on environmental monitoring data, which may not be high-volume, MQTT's small packet size and minimal message overhead are well-suited for handling low data rates.
• Data Reliability is required: In a critical forest fire context, reliable data transmission is essential. MQTT's Quality of Service (QoS) levels provide options to ensure data reliability, especially for important sensor readings.
• Asynchronous Data Transfer required: In remote areas, network connectivity can be intermittent. MQTT's asynchronous nature allows the sensors to publish data when connectivity is available, and the conservation team's data receiver can subscribe to the topics to retrieve data when connected.
• High Scalability is required: Forest areas can have many sensors spread across vast regions. MQTT's publish-subscribe model allows seamless scalability, enabling new sensors to publish data, and multiple receivers to subscribe to relevant topics for monitoring and analysis.

CoAP Overview

In this section, we will go through the fundamental concepts of CoAP, gaining insights into its data flow and operational principles. Additionally, we will examine a real-world scenario where CoAP proves to be beneficial.

CoaP Fundamentals

Constrained Application Protocol (CoAP) stands as a prominent communication protocol designed to address the challenges posed by resource-constrained devices and low-power networks. CoAP is an application layer protocol that constitutes fo Messaging layer and Request Response layer. It uses UDP as the transport layer and IP as the network and Data Link layer.

In this section we will we will go through some of the fundamental concepts of the CoAP protocol, examining its client/server model, RESTful design principles, and how it excels as a Constrained Application Protocol. So let’s have a look at some of the fundamental concepts of CoAP.

Client/Server Model

At the core of CoAP's architecture lies the client/server model, where communication occurs between CoAP clients and CoAP servers. CoAP clients, often IoT devices, initiate requests to CoAP servers, which can be other IoT devices or CoAP-enabled gateways. CoAP clients issue requests to CoAP servers using standard methods such as

• GET(retrieve resource representation)
• POST(create resource)
• PUT(update resource)
• DELETE(delete resource)

On the other end, CoAP servers receive and process these requests, and based on the method and URI, they perform the corresponding operation on the resource. CoAP servers then generate responses, which are sent back to the clients. The response may include resource representation, status information, and additional metadata.

Example: Open thread utilizes CoAP as a communication protocol. In OpenThread Stack, CoAP enables efficient and lightweight messaging between Thread devices

Restful Design Principles

CoAP's architectural design adheres to RESTful (Representational State Transfer) principles. CoAP uses a similar request-response paradigm as HTTP, where clients send requests to servers to access resources identified by URLs. This RESTful approach enables seamless integration with web services, simplifying interactions between CoAP clients and servers and ensuring interoperability across diverse platforms.

Constrained Application Protocol

CoAP's architecture is built specifically for constrained environments, such as resource-constrained IoT devices and networks. The following reasons make it suitable for resource-constrained applications:

• Lightweight Messaging: CoAP uses a compact message format, minimizing data overhead and reducing the size of transmitted packets.
• Binary Header and Options: CoAP employs binary encoding for headers and options, further reducing message size and conserving bandwidth.
• UDP Protocol: CoAP operates over UDP, a lightweight transport protocol, which consumes less memory and processing power compared to TCP.
  Simple
• Request-Response Model: CoAP follows a simple request-response paradigm, enabling straightforward interactions between clients and servers.
• Caching and Proxying: CoAP supports caching and proxying, reducing the need for frequent client-server interactions and enhancing overall efficiency.

CoAP Handshake and Data Flow

CoAP is designed for resource-constrained devices, making the handshake lightweight and suitable for the IoT ecosystem. Below is a step-by-step breakdown of the Client and Server handshake for CoAP

Client Discovery and Server Response

The CoAP client initiates communication by discovering available resources on the CoAP Server.

The client sends a CoAP multicast message (DISCOVERY) to the local network, asking for available resources or services from CoAP servers. The CoAP server once receiving the discovery request responds with a multicase message (RESPONSE), providing information about available resources and their URIs (Uniform Resource Identifiers).

Client Request and Server Response

After receiving Server’s response on available resources, The CoAP client sends a request to the server’s URI to access the resources. The request can be of different types, such as GET, POST, PUT or DELETE. On receiving the client request, the server responds with a CoAP response containing the requested information or indicating the success or failure of the requested operation. CoAP responses include various codes, such as 2.05 (Content) for a successful response or 4XX (Client Error) and 5XX (Server Error) for error responses.

Example: For example, you have an IoT device say a thermostat which uses CoAP as the underlying protocol. The thermostat is a CoAP client that wants to retrieve the current temperature reading from a CoAP server, which is a central gateway managing various smart home devices. The CoAP client request might look like this.

GET /temperature

The CoAP server receives the GET request from the client and processes it. It then sends back a CoAP response with the requested temperature resource representation and a success status code.

2.05 Content
Content-Format: application/json

{
  "temperature": 25.5,
  "unit": "C"
}

When to use CoAP?

Let’s take up a scenario to answer this question. Consider a use case of a home automation system, various smart devices, such as thermostats, light switches, and door locks, need to communicate with a central smart home hub or gateway. The system requires low latency, fast data transfer, low power consumption, and a low data rate for efficient communication.

CoAP is used when:

• Low Latency and Fast Data Transfer are required: CoAP is designed for low-latency applications, making it well-suited for home automation applications that demand quick response times, such as controlling lights or adjusting thermostat settings.
• Low Power Requirements: In a home automation setup, many smart devices are battery-powered or energy-efficient. CoAP's lightweight nature and UDP-based communication minimize power consumption, extending the battery life of IoT devices.
• Low Data Rate: Home automation applications typically involve transmitting small amounts of data, such as sensor readings or status updates. CoAP's compact message format and minimal overhead ensure efficient data transfer, optimizing network bandwidth.

OpenThread and CoAP

OpenThread is an open-source implementation of the Thread networking protocol for IoT devices. Thread is an IPv6-based mesh networking protocol that facilitates efficient and secure communication between smart devices. OpenThread uses CoAP as its application layer protocol for effective message exchange in IoT networks. CoAP's simplicity, low overhead, and real-time capabilities align perfectly with the requirements of IoT devices operating in a Thread mesh network, making it an ideal choice for home automation scenarios implemented with OpenThread.

HTTP Overview

In this section, we will go through the fundamental concepts of HTTP Websockets and HTTP push, gaining insights into its HTTP WebSockets data flow and operational principles. Additionally, we will examine a real-world scenario where CoAP proves to be beneficial.

HTTP Fundamentals

Hypertext Transfer Protocol (HTTP) serves as the foundation of data communication on the World Wide Web. HTTP's architectural design plays a vital role in how web browsers interact with servers to request and retrieve resources. Traditionally, HTTP followed a request-response model, where clients initiated requests, and servers responded with the requested data. However, with the evolution of web applications and the demand for real-time and interactive features, this traditional approach started showing limitations. To address these limitations and usher in a new era of communication, HTTP WebSockets and HTTP Push emerged as revolutionary solutions, transforming the HTTP protocol and enabling seamless real-time communication between clients and servers.

Overview of HTTP WebSockets

WebSockets introduced a paradigm shift by establishing a full-duplex, bidirectional communication channel over a single TCP connection. Unlike traditional HTTP, where connections were opened and closed for each request, WebSockets maintained a persistent connection, allowing both clients and servers to send and receive messages simultaneously. This capability revolutionized real-time applications, enabling features like instant messaging, live updates, and collaborative editing. With low latency, reduced overhead, and efficient data exchange, WebSockets have become the go-to solution for interactive and responsive web applications.

HTTP WebSockets Handshake and Dataflow

The WebSocket handshake begins when the client sends an HTTP GET request to the server. The client's request contains an additional header called "Sec-WebSocket-Key," which is a randomly generated key used for security purposes during the handshake.

Upon receiving the client's request, the server evaluates the "Sec-WebSocket-Key" and performs a cryptographic process to create an "Accept" key. This "Accept" key is then included in the server's HTTP response to the client.

The server's response includes the "Upgrade" header set to "websocket," indicating the successful upgrade to the WebSocket protocol. It also sets the "Connection" header to "Upgrade," affirming the transition to the new protocol.

With the handshake successfully completed, the WebSocket connection is established, allowing both the client and server to communicate bidirectionally in real time. This persistent, low-latency connection remains open until either the client or server decides to close it.

Overview of HTTP Push

HTTP Push, also known as HTTP Server Push or HTTP Streaming, is a communication technique that enables servers to proactively send data to clients without waiting for client requests. This approach revolutionizes traditional request-response communication by allowing real-time updates and event-driven information delivery to clients. HTTP Push is particularly beneficial for applications that require instant updates, live notifications, and continuous data streams.

To implement HTTP Push, servers use two primary methods: Long Polling and Server-Sent Events (SSE).

Long Polling Method

Long Polling is a simple HTTP Push technique where the client sends a request to the server, and the server holds the request open until new data or events are available. Once the server has new information to provide, it responds to the client's request with the updated data. Upon receiving the response, the client immediately sends a new request to the server, effectively maintaining a persistent connection.

Server-Sent Events

Server-Sent Events (SSE) is a standardized HTTP Push technology that offers a more efficient and standardized approach compared to Long Polling. SSE facilitates a unidirectional stream of events from the server to the client over a single, long-lived HTTP connection. Unlike Long Polling, SSE leverages the existing HTTP infrastructure, making it easier to implement and reducing the overhead of frequent connection setups.

With SSE, the server can send a stream of events to the client without waiting for requests, enabling real-time updates in an efficient manner. The data sent by the server is encapsulated in a series of events, which the client can process as they arrive. SSE supports various event types, including message events, error events, and even custom events, providing flexibility in handling different data types.

When to use HTTP?

Let’s take up a scenario to answer this question. Consider a use case of a kiosk system, a stand-alone interactive terminal is set up to provide information, services, or self-service options to users in public spaces. The kiosk requires web integration, human-device interaction and support for firmware updates and configuration. These cases make HTTP a prominent choice.

HTTP is used when:

• Web Integration is required: HTTP is a natural choice for kiosk systems that require web integration to access online services, display dynamic content, or interact with web APIs. It enables seamless communication with web servers and cloud platforms, allowing the kiosk to display real-time information or retrieve data from online sources.
• Human-Device Interaction: Kiosk systems often have touchscreens or input interfaces to facilitate user interaction. HTTP's human-readable format makes it convenient for developers to implement user-friendly interfaces and enables users to interact with the kiosk through web browsers.
• Firmware Updates and Configuration: Kiosks may require occasional firmware updates or configuration changes. HTTP can be used to fetch firmware files from remote servers and apply updates to the kiosk software, ensuring it stays up-to-date with the latest features and security enhancements.
• In IoT Gateways: In some cases, kiosk systems may serve as gateways to
  IoT devices or networks. HTTP is often employed in IoT gateways to translate communication between IoT devices (using protocols like CoAP or MQTT) and web services or cloud platforms that utilize HTTP as their primary communication protocol.

Comparison Summary (mqtt vs coap vs http)

Adoption Trends in IoT

According to an IoT developer survey conducted in 2022 by the Eclipse Foundation, MQTT continues to be the most widely used IoT communication protocol. MQTT benefits from a rich library ecosystem, including implementations like Eclipse Paho and MQTT.js, making it easy to integrate into various platforms. Although CoAP and HTTP also enjoy some library and community support, MQTT's extensive adoption and well-established presence in the IoT domain give it an upper hand.

While MQTT stands out as the most widely adopted protocol with extensive library and community support, the choice of the ideal protocol should align with project requirements and expertise. Each protocol has its strengths, and careful consideration of library and community aspects ensures successful IoT solutions based on MQTT, CoAP, or HTTP.

Conclusion

In conclusion, MQTT, CoAP, and HTTP offer distinct advantages for IoT applications. MQTT's lightweight publish-subscribe model suits low-bandwidth networks. CoAP's RESTful design and UDP support optimize energy consumption in constrained environments. HTTP WebSockets' full-duplex communication, and HTTP Push's asynchronous data delivery—enhance responsiveness. Each protocol brings unique features to cater to specific IoT needs. Choosing the right one depends on project requirements, ensuring seamless communication, security, and performance in the evolving Internet of Things.

I hope you find this guide useful. As we continue to come up with more interesting tutorials and maker content, we invite you to explore Bytebeam further and see how our platform can empower your projects.

Stay Tuned!