Choosing the Right IoT Protocol: A performance analysis on MQTT, CoAP, and HTTP

The importance of selecting the right protocol cannot be overstated. In an increasingly interconnected world, where seamless communication is the lifeblood of IoT ecosystems, making an informed protocol choice can significantly impact the efficiency, reliability, and scalability of your project. To help you with choosing the right protocol, we have compiled a multipart guide. In this series, we will take up different aspects like fundamental concepts, performance analysis and security analysis of MQTT, CoAP and HTTP, draw a comparison between respective protocols and provide you with different matrics to make an informed decision.

In Part 1 of our multipart series, "Choosing the Right IoT Protocol for Your Project: A guide on MQTT, CoAP and HTTP" we explored the fundamental concepts, data flow intricacies, and contextual use cases associated with MQTT, CoAP, and HTTP. In this part, we will take up the performance aspect of each of these protocols.

What to expect:
In this blog, we will be conducting a performance analysis of MQTT, HTTP, and CoAP. We will be using the Wireshark tool to break down protocol packets to look into necessary overheads, quantifying speed and efficiency at different hierarchical levels. In Part 3 of this series, we will look at the security aspects of  coap vs mqtt vs http

Why Performance Analysis Matters

In Internet of Things (IoT) communication, the performance of protocols plays a pivotal role in determining the success and efficiency of an IoT project. Each protocol—MQTT, CoAP, and HTTP—brings its own set of performance attributes that shine under distinct circumstances and use cases.

MQTT, renowned for its lightweight nature and real-time capabilities, excels in scenarios where reliable data transfer and low latency are paramount. CoAP is designed specifically for resource-constrained devices. Being lightweight in nature, CoAP thrives in environments with limited network bandwidth and low power requirements. HTTP showcases its expertise in scenarios where interoperability and human-device interaction are vital.

By understanding how each protocol's performance attributes align with various use cases, IoT practitioners can make informed decisions that lead to efficient, responsive, and optimized communication within their IoT ecosystems.

MQTT Performance Metrics

In this section, we will be going through the MQTT protocol stack. Using Wireshark, We will evaluate the MQTT connection and publish packets. We will quantify packet overheads at different layers, and analyze latency and data rate at different layers

MQTT Protocol Stack

• Application Layer: MQTT Protocol
  • Facilitates lightweight publish-subscribe messaging patterns for IoT devices.
  • Allows efficient publication of messages from senders (publishers) to a central hub (broker) and delivery to interested recipients (subscribers).
• Transport Layer: TCP (Transmission Control Protocol) or UDP(User Datagram Protocol)
  • Provides reliable and connection-oriented communication.
  • Manages logical communication channels, ensuring orderly delivery of MQTT messages.
  • Implements flow control, error correction, and acknowledgement for robust and reliable data exchange.
• TLS Layer (in case of TCP Connection) or DTLS (in case of UDP connection) - Optional
  • TLS (Transport Layer Security) Establishes secure and encrypted communication channels between MQTT clients and brokers.
  • Ensures data confidentiality, integrity, and authenticity, preventing unauthorized access and tampering.
• Network Layer: IP (Internet Protocol)
  • Enables device identification and communication across interconnected networks.
  • Handles IP routing to efficiently direct MQTT messages to intended destinations.
• Data Link Layer: WiFi or Ethernet (Often Integrated with Network Layer)
  • Manages communication within the same local network segment.
    Ensures reliable transmission of data frames, error detection, and media access control.
• Physical Layer: WiFI or Ethernet
  • Defines hardware connections, signalling, and data encoding for data transmission.
  • Facilitates data exchange over physical mediums like cables.

Quantifying packet overheads at different layers

We used Wireshark to evaluate the performance of MQTT packets. We analysed the connection, Publish and Subscribe requests.

To carry out this experiment we have chosen a payload of the size of 58 bytes with QoS 0 and topic name “mqtt/rpi”. The payload looks like this

{"payload": "Hello, This message is from publisher client"}

Transport-level security handshake

It involves the client sending a "Client Hello" message to the broker. The size of “Client Hello” is 102 bytes. It contains information about signature algorithms, protocol, key exchange mode, session tickets etc. The broker responds with a "Server Hello" message. The size of “ServerHello” is 86 Bytes. Then the client exchanges keys with a broker. The size of “Client Key Exchange” is 66 bytes. Each of these has a TLS record header of 5 bytes and a TLS handshake header of 4 bytes common

Connection request and acknowledgement

The size of the connection request was 80 bytes. The connection request consisted of an IP header, TCP header and MQTT header. The size of the IP header is 20 bytes and it consists of information about the destination IP address and source IP address. The size of the TCP header is 32 bytes and it consists of information about the source and destination

port. The size of the MQTT Header is 14 bytes and it consists of information about flag, message length, protocol name length, protocol name, version, connect flags, keep alive and client ID. Connection Acknowledgement consists of 4 bytes

MQTT Connect Header looks like this

MQTT Connect Acknowledgement Header looks like this

Publish

The size of the publish request is 136 bytes. The publish request consisted of an IP header, TCP header and MQTT header. The size of the IP header is 20 bytes and it consists of information about the destination IP address and source IP address. The size of the TCP header is 32 bytes and it consists of information about the source and destination port. The size of the MQTT Publish Header is 70 bytes it consists of information about header flags of 1 byte, message length, topic length, topic name, and message. For this experiment, we have considered a payload of 58 bytes and a topic of 8 bytes

MQTT Publish Header looks like this

Subscribe request and acknowledgement

The size of the subscribe request was 81 bytes. The subscribe request consisted of an IP header, TCP header and MQTT header. The size of the IP header is 20 bytes and it consists of information about the destination IP address and source IP address. The size of the TCP header is 32 bytes and it consists of information about the source and destination port. The size of the MQTT Subscribe Header is 15 bytes. it consisted of information about header flags of 1 byte, message length, message identifier, topic length, topic name, and request QoS. For this experiment, we have considered the topic of 8 bytes

MQTT Subscribe Header looks like this

MQTT Subscribe Acknowledgement header looks like this

Analyzing Latency

Experiment setup: We used Wireshark to evaluate the latency of MQTT packets. Our hardware setup consists of a Raspberry Pi as an MQTT broker and a laptop as an MQTT publish client. We have used Mosquitto as an MQTT broker. Our Python script uses paho-mqtt dependency which acts as an MQTT client. You can find the respective code here. We analysed the time consumed by connection requests, publish requests and subscribe requests.

To carry out this experiment we have chosen a payload of the size of 58 bytes with QoS 0 and topic name “mqtt/rpi”. The payload looks like this

Host: ip assigned to raspberry
Topic: mqtt/rpi
Port: 1883

{"payload": "Hello, This message is from publisher client"}

CoAP Performance Metrics

In this section, we will be going through the CoAP protocol stack. Using Wireshark, We will evaluate the CoAP GET request and response. We will quantify packet overheads at different layers and analyze latency

CoAP Protocol Stack

• Application Layer: CoAP Protocol
  • CoAP sits on the application layer. CoAP Application comprises of Request/Response layer and messaging layer.
  • Facilitates interactions using a request-response model for resource management.
• Presentation Layer: DTLS (Datagram Transport Layer Security)
  • Provides encryption, authentication, and data integrity through secure communication channels.
  • Ensures confidentiality and secure data exchange between CoAP clients and servers.
• Transport Layer: UDP (User Datagram Protocol)
  • Connectionless protocol supporting low-latency and efficient CoAP message transmission.
• Security Layer: DTLS (Datagram Transport Layer Security)
  • Since CoAP uses UDP, which is connectionless and lacks the reliability and security features of TCP, DTLS is used to provide security features equivalent to TLS.
  • DTLS operates at the transport layer like TLS but is adapted for use with datagram protocols like UDP.
  • DTLS provides encryption, authentication, and data integrity to secure CoAP communication over UDP.
• Network Layer: IP (Internet Protocol)
  • Enables device identification and communication across networks.
  • IP routing ensures accurate message delivery regardless of device locations.
• Data Link Layer: WiFi or Ethernet
  • Manages direct communication within local network segments.
  • Ensures reliable data frame transmission, error detection, and media access control.
• Physical Layer: WiFI or Ethernet
  • Defines hardware connections, signalling, and data encoding for data transmission.
  • Facilitates data exchange over physical mediums like cables.

Quantifying packet overheads at different layers

We used Wireshark to evaluate the performance of CoAP packets. We analysed the GET requests and responses.
To carry out this experiment we have chosen a payload of the size of 58 bytes uri “coap”. The payload looks like this

{"payload": "Hello, This message is from publisher client"}

Datagram Transport-level security handshake                                                         Datagram TLS security handshake is similar to MQTT. The size of Client Hello, Server Hello and Client Key Exchange may vary depending on the size of certificates and keys.

GET Request (CON)

The size of the get request is 54 bytes. The request consists of an IP header, UDP header and CoAP header. The size of the IP header is 20 bytes and it consists of information about the destination IP address and source IP address. The size of the UDP header is 8 bytes. It consists of information about the source and destination port and checksum. The size of the CoAP header is 12 bytes and it consists of information about the version, request code, message ID, token, opt name, opt delta and URI path.

The CoAP request header looks like this:

Get Response(ACK)

The size of the get response is 107 bytes. The GET response consists of an IP header, UDP header and CoAP header. The size of the IP header is 20 bytes and it consists of information about the destination IP address and source IP address. The size of the UDP header is 8 bytes. It consists of information about the source and destination port and checksum. The size of the CoAP header is 7 bytes + 58 bytes of payload. It consists of information about the version, request code, message ID, token, and payload.

The CoAP response header looks like this

Analyzing latency

Experiment setup: We used Wireshark to evaluate the latency of CoAP packets. Our hardware setup consists of Raspberry Pi as a CoAP server and a laptop as a CoAP client. Our Python script uses aiocoap dependency. You can find the respective code here.  We have analysed the time consumed by GET requests and responses.
To carry out this experiment we have chosen a payload of the size of 58 bytes with URI “coap”. The payload looks like this

{"payload": "Hello, This message is from publisher client"}

HTTP Performance metrics

In this section, we will be going through the HTTP protocol stack. Using Wireshark, We will evaluate the HTTP GET request and response. We will quantify packet overheads at different layers and analyze latency

HTTP Protocol Stack

• Application Layer: HTTP Protocol
  • Governs web content exchange between clients and servers.
  • Facilitates retrieval and presentation of hypertext resources.
• TLS Layer(optional): TLS (Transport Layer Security) or SSL (Secure Sockets Layer)
  • Encrypts communication for data confidentiality and integrity.
  • Provides authentication and secure data transmission.
• Transport Layer: TCP (Transmission Control Protocol)
  • Ensures reliable, connection-oriented communication.
  • Facilitates orderly and successful data transfer.
• Network Layer: IP (Internet Protocol)
  • Routes and addresses data packets across networks.
  • Accurately directs data to intended destinations.
• Data Link Layer: WiFi or Ethernet
  • Manages communication within local network segments.
  • Ensures reliable data frame transmission and error detection.
• Physical Layer: WiFi or Ethernet
  • Defines hardware connections and signalling methods.
  • Facilitates data transfer over physical mediums.

Quantifying packet overheads at different layers

We used Wireshark to evaluate the performance of HTTP packets. We analysed the GET requests and responses.

To carry out this experiment we have chosen a payload of the size of 58 bytes uri “http”. The payload looks like this

{"payload": "Hello, This message is from publisher client"}

Transport-level security handshake

Datagram TLS security handshake is similar to MQTT or CoAP. The size of Client Hello, Server Hello and Client Key Exchange may vary depending on the size of certificates and keys.

GET Request

The size of the get response is 221 bytes. The GET response consists of an IP header, TCP header and HTTP header. The size of the IP header is 20 bytes and it consists of information about the destination IP address and source IP address. The size of the TCP header was 32 bytes and it consisted of information about the source and destination port, checksum, flags and options. The size of the HTTP header is 155 bytes. HTTP header contains information about the host, user-agent, encoding, connection etc. This is why bandwidth consumption in the case of HTTP is more than MQTT and CoAP. HTTP header looks like this

GET /http HTTP/1.1\r\n
Host:ip-address:port\r\n
User-Agent: python-requests/2.22.0\r\n
Accept-Encoding: gzip, deflate\r\n
Accept: */* \r\n
Connection: keep-alive\r\n
\r\n

GET Response

The size of the get response is 260 bytes. The GET response consists of an IP header, TCP header and HTTP header. The size of the IP header is 20 bytes and it consists of information about the destination IP address and source IP address. The size of the TCP header was 32 bytes and it consisted of information about the source and destination port, checksum, flags and options. The size of the HTTP header is 136 bytes + 58 bytes of message. HTTP header contains information about status code, Content type, Charset, content length etc.

HTTP/1.0 200 OK\r\n
Content-Type: text/html; charset=utf-8\r\n
Content-Length: 58\r\n
Server: Werkzeug/1.0.1 Python/3.7.3\r\n
Date: Tue, 01 Aug 2023 18:55:38 GMT\r\n
\r\n
Payload

Analyzing Latency

Experiment setup: used Wireshark to evaluate the latency of HTTP packets. Our hardware setup consists of Raspberry Pi as an HTTP server and a laptop as an HTTP client. Our Python script uses requests dependency. You can find the respective code here.  We have analysed the time consumed by  GET requests and responses.

To carry out this experiment we have chosen a payload of the size of 58 bytes with URI “http”. The payload looks like this

{"payload": "Hello, This message is from publisher client"}

Performance Comparison Summary

Conclusion

In our investigation comparing the performance of MQTT, CoAP, and HTTP protocols, we centred our attention on two critical factors: protocol overhead and latency. The results of our experiments provide clear insights into the strengths of each protocol.

Our Analysis of protocol overhead unveiled that CoAP exhibits the smallest overhead among the three, making it a compelling choice for applications where minimizing resource consumption is paramount.

Turning to latency, our finding revealed that MQTT is the fastest performer. Despite lower packet overhead, we found that CoAP has higher latency. This is because of multiple factors. Firstly CoAP sessions are generally shorter-lived. Clients make a request to a server, receive a response, and the session terminates. Whereas, In the case of MQTT, The initial connection time is longer. But, once the connection is made it stays active. Secondly, CoAP uses UDP, which lacks the congestion control and flow control mechanisms of TCP. This can lead to increased latency

In summation, our performance evaluation underscores MQTT's agility and CoAP's efficiency in terms of protocol overhead. For applications where low latency is pivotal, MQTT emerges as the frontrunner. CoAP, with its minimal overhead, proves advantageous for resource-constrained devices

I hope you gained valuable insights from this comparison summary. As we continue to come up with more interesting tutorials and maker content, we invite you to explore Bytebeam further and see how our platform can empower your projects.

Stay tuned!