The Rust based MQTT broker (Rumqtt) gets new internals
Overview
Rumqttd was born out of our need for an extensible and performant broker to support the wide variety of IoT use cases that we solve at Bytebeam. We’ve always believed that open source is the right way to build core infrastructure components. Towards that end, we are now open-sourcing a new version of our MQTT broker.
From an MQTT client perspective, this new version is backward compatible with the earlier version of rumqtt. However, the internals and config file syntax have been revamped to make future development easier.
We will maintain both versions for the next two months, post which the current version will be replaced by the new version. The new version is in the rumqttd folder in the rumqtt project. The earlier version has been moved to the rumqttd-old folder.
What has changed?
The internal architecture has been revamped to support protocols other than MQTT. This will allow us to implement other protocols used in the IoT world such as WebSockets or plain TCP/TLS based protocols.
Multi protocol support
MQTT5 Support
This version has a first cut implementation of MQTT5 support. The following features are supported:
- Connects
- Publish
- Subscribe
We are yet to implement some of the more advanced features of MQTT5, such as:
- Topic aliases
- AUTH packet
- Shared subscriptions
- Handling MQTT5 specific packet properties
Improved commit log implemenation
Rumqtt internally maintains a commitlog that publishers write to and subscribers read from. The earlier design maintained a copy of the entire commit log for each subscriber. In this version, there is now only one copy of the commit log for all subscribers and we maintain an offset into the commit log. This drastically reduces memory usage when there are multiple subscribers for the same subscription.
Config file changes
Some changes have been made to the `rumqttd` configuration file to better support all the new features. This includes a new naming scheme for servers to include the MQTT protocol version and more. See this diff for list of changes.
Improved logging
Logs have been improved with more information like device id, packet and event types, and metadata along with formatting and indentation to be able to filter and debug better.
How to try?
The fastest way to test out the new rumqttd is by running it in docker, using the following commands:
$ docker run -p 1883:1883 -it bytebeamio/rumqttd --config demo.toml
You can then connect to the broker with any MQTT client on port 1883.
How to migrate?
No changes are required on the client side. Existing clients should work as is. However, you need to upgrade your config file to the newer syntax. Please refer to this diff to see what changed
