Visualizing Wireless Sensor network data on bytebeam cloud using Bytebeam SDK for ESP-IDF

Learn creating wireless sensor network using Sensor nodes and gateway using Bytebeam ESP-IDF SDK and visualizing sensor data in Bytebeam cloud

In a previous blog post titled "Visualizing Wireless Sensor Network Data on Bytebeam Cloud using Bytebeam SDK for Arduino," we delved into the intricacies of wireless sensor networks (WSNs) and explored the process of setting up an ESP32 gateway and sensor nodes using the ESP Now protocol. Building upon that foundation, this blog takes a leap forward as we dive into the world of ESP-IDF, to achieve the same goal—visualizing WSN data on Bytebeam Cloud.

In this step-by-step guide, we will establish point-to-point communication between the ESP32 gateway and ESP32-based sensor nodes. This can be accomplished using the following easy steps

1. Setting up the Bytebeam cloud console: We will be setting up a new project on the Bytebeam cloud console.
2. Provisioning ESP32 Gateway: We will provision ESP32 Gateway using provisioning JSON.
3. Configuring ESP32 Gateway: We will create an ESP32 gateway to receive sensor node messages using the ESP Now protocol and send the data to the Bytebeam cloud.
4. Configuring ESP32 Sensor Node: We will go through configuring ESP32 as a slave device and send sensor data using ESP Now.
5. Data visualization on the Bytebeam cloud console: We will explore the data visualization tools by Bytebeam and learn how to view and analyze data from the connected Arduino device in real time.

So let's get started

Hardware Requirements

• ESP 32 dev module (2 for sensor node and 1 for gateway)
• SHT35 temperature and humidity sensors (two for each sensor node)

Getting Started with ByteBeam Cloud

Firstly, you need to create an account on Bytebeam Cloud (cloud.bytebeam.io). In case you already have an account with Bytebeam Cloud then you can proceed further with this guide or else follow Getting started with Bytebeam cloud.

Creating a new project

Login to your Bytebeam account to access the cloud console. Now, create a New Project and give it a name. I have given it the name “esp32wsn”. In the next screen, you will be asked to select the project. Select the newly created project and proceed further

Provisioning a new device

Click on Create Device button to provision a new device to Bytebeam cloud. And hit Submit.

• As you hit submit in the prompt. A new JSON file will be downloaded which contains the authentication certificate, broker, device_id, and project id. You need to save this file in the ESP file system. We will talk about this in a later section. The JSON will look like this.

{
 "project_id": "****",
 "broker": "******",
 "port": 8883,
 "device_id": "**",
 "authentication": {
      "ca_certificate":*"*********",
      "device_certificate": "**********",
      "device_private_key": "***********"
      }
}

Setting up ByteBeam SDK for ESP-IDF

ByteBeam provides an SDK, built on top of the ESP-IDF framework to connect to the ByteBeam cloud platform.  This guide assumes that you have a fair understanding of ESP-IDF and that you have already set up ESP-IDF on your machine. In case you haven’t done this you should follow ESP-IDF Getting Started Guide before going further.

Next, you can follow Getting started with Bytebeam Part 1 to configure the Bytebeam ESP-IDF SDK.

Provision ESP32 Gateway with device config data

Earlier in this guide, we had provisioned a new device in the bytebeam cloud console and downloaded a JSON file. We need to flash this file to SPIFFS to connect our device to the cloud.

• First, rename the downloaded file to device_config.json.
• Now navigate to the provisioning, Here you will find the config_data folder inside it.
• Put the device_config.json inside config_data.
• Now open ESP-IDF terminal and navigate to the provisioning project.
• Clear entire flash

idf.py erase-flash

• Once the flash is erased, upload the provisioning example. This will flash the device_config.json file to the device

Idf.py -p PORT flash monitor

Setting up ESPNow nodes and gateway

In this section of the blog, we will walk through the process of setting up ESPNow sensor nodes and a gateway to create a wireless sensor network using ESP32 microcontrollers and SHT35 temperature and humidity sensors. This network will enable us to collect temperature and humidity data from sensor nodes and relay it to the Bytebeam cloud platform. To get started with this

• Clone wsn_bytebeam_idf repository from GitHub using the clone command. This repository contains code for the gateway and sensor node.

git clone https://github.com/vbshightime/wsn_bytebeam_esp_idf.git

Before we start coding, let's wire up the SHT35 sensors to the ESP32 sensor nodes. SHT35 comes with an I2C interface. The connection of ESP32 and SHT35 is shown in the diagram below

Setting up the ESPNow Sensor Nodes

To set up ESP Now Sensor Node. Grab your sensor node setup and go through the following steps.

• Navigate to wsn_bytebeam_idf repository and open the sensor_node folder
• Here you will find the code for the sensor node.
• Sending data from sensor nodes to the gateway involves 3 steps. First is initialising ESP Now. The second is Scanning for available gateway. The third is sending data to the following gateway. Let's look into these steps in detail.
• First, initialise a peer object to store information about available peers or gateways

static esp_now_peer_info_t peer_info;

• Create a function to initialise esp Now

void init_esp_now() {
  esp_err_t err = esp_now_init();
  if (err != ESP_OK) {
    ESP_LOGE("ESP_NOW_INIT", "Failed to initialize ESP-NOW: %s", esp_err_to_str(err));
    return;
  }

  err = esp_now_set_self_role(ESP_NOW_ROLE_SLAVE);
  if (err != ESP_OK) {
    ESP_LOGE("ESP_NOW_SET_SELF_ROLE", "Failed to set self role: %s", esp_err_to_str(err));
    return;
  }

  err = esp_now_register_recv_cb(recv_cb);
  if (err != ESP_OK) {
    ESP_LOGE("ESP_NOW_REGISTER_RECV_CB", "Failed to register receive callback: %s", esp_err_to_str(err));
    return;
  }
}

• Now in the next step create a function to scan for available gateways.

// Scan for gateways in AP mode
void ScanForGateway() {
  err = esp_now_add_peer(&peer_info);
  if (err != ESP_OK) {
    ESP_LOGE("ESP_NOW_ADD_PEER", "Failed to add peer: %s", esp_err_to_str(err));
    return;
  }
  }

• In the next step create a function to send sensor data to the associated gateway.

esp_now_register_send_cb(OnDataSent);

• In this setup register a callback to get the status of the transmitted packet

void OnDataSent(const uint8_t *mac_addr, esp_now_send_status_t status) {
  char macStr[18];
  snprintf(macStr, sizeof(macStr), "%02x:%02x:%02x:%02x:%02x:%02x",
           mac_addr[0], mac_addr[1], mac_addr[2], mac_addr[3], mac_addr[4], mac_addr[5]);
  Serial.print("Last Packet Sent to: "); Serial.println(macStr);
  Serial.print("Last Packet Send Status: "); Serial.println(status == ESP_NOW_SEND_SUCCESS ? "Delivery Success" : "Delivery Fail");
}

• Now flash the code to your sensor node

Setting up the ESPNow Gateway

To set up ESP Now Gateway. Grab your ESP32 and go through the following steps.

• Navigate to wsn_bytebeam_idf repository and open the gateway folder
• Here you will find the code for the gateway.
• Here Gateway performs two tasks. First is receiving data from the sensor node and processing it. Second, is sending data from the sensor node to the Bytebeam cloud
• Receiving data from sensor nodes to the gateway involves 3 steps. The first is setting up the ESP gateway in discoverable mode by configuring it as AP. The second is initialising ESP Now. The third is receiving data from the sensor nodes. Let's look into these steps in detail.
• Firstly, create a function to configure the ESP32 gateway in AP mode to make it discoverable by nearby sensor nodes.

bool configESPNowDeviceAP() {
  WiFi.mode(WIFI_AP_STA);
  DEBUG_PRINTLN("Confuring Device AP");
  String Prefix = AP_MODE_SSID;
  uint8_t mac[6];
  String macStr = getLast3ByteMac(mac,false);
  String ssid = Prefix + macStr;
  DEBUG_PRINTLN("Configuring to:"+ssid);
  bool result = WiFi.softAP(ssid.c_str(), "", ESPNOW_CHANNEL, 0);
  if (!result) {
    DEBUG_PRINTLN("AP Config failed.");
    return false;
  }
  return true;
}

• Now, Initialize ESP Now

bool switchToESPNowGateway()
{
  configESPNowDeviceAP();
  WiFi.disconnect();
  int ESPNow_err = esp_now_init();
  if (ESPNow_err != ESP_OK) {
    DEBUG_PRINTLN("ESPNow Init failed");
    return false;
  } else {
    DEBUG_PRINTLN(" ESPNow Init Success");
  }
  return true;
}

• Next, Create a callback to receive messages for sensor nodes.  

esp_now_register_recv_cb(OnDataRecv);

void OnDataRecv(const uint8_t *mac_addr, const uint8_t *data, int data_len) {
  // Convert the MAC address to a string.
  char mac_str[18];
  snprintf(mac_str, sizeof(mac_str), "%02x:%02x:%02x:%02x:%02x:%02x",
           mac_addr[0], mac_addr[1], mac_addr[2], mac_addr[3], mac_addr[4], mac_addr[5]);

  // Check if the data length is less than the size of the `SensorPayload` structure.
  if (data_len < sizeof(SensorPayload)) {
    ESP_LOGE("Bad data from espnow");
    return;
  }

  // Convert the data to a `SensorPayloadTH` structure.
  SensorPayload *payload = (SensorPayload*)data;

  // Print the temperature and sensor profile.
  ESP_LOGI("Temperature: %.1f", payload->temperature);
  ESP_LOGI("Sensor profile: %d", payload->sensor_profile);

  const char* mac_str_str = mac_str; 
 // Add the sensor payload data to linked list
  addNode(&head, payload, mac_str_str);
}

• The second task of the ESP32 gateway is to process and relay sensor node data to the Bytebeam cloud platform. To do that  first include Bytebeam Arduino header
• Then begin the Bytebeam client
• Next, Create a function to publish sensor data to the Bytebeam cloud

#include "bytebeam_sdk.h"

static bytebeam_client_t bytebeam_client;

static int publish_sht_values(bytebeam_client_t *bytebeam_client)
{   
    while (countNodes(head)>0){
      struct SensorPayload *payload = nullptr;
      struct Node* currentNode = head;
      payload = currentNode->payload;
      struct timeval te;
      long long milliseconds = 0;
      static uint64_t sequence = 0;
      cJSON *device_shadow_json_list = NULL;
      cJSON *device_shadow_json = NULL;
      cJSON *timestamp_json = NULL;
      cJSON *temperature_json = NULL;
      cJSON *humidity_json = NULL;
      char *node_id_json = NULL;
      char *string_json = NULL;
      device_shadow_json_list = cJSON_CreateArray();

      if (device_shadow_json_list == NULL)
      {
        ESP_LOGE(TAG, "Json Init failed.");
        return -1;
      }
      device_shadow_json = cJSON_CreateObject();

      if (device_shadow_json == NULL)
      {
          ESP_LOGE(TAG, "Json add failed.");
          cJSON_Delete(device_shadow_json_list);
          return -1;
      }
      // get current time
      gettimeofday(&te, NULL);
      milliseconds = te.tv_sec * 1000LL + te.tv_usec / 1000;
      timestamp_json = cJSON_CreateNumber(milliseconds);

      if (timestamp_json == NULL)
      {
          ESP_LOGE(TAG, "Json add time stamp failed.");
          cJSON_Delete(device_shadow_json_list);
          return -1;
      }
      cJSON_AddItemToObject(device_shadow_json, "timestamp", timestamp_json);

      node_id_json =  cJSON_CreateString(currentNode->mac_str);

      if (node_id_json == NULL)
      {
          ESP_LOGE(TAG, "Json add temperature failed.");
          cJSON_Delete(device_shadow_json_list);
          return -1;
      }
      
      cJSON_AddItemToObject(device_shadow_json, "node_id", node_id_json);

      temperature_json = cJSON_CreateNumber(payload->temperature);

      {
          ESP_LOGE(TAG, "Json add temperature failed.");
          cJSON_Delete(device_shadow_json_list);
          return -1;
      }
      
      cJSON_AddItemToObject(device_shadow_json, "temperature", temperature_json);

      humidity_json = cJSON_CreateNumber(payload->humidity);

      if (humidity_json == NULL)
      {
          ESP_LOGE(TAG, "Json add humidity failed.");
          cJSON_Delete(device_shadow_json_list);
          return -1;
      }
      
      cJSON_AddItemToObject(device_shadow_json, "humidity", humidity_json);

      cJSON_AddItemToArray(device_shadow_json_list, device_shadow_json);

      string_json = cJSON_Print(device_shadow_json_list);

      if(string_json == NULL)
        {
            ESP_LOGE(TAG, "Json string print failed.");
            cJSON_Delete(device_shadow_json_list);
            return -1;
        }

      ESP_LOGI(TAG, "\nStatus to send:\n%s\n", string_json);

      // publish the json to sht stream
    int ret_val = bytebeam_publish_to_stream(bytebeam_client, sht_stream, string_json);

    cJSON_Delete(device_shadow_json_list);
    cJSON_free(string_json);
    
    }
}

Data Visualization in Bytebeam Cloud

• After a successful connection of ESP32 with Bytebeam Cloud, we can see the recent device shadow and heartbeat on the cloud console.

• Next, Create a dashboard and choose a panel of your choice. We have chosen a line chart for visualisation.
• Go to a line chart. Give a title to your line chart. Select Stream "device_Shadow". Select field as "temperature". Aggregator as "max"

• For Advance settings Click on Advanced. Add node_id in Group by Field and hit Confirm

• Similarly, create a humidity chart as well.

• Create multiple visualizations for your dashboard

Conclusion

In conclusion, we've gone through advanced filtering in line charts. Using wireless sensor network use case, with ESP32 acting as the gateway and sensor node, we've demonstrated how this feature empowers you to harness the full potential of your data. This same scenario can be replicated in edge IoT solutions, wireless sensor networks, or countless other applications.

Bytebeam offers a comprehensive suite of features and tools to enhance your IoT and data management experience. Check out our data Visualization guide for creating interactive dashboards,  OTA guide for remote updates and the Handling Action Guide for remote operations.

I hope you find this guide useful. As we continue to come up with more interesting tutorials and maker content, we invite you to explore Bytebeam further and see how our platform can empower your projects.

Stay tuned!