Getting Started with ESP32 IDF and Bytebeam Part 1 - Data Visualization

Gautam BT

Dec 21, 2022 • 7 min read

Learn about managing IoT devices and visualising device data using Bytebeam cloud

In this multi-part series of tutorials, We will deep dive into the Bytebeam SDK for ESP-IDF. We will get familiar with the concepts of the Bytebeam cloud console and go through different analytics tools provided by Bytebeam. In Part 1 of this tutorial, we will learn about the following:

Getting Started with Bytebeam cloud
Device provisioning on Bytebeam cloud.
Setting up Bytebeam SDK for ESP-IDF.
Data Visualization using ByteBeam cloud.

In this tutorial, We will use ESP32 to compile our example. ESP32 comes with capacitive touch GPIOs which can sense human touch. So, we will read and visualize touch sensor data using bytebeam dashboard.

Hardware Requirements

ESP 32 dev module

Getting Started with ByteBeam cloud

Firstly, you need to create an account on Bytebeam Cloud (cloud.bytebeam.io). In case you already have an account with Bytebeam Cloud then you can proceed further with this guide or else follow Getting started with Bytebeam cloud.

Creating a new project

Login to your Bytebeam account to access the cloud console.
Now create a New Project and give it a name. I have given it the name “esp32Demo”

In the next screen, you will be asked to select the project. Select the newly created project and proceed further

Provisioning a new device

Click on Create Device button to provision a new device to Bytebeam cloud. And hit Submit.

As you hit submit in the prompt. A new JSON file will be downloaded which contains the authentication certificate, broker, device_id, and project id. You need to save this file in the ESP file system. We will talk about this in a later section. The JSON will look like this.

{
 "project_id": "****",
 "broker": "******",
 "port": 8883,
 "device_id": "**",
 "authentication": {
      "ca_certificate":*"*********",
      "device_certificate": "**********",
      "device_private_key": "***********"
      }
}

Now you can find the newly created device in the console. The provisioned device will have the following columns
- id: This column shows the device id of the provisioned device.
- Last Heartbeat: last online status of the device.
- Device Shadow: latest data received from the remote device.
- MetaData: Device Metadata, this gives information about device data like customer id, organization id, and others as desired by the user
- Last Action: This column gives us information about the last action id, last action status, its progress and type.
- Operations: From this column, we can download the device provision JSON file.

Configuring a Stream

Next, we need to configure the stream where we will define our data keys and their data types. For this go to Settings in the top right corner and select Streams. You will find the Table Name, Column Name, and Column Type in Streams. Add your data keys in Column Name and respective data types in Column Type. Here I have added state and touch keys with int8 and int16 as their respective data types and device_shadow as your Table Name. In this example we are using the following data keys:
- sequence: this is a message counter for all the successful publishing
- timestamp: time of message publishing
- touch: ESP32 touch sensor values.

Now we are all set to configure our hardware to push data to the bytebeam cloud. Next, we will set up ESP Bytebeam SDK.

Setting up ByteBeam SDK for ESP-IDF

ByteBeam provides an SDK for ESP IoT Development Framework (IDF) to connect to the ByteBeam cloud platform. This guide assumes that you have a fair understanding of ESP-IDF and that you had already set up ESP-IDF on your machine. In case you haven’t done this you should follow ESP Getting Started Guide before going further.

Clone Bytebeam ESP SDK

Clone esp-bytebeam-sdk using the git clone command.

git clone https://github.com/bytebeamio/esp-bytebeam-sdk.git

The Repository contains the following directories
- components: This directory contains bytebeam_esp_sdk as a component that you can add to your ESP application
- examples: examples directory contains a list of examples, some of the important ones are
  - esp32_touch_example: This is a capacitive touch example which we are demonstrating in this blog
- provisioning: example to push device JSON got from bytebeam cloud to SPIFFS.
Follow Setting up bytebeam sdk guide to get more information about setting up WiFi configuration and partition table.

Provision ESP32 with device config data

Earlier in this guide, we had provisioned a new device in bytebeam cloud console and downloaded a JSON file. We need to flash this file to SPIFFS to connect our device to the cloud.

First, rename the downloaded file to device_config.json.
Now navigate to the provisioning, Here you will find the config_data folder inside it.
Put the device_config.json inside config_data.
Now open ESP-IDF terminal and navigate to the provisioning project.
Clear entire flash

idf.py erase-flash

Once the flash is erased, upload the provisioning example. This will flash the device_config.json file to the device

Idf.py -p PORT flash monitor

Getting ESP32 capacitive touch values

Navigate to examples -> esp32_touch_example -> main -> app_main.c.
include touch sensor driver

#include "driver/touch_pad.h"

Now configure the frequency at which data will be updated to the cloud. We are providing a delay of 1 sec.

#define APP_DELAY_ONE_SEC 1000u
static int config_publish_period = APP_DELAY_ONE_SEC;

Next, declare the bytebeam_client_t variable. It is a structure containing host, port, certificates, project id and device id information.

static bytebeam_client_t bytebeam_client;

ESP32 comes with several touch-sensitive pins we will make use of GPIO 2 in this example. Define touch GPIO.

#define TOUCH_PAD TOUCH_PAD_NUM2

Next, define a macro to specify the touchpad threshold

#define TOUCH_THRESH_NO_USE 0

esp_touch_init() initializes the touchpad and configures GPIO 2 as the touch pin.

static void esp_touch_init(void)
{   
    // initialize the touch pad
    touch_pad_init();

    // set the touch pad reference voltage
    touch_pad_set_voltage(TOUCH_HVOLT_2V7, TOUCH_LVOLT_0V5, TOUCH_HVOLT_ATTEN_1V);
    
    // configure the touch pad
    touch_pad_config(TOUCH_PAD, TOUCH_THRESH_NO_USE);
}

esp_touch_value_get() reads the touchpad to get the touch values

uint16_t esp_touch_value_get(void)
{
    uint16_t touch_value;

    // read the touch pad
    touch_pad_read(TOUCH_PAD, &touch_value);

    return touch_value;
}

inside app_main call example_connect() to configure Wi-Fi or Ethernet, as selected in menuconfig.
then call sync_time_from_ntp() to sync time from NTP. Then, initialize touchpad using esp_touch_init()
In the end, initialize and start the bytebeam client.

void app_main(void)
{
    ESP_ERROR_CHECK(example_connect());
    
    // sync time from the ntp
    sync_time_from_ntp();
    
    // initialize the touch pad
    esp_touch_init();
    
    // initialize the bytebeam client
    bytebeam_init(&bytebeam_client);
    
    // start the bytebeam client
    bytebeam_start(&bytebeam_client);
    
    // start the main application
    app_start(&bytebeam_client);
}

In publish_esp_touch_values() configure your JSON payload to send to Bytebeam cloud

cJSON *device_shadow_json_list = NULL;
cJSON *device_shadow_json = NULL;
cJSON *touch_json = NULL;
device_shadow_json_list = cJSON_CreateArray();
device_shadow_json = cJSON_CreateObject();
uint16_t touch_value = esp_touch_value_get();
// get the esp touch values
touch_json = cJSON_CreateNumber(touch_value);
cJSON_AddItemToObject(device_shadow_json, "touch", touch_json);
cJSON_AddItemToArray(device_shadow_json_list, device_shadow_json);

Inside app_start, call publish_esp_touch_values() to send the payload with a frequency as specified in macro APP_DELAY_ONE_SEC

static void app_start(bytebeam_client_t *bytebeam_client)
{
    int ret_val = 0;

    while (1)
    {
        // publish esp touch values
        ret_val = publish_esp_touch_values(bytebeam_client);

        if (ret_val != 0)
        {
            ESP_LOGE(TAG, "Failed to publish esp touch values.");
        }

        vTaskDelay(config_publish_period / portTICK_PERIOD_MS);
    }
}

Now flash and upload the touch example

Idf.py -p PORT flash monitor

After flashing ESP32 with a touch example. Go to bytebeam cloud console
In the Device Management tab, you can see the updated Device Shadow and Last Heartbeat.

Creating a Dashboard

Now let’s create a dashboard to visualise our data. Click on Dashboards in the bytebeam console. And then click on Create Dashboard.

Add a title to your dashboard like we have given a title ESP32touchDashboard. and leave other columns as defaults.

Now Navigate to the Dashboards tab. And click on the dashboard title.

Click on the Panel button to add visualizations.

Select from different panels available. Like we are choosing Line Chart.

Give a name to your panel in the Title. select table (device_shadow in our case) from the Table drop-down. select a column (touch in our case) from the Column dropdown. and select max from the third drop-down. now your graph will look like this.

Like this, you can create multiple visualizations at a user-defined refresh period for your dashboard. We are getting values every second from the device and we are updating the dashboard every 5s. So even sensitive information like touch can be plotted in a graph.

Conclusion

I hope you find this tutorial useful. We will come up with other interesting features offered by ByteBeam like OTA, remote debugging, actions and notifications, and more. In Part 2 of this multi-part series, we will be exploring the remote OTA option provided by ByteBeam. Stay in touch to get information on many other interesting IoT projects.