Arduino JSON for IoT

JSON and XML are the most common data serialization formats. The Internet is a hub of billions of different devices, applications, and services. These devices and applications are built in different programming languages and around diverse platforms. For all these entirely different devices and applications to effectively communicate with each other, data is serialized and de-serialized at the endpoints in a format that is shareable. JSON and XML are the two most important markup languages in the world of the internet that let exchange data across different platforms, devices, and applications in a standardized manner. The devices and applications serialize the data into JSON or XML format before transmitting it over to the internet. The receiving device or application at the other end of the internet de-serializes the JSON/XML data and extracts the required information as per its user code.

All IoT devices are also part of the same internet universe. Most IoT devices use JSON format for communicating data with a server, gateway, or other devices (in the same network). JSON is preferred over XML because it is simpler and less verbose. For IoT devices, even a little overhead can be a great burden for microcontroller-run devices. That is why most IoT devices use JSON to exchange data over the internet. Most IoT platforms and services also essentially use JSON, while they may also be using XML format.

In this article, we will discuss how Arduino and Arduino-compatible microcontrollers serialize and de-serialize JSON data for standard universal communication in the realms of the IoT. Arduino and its compatible boards rely on the ArduinoJSON library to encode and decode JSON data for IoT and the internet.

What is JSON?
JSON stands for JavaScript object notation. It is a lightweight text-based data-interchange format. JSON is widely used in web applications and APIs to exchange data due to its simplicity and its compatibility with several programming languages. It is used for serializing and transmitting structured data over a network connection, like data exchanged between a server and a client. JSON is human-readable as well as easy for machines to parse and generate.

JSON syntax
The syntax of JSON is derived from JavaScript object notation, but it is language-independent. JSON represents data as name/value pairs (similar to JavaScript object properties), arrays, or a combination of both. An object in JSON is enclosed in curly braces { }. Inside the object, data is represented as a collection of key/value pairs. A key/value pair consists of a field name (in double quotes), followed by a colon (:), followed by a value. Commas separate the key/value pairs (,). The values in JSON can be strings (in double quotes), numbers (integer or floating point), boolean values (true or false), arrays, objects, or null. An array in JSON is an ordered list of values and is enclosed in square brackets [ ]. Commas separate the values in the array (,). The values in an array can be objects, numbers, strings, boolean, or even other arrays. Data represented in JSON looks like the following:

{“key1″:”value1”, “key2″:”value2”, “key3″:”value3”, ….}

The following is a valid example of JSON data:

{“temperature”:25.21, “humidity”:59.07, “pressure”:1008.23}

Most of the time, values returned by IoT servers or applications in JSON are other JSON objects or arrays.

Arduino, IoT, and JSON
JSON is a common data-interchange format across the internet. Any IoT device powered by Arduino or a similar microcontroller may be receiving data from a web server, API, or a web service in JSON format only. As JSON is simple and straightforward, even tiny machines like that of microcontrollers are capable of parsing and generating it. The markup language allows the transmission of useful data without any extra overhead. That is why most web applications that power IoT devices exchange data with devices through JSON. The same is the case when devices have to send data to a web server, API or a web service. As microcontrollers can easily encode data into JSON, it is the de facto standard for embedded devices and applications to serialize data (like sensor data) for transmission over the internet. JSON enables IoT devices and IoT platforms and services to exchange data in a minimal footprint.

What is ArduinoJSON?
Arduino and other microcontrollers need to be connected to the internet through Ethernet or WiFi to exchange data over the internet as an IoT device. For Arduino, either the board needs an Ethernet shield, or connect to the internet through a WiFi breakout board, or must have built-in WiFi or Ethernet functionality. On the programming side, apart from Ethernet or WiFi libraries, Arduino and its compatible boards need ArduinoJSON library to encode or decode JSON for effective communication in the IoT landscape.

ArduinoJSON is designed for constrained environments and is optimized for minimal memory usage, making it suitable for microcontrollers with limited RAM. It also supports both static and dynamic memory allocation, giving developers flexibility based on their memory management strategies. The library will also filter JSON data and transform it into different structures as needed by the application; the library uses a DOM-like API, which means it can parse JSON data into a tree structure, making it easy to traverse and extract values. Additionally, it supports static and dynamic memory allocation, giving developers flexibility in memory management strategies.

Installing ArduinoJSON library
ArduinoJSON is not linked to the Arduino libraries by default. So, you need to install the library explicitly in the Arduino IDE. You can download the latest version of the library from here as a ZIP file. Rename the ZIP folder to ArduinoJSON. Open Arduino IDE and navigate to Sketch -> Include Library -> Add .ZIP Library. Select the ArduinoJSON ZIP folder and install it. Alternatively, the library can be found by navigating Tools -> Manage Libraries and searching for ArduinoJSON in the search box. Select ArduinoJSON and click the install button. After installing the library, it can be imported into a sketch using the following statement.

#include <ArduinoJson.h>

ArduinoJSON library classes & functions
ArduinoJSON library includes the following classes:

DynamicJsonDocument
StaticJsonDocument
JsonDocument
JsonPair
JsonVariant
JsonObject
JsonArray
DeserializationError

DynamicJsonDocument
This class represents a JSON document and uses dynamic memory allocation. It’s suitable for situations where the size of the JSON document can vary. The memory for the document is allocated on the heap, making it more flexible but slightly slower than StaticJsonDocument. The class provides the following functions useful in manipulating JSON documents:

clear(): Clears the document and releases all the memory.
capacity(): Returns the current capacity of the document.
shrinkToFit(): Reduces the capacity to match the current usage.
as<T>(): Converts the document to a specific type (like JsonObject or JsonArray).
to<T>(): Creates a copy of the document as a specific type.
operator[]: Provides access to the elements of the JSON object or array.
memoryUsage(): Returns the amount of memory used by the document.

StaticJsonDocument
This class is similar to DynamicJsonDocument but uses static memory allocation. It’s faster and more memory-efficient for smaller, fixed-size JSON documents, as the memory is allocated on the stack. This class is more suitable for environments with limited memory resources. It provides the same functions as the DynamicJsonDocument class.

JsonDocument
This is a base class for DynamicJsonDocument and StaticJsonDocument. It provides the common functionality between these two classes, such as parsing JSON data from a string or stream and serializing the JSON document back to a string or stream. This class provides the following functions:

deserializeJson(): Parses a JSON string or stream into the document.
serializeJson(): Serializes the document into a JSON string or stream.
to<JsonObject>(): Converts the document to JsonObject.
to<JsonArray>(): Converts the document to JsonArray.

JsonPair
This class allows manipulating a key-value pair in a JsonObject. This class is useful when iterating over all elements in a JsonObject. This class provides the following functions:

key(): Returns the key of the key-value pair.
value(): Returns the value of the key-value pair.

JsonVariant
This class allows manipulating a JSON value, which can be any JSON data type: a number, a string, a boolean, an array (JsonArray), an object (JsonObject), or null. It provides functions to convert between types and check the actual type of the stored value. This class provides the following functions:

as<T>(): Converts the value to a specified type.
is<T>(): Checks if the value can be converted to a specified type.
to<T>(): Creates a copy of the value as a specified type.
set(): Assigns a new value.
clear(): Resets the value to null.

JsonObject
This class allows manipulating a JSON object, a collection of key-value pairs. It allows adding, accessing, and removing members. The keys are strings, and the values can be of various types, including nested JsonArray or JsonObject. This class provides the following functions:

createNestedObject(): Creates a nested JsonObject.
createNestedArray(): Creates a nested JsonArray.
containsKey(): Checks if the object contains a specified key.
remove(): Removes the element with the specified key.
clear(): Removes all key-value pairs from the object.
size(): Returns the number of key-value pairs.
begin() / end(): Provide iterators to traverse the object.
operator[]: Accesses or modifies a value associated with a key.

JsonArray
This class allows manipulating a JSON array. It provides functions to add, remove, and access elements within the array. You can iterate over the elements in a JsonArray. This class provides the following functions.

add(): Adds a new element at the end of the array.
remove(): Removes the element at the specified index.
clear(): Removes all elements from the array.
size(): Returns the number of elements in the array.
begin() / end(): Provide iterators to traverse the array.
operator[]: Accesses an element at a specified index.

DeserializationError
When parsing JSON data, this class is used to represent the outcome. It indicates whether the de-serialization was successful or if an error occurred. It provides methods to check the type of error and retrieve an error message. This class provides the following functions.

code(): Returns the error code.
c_str(): Returns a C-style string representation of the error message.

Parsing JSON data with Arduino
Parsing and de-serializing JSON data from a web server, API, or web service involves the following steps:

Include the ArduinoJSON library: First, include the ArduinoJSON library in your Arduino sketch. The library must be installed in the Arduino IDE. The library is included in a sketch using the following statement:

#include <ArduinoJson.h>

Allocate a JSON document: Next, you need to create a JsonDocument object. You can choose between DynamicJsonDocument and StaticJsonDocument as per your memory management strategy. The following statement creates a dynamic JSON document 1024 bytes in size:

DynamicJsonDocument doc(1024);

Parse JSON data: Fetch JSON data into a string and de-serialize it into the JSON document. Following is a valid example of parsing a JSON string.

const char* jsonString = “{\”sensor\”:\”gps\”,\”time\”:1351824120,\”data\”:[48.756080,2.302038]}”;
DeserializationError error = deserializeJson(doc, jsonString);
if (error) {
Serial.print(“deserializeJson() failed: “);
Serial.println(error.c_str());
return;
}

Access specific data: Once the JSON is de-serialized, you can access the data using the JsonDocument. Following is a valid example of accessing specific values from a de-serialized JSON document.

const char* sensor = doc[“sensor”]; // “gps”
long time = doc[“time”]; // 1351824120
double latitude = doc[“data”][0]; // 48.756080
double longitude = doc[“data”][1]; // 2.302038

The nested objects and arrays are also accessed similarly as regular key-value pairs. Remember, you must always check the size of your JSON data and adjust the JsonDocument size accordingly. The required size of the JSON Document can be calculated using ArduinoJson Assistant, which is available on the ArduinoJSON website.

Generating JSON data with Arduino
Generating and serializing JSON data using the ArduinoJSON library involves creating a JsonDocument, populating it with data, and then serializing that data into a JSON string. It involves the following steps:

Include the ArduinoJSON library: First, include the ArduinoJSON library in your Arduino sketch. The library must be installed in the Arduino IDE. The library is included in a sketch using the following statement:

#include <ArduinoJson.h>

Create a JSON document: Next, you need to create a JsonDocument object. You can choose between DynamicJsonDocument and StaticJsonDocument as per your memory management strategy. The following statement creates a dynamic JSON document 1024 bytes in size.

DynamicJsonDocument doc(1024);

Populate the JSON document: Add data to the JsonDocument. The document can consist of values, arrays, or nested objects. The following is a valid example of populating a JSON document:

doc[“sensor”] = “gps”;
doc[“time”] = 1351824120;
JsonArray data = doc.createNestedArray(“data”);
data.add(48.756080);
data.add(2.302038);

Serialize the JSON document: Finally, convert the JsonDocument to a JSON string using serializeJson. You can serialize it directly to a String, a char array, or any output stream (like Serial). The following is a valid example of serializing the JSON document.

String output;
serializeJson(doc, output);

For serializing the document to the Arduino’s Serial port, use the following statement.

serializeJson(doc, Serial);

The JSON document should be large enough to hold your entire JSON object. You can use the ArduinoJson Assistant to calculate the required size. Remember that each addition to the document consumes memory. So, be careful of memory constraints, especially on smaller microcontrollers. The serialized JSON is in a compact format. If you need a human-readable format, you can use the serializeJsonPretty() function instead of the serializeJson().

Conclusion
Whenever an IoT device communicates data with a web server, API, or web service, the data is received in JSON format or needs to be serialized into JSON format. The markup language is a standard for exchanging data over the internet by devices, applications, and platforms. On Arduino and compatible boards, encoding and decoding JSON documents is made possible with the help of the ArduinoJSON library. The library helps decode JSON documents received from a server/API/web service through an HTTP request and extracts required values from the document. The library is also useful in serializing data into JSON for transmission to a web server or API.