How to make HTTP requests using Arduino for the IoT

One essential step in developing Internet of Things (IoT) applications is setting up communication between the IoT devices and a web server or services.

IoT devices communicate with a web server, service, or an API through HTTP requests. An HTTP request is a message sent by a client to a server in the context of the World Wide Web. This enables smart devices to act as a web client, exchanging data and information with servers via the Internet. Many IoT projects are built using Arduino and Arduino-compatible boards. Arduino is currently the most popular microcontroller platform for prototyping typical embedded applications, including the IoT.

In this article, we’ll discuss how Arduino and compatible boards are configured and programmed to “talk” over the Internet. Communication can occur, thanks to HTTP requests.

What is an HTTP request?
An HTTP request is a message sent by a client to a server in the context of the World Wide Web. It’s the standard way web clients (web browsers and other web-connected entities, such as IoT devices) request data from servers or submit data to servers.

These requests are the foundation of data communication on the web, and they operate as part of the Hypertext Transfer Protocol (HTTP) — an application layer protocol in the Internet protocol suite.

An HTTP request includes the following parts.

1. Start line: contains three elements: the HTTP method, request target, and HTTP version. The HTTP method indicates the action to be performed on the resource like GET, POST, PUT DELETE, etc. The request target is a URL or the path to the resource on the server. The HTTP version specifies the HTTP protocol version, such as HTTP/1.1.

2. Headers: contain metadata about the request, such as the host, user agent, content type, and other information about the client or the data being sent.

3. Empty line: the end of the header section and the beginning of the optional (HTTP Request) body is indicated by an empty line.

4. Body: is not present in every request (like in most GET requests), but when it is, it contains the data to be sent with the request. This is typical in POST requests where form data or file uploads are sent to the server.

Whenever you browse a webpage, fill out a form, or call an API from a web application, the client sends an HTTP request. After receiving the request, the server processes it and replies with an HTTP response that includes the requested data, the request status, and sometimes a body containing additional requested data or the operation’s outcomes.

HTTP methods
HTTP method indicates the action to be performed on the resource. Several HTTP methods exist, including GET, POST, PUT, DELETE, HEAD, CONNECT, PATCH, TRACE, OPTIONS, LINK, UNLINK, etc.

The GET method requests data from a resource.
The POST method submits data to be processed to a specified resource.
The PUT method updates a specified resource with the supplied data.
The DELETE method removes or deletes a specified resource.

GET and POST are the most common HTTP methods used for the World Wide Web and IoT. Contrary to its definition, the GET method can get and send data to a web server.

Arduino and HTTP requests
Arduino boards must connect through Ethernet or WiFi to make HTTP requests and communicate with a web server. An Ethernet shield is required to set up communication between Arduino and the Internet over Ethernet. Arduino’s WiFi shield is necessary to allow Arduino to access a WiFi network.

Some breakout boards let Arduino connect with WiFi networks. Several Arduino boards also now come with built-in WiFi functionality, including Arduino Uno WiFi Rev2, Arduino Nano 33 IoT, Arduino MKR WiFi 1010, Arduino MKR1000 WiFi, Arduino Portenta H7, Arduino Yun, Arduino Yun Mini, and Arduino Yun Rev2.

ESP8266 and ESP32 are alternative Arduino-compatible microcontroller boards with built-in WiFi and Bluetooth functions. If an Arduino board does not have built-in WiFi functionality, it requires an Ethernet or WiFi shield to connect to the Internet.

How Arduino communicates
When an IoT device built using Arduino or any microcontroller communicates with a web server, service, or API, it operates as a web client. The web server, service, and API are resources identified by a URL. Communication takes place through HTTP requests to the URL of the specified resource. This may also contain a query string containing the variables or data to be communicated to the specified resource. Then, the specified resource sends an HTTP response. The IoT device (Arduino or any microcontroller) receives and analyzes the HTTP response.

The Arduino libraries
There are several libraries available for sending HTTP requests from Arduino. The libraries simplify the process, especially when using networking modules like the ESP8266 or ESP32, which have built-in WiFi capabilities or Ethernet shields.

Commonly used Arduino libraries for making HTTP requests are as follows.

Ethernet library: for Arduino boards that connect to the Internet using an Ethernet shield. It lets you perform all networking tasks over Ethernet.

WiFi.h: for Arduino boards with native WiFi support, such as Arduino Nano 33 IoT. It’s used for managing WiFi connections and sending HTTP requests.

WiFiNINA library: designed for boards with the u-blox NINA WiFi module, such as the Arduino MKR WiFi 1010, Arduino MKR VIDOR 4000, and Arduino UNO WiFi Rev.2.

ESP8266WiFi library: is for the ESP8266 module and is often used to manage WiFi connections and perform networking tasks, including sending HTTP requests.

ESP32 WiFi library: similar to ESP8266WiFi but designed for the ESP32 module. It allows you to connect to a WiFi network and send HTTP requests.

ESP8266HTTPClient library: for ESP8266, providing additional functionality specifically for creating HTTP requests and handling responses.

HTTPClient library: for ESP32, providing additional functionality specifically for creating HTTP requests and handling responses.

Ethernet library
The Ethernet library enables network connection with LAN or the Internet using the Arduino Ethernet shield. The library provides both client and server functionalities. It also connects to a local DHCP network and resolves DNS. The library is compatible with all architectures of the Arduino platform.

This library can be used with Arduino Ethernet Shield, Arduino Ethernet Shield 2, Leonardo Ethernet, and other W5100/W5200/W5500-based devices. It can configure a board as a server accepting incoming connections or a client making outgoing ones. It supports up to eight or four concurrent connections. The Ethernet shield interfaces with Arduino via the SPI bus.

The library is imported in a sketch using these statements:
#include <SPI.h>
#include <Ethernet.h>

The library provides six classes:

1. Ethernet class
2. IPAdress class
3. Server class
4. Client class
5. EthernetUDP class
6. UDP class

To make HTTP requests using Arduino, methods from the client class are used in the sketch below. The Ethernet client is initialized using the EthernetClient() method, creating a client that connects to a specified Internet IP address and port as defined in the client.connect() function.

It has the following prototype.
EthernetClient()

The method requires no parameters. A valid example of initializing the Ethernet client is as follows.
EthernetClient client;

The client class of the Ethernet library provides the following methods…

client.connect(): connects to a specified IP address and port. It also supports DNS lookups when using a domain name. The value returned by the method indicates the success or failure of a connection.

It has this syntax:
client.connect()
client.connect(ip, port)
client.connect(URL, port)

It takes an IP and a port, or a URL and a port, or it may require no argument. IP is the IP address the client will connect to, represented by an array of 4 bytes. Port is the port that the client will connect to.

For HTTP requests, the port is typically 80. URL is the domain name the client will connect to. The function returns 1,-1,-2,-3 or-4, where 1 indicates SUCCESS, -1 TIMED_OUT, -2 INVALID_SERVER, -3 TRUNCATED, and -4 INVALID_RESPONSE.

client.connected(): detects if the Ethernet client is connected. A client is considered connected even if the connection has been closed, but there is still unread data.

It has this syntax:
client.connected()

The method does not take arguments and returns “true” if connected; otherwise, it returns “false.”
client.localPort(): returns the local port number the client is connected to.

It has this syntax:
client.localPort

client.remoteIP(): returns the IP address of the client.

It has this syntax:
client.remoteIP()

client.remotePort(): returns the port of the host that sent the current incoming packet.

It has this syntax:
client.remotePort()

client.setConnectionTimeout(): sets the timeout for client.connect() and client.stop(). The initial value is 1000 ms, which can be lowered to make the program more responsive.

It has this syntax:
client.setConnectionTimeout(milliseconds)

client.write(): writes data to the server the client is connected to. This data is sent as a byte or series of bytes.

It has this syntax:
client.write(val)
client.write(buf, len)

The method takes either a byte value or a buffer containing an array of bytes as an argument.

print(): prints data to the server that a client is connected to. It prints numbers as a sequence of digits — each an ASCII character.

It has this syntax:
client.print(data)
client.print(data, BASE)

The method takes data to be printed as an argument. It also allows specifying the base of the numbers indicated as DEC for decimals, OCT for octal, and HEX for hexadecimal.

client.println(): prints data followed by a carriage return and new line to the server the client is connected to. It also prints numbers as a sequence of digits — each as an ASCII character.

It has this syntax:
client.println()
client.println(data)
client.print(data, BASE)

client.available(): returns the number of bytes available for reading from the connected server.

It has this syntax:
client.available()

client.read(): reads the next byte received from the server the client is connected to.

It has this syntax:
client.read()

client.flush(): halts everything until all outgoing characters in the buffer have been sent.

It has this syntax:
client.flush()

client.stop(): disconnects from the server.

It has this syntax:
client.stop()

Making GET HTTP requests using the Ethernet library
To send an HTTP GET request from Arduino using the Ethernet library, you’ll need Arduino with an Ethernet shield or built-in Ethernet capabilities, such as Arduino Portenta H7 or ESP8266/ESP32-based boards.

Components

Arduino board
Ethernet shield
Ethernet cable
A server to receive the GET request

For demonstration, we’ll send an HTTP GET request to httpbin.org. To do so, attach the Ethernet shield to the Arduino board and connect the shield to a router using the Ethernet cable. Next, connect Arduino to your computer and upload the following sketch.

Remember to replace the MAC array with the MAC address of your Ethernet shield, which is typically printed on a sticker on the shield. The server array should contain the domain name of the server you’re making the request to. Upload the sketch to Arduino and observe the output on Arduino IDE’s Serial Monitor.

Making POST HTTP requests using the Ethernet library
To send an HTTP POST request from Arduino using the Ethernet library, you’ll need Arduino with an Ethernet shield or built-in Ethernet capabilities.

Components

Arduino board
Ethernet Shield
Ethernet cable
A server to receive the POST request

For demonstration, we’ll send an HTTP POST request to httpbin.org. Attach the Ethernet shield to the Arduino board and connect the shield to a router using the Ethernet cable. Next, connect Arduino to your computer and upload the following sketch.

In the postData variable, put the data you want to send in application/x-www-form-urlencoded format, which is the standard format for POST data in web forms. Each key-value pair is separated by an ampersand (&), and each key is separated from its value by an equal sign (=).

Upload the sketch to Arduino and observe the output on Arduino IDE’s Serial Monitor. When this sketch is run, it will attempt to post the data to httpbin.org/post, which echoes the posted data in JSON format.

It should be noted that sending HTTP requests using plain Arduino and Ethernet shield is suitable for basic applications, but for more secure communication over the Internet, consider using HTTPS. It’s not automatically supported on Arduino but can be used with additional libraries and hardware, such as the Arduino MKR1000 or ESP8266/ESP32.