Application protocols for the Internet of Things

The Internet of Things is growing rapidly. Millions, if not billions, of new devices will be connected worldwide in the coming years. This confronts the Internet with new challenges to master. Due to its architecture and technologies, the World Wide Web is well equipped for growth and scaling. But what about the Internet of Things?

The World Wide Web moves countless types of data and information: Websites, pictures, movies, audio and video streams, applications and much more. We are on the demand side, where we use browsers, smartphones, or music devices to download content from providers online. Many specialized application protocols are in use. Probably the best known of is HTTP, the Hypertext Transfer Protocol. In a long-term development evolution, it has been adapted to the needs and is still undergoing adjustments and optimizations, for example in the form of HTTP/2. In the Internet of Things, a vast amount of data and information is also moved, although the characteristics of the data differ considerably. Therefore, it is obvious that other technologies are used for small and very small intelligent devices.

The data characteristics in the IoT are more complex

One application area could be temperature and humidity measurements in your own home. In a simple case, small sensors can measure values and send them via the local Internet connection (WLAN) to a backend service in the cloud. The sensors do not have to create complex data objects; in principle, the raw data (temperature in degrees Celsius and humidity in percent), which can be enriched with structures or metadata (e. g. time of day) – if the device has this information – is sufficient. This is an example of pure one-way communication: device towards cloud. The user then has the possibility to view and evaluate his measurement data via a website offered by his provider.

A more complex case arises if the backend service is to issue commands from the cloud in the direction of the device, e. g. to start a fan if the temperature is too high. Complexity in a different direction arises when smart devices want to communicate with each other. Then the fan can -in principle- request the temperature directly from the sensor, or both are connected to a common message bus. How can they exchange data, especially if they are not made by the same manufacturer?

In addition to questions about technical basics (accessibility, addressing, trust settings between devices, IPv4/IPv6, network routers and firewalls), there are also challenges at protocol and application layer level: How does a device recognize valid requests, how does it block invalid requests? How often does a device listen to requests and how do device and cloud synchronize in terms of request frequency and timing? How many resources does a device have for answering requests?

Smart devices of the Internet of Things are often referred to as “constrained devices”. This means that the small devices have very little resources to do their work, compared to conventional work equipment such as smartphones, tablets or notebooks. While 64-bit multi-core processors in the gigahertz range are used almost universally in the larger device classes and are equipped with several gigabytes of memory, the smart devices segment is rather contemplative.

In this case, a 32-bit single-core processor and 1-2 megabytes of memory is already considered decent. The reason for this is the same as in electricity consumption and price: If a smart heating valve is to have a sales price of e. g. 20 €, the costs for the processor must only be in the really low-digit euro range. And the battery should keep the device alive as long as possible. Ideally, devices generate their own electricity through energy harvesting. The processor and the application it is running should be as economical as possible in all respects, including communication in networks.

Web Protocol for the Internet of Things

Now a web protocol like HTTP can also be used on small devices. In the Arduino boards area, for example, there are many examples of smart sensors that have their own web server. The user can direct his browser to the IP address of the device and receive data directly in the form of a web page. Why don’t all small devices automatically become “web services”?
It is also worth taking a closer look at the way HTTP works. The protocol is used to interact with web resources (e. g. web pages or forms) on a server. Several methods are available for this purpose. Most inquiries are made to request data (“GET”), with other inquiries data can be transferred, e. g. when an order form has to be filled out in the online shop (method “POST”). In principle, these methods can also be used for interaction with devices: Either you want to have data from the device (“GET /temperature”), or you want to control things (“POST /fan/control”). However, there are several reasons why HTTP is not well suited to interacting with constrained devices.

HTTP: Not necessarily suitable for small devices

First, HTTP is a chatty protocol from the point of view of resource restriction. For example, meta information such as the accepted formats or the desired language are transmitted to the server in plain text form. The header of an HTTP request can already be large, and the header of an HTTP response can easily contain several hundred bytes. For the query of a web page with several megabytes of size this is not bad, but for the query of a temperature value in the range of approx. 5 bytes it is significant whether the transmission protocol is slim or rather broad.
Furthermore, the core of the Web area does not assume that a resource (such as a Web page) cannot be delivered in short sections. Although HTTP does provide identification fields for this, providers try to set up as much performance on the server side as is required by customers in the network: A whole industry sector takes care of the fast and decentralized delivery of web content from caching farms.

Small devices have to do other things besides the execution of network protocols. In the Smart-Home example, the temperature of corresponding building blocks must also be read and converted. And when the user presses a button on the device, an answer should be recognizable within a certain time frame, not only after 5 seconds, for example because the device was “busy” with a web query. Protocols for constrained devices must take this into account, even though a device can provide an answer, but only in a few seconds (after more important things have been processed).
There are also technical limitations in the network area. HTTP is based on TCP, the data stream-based transmission method in the TCP/IP stack. Among other things, TCP requires that data packets are equipped with counters in order to be able to correct the order of incoming packets in case of doubt. TCP devices need buffers to store and manage packets between them.

Many small devices therefore use UDP, which delivers datagrams individually and does not guarantee delivery or the actual order. This makes the implementation leaner and more resource-efficient. Unfortunately, HTTP does not pay attention to UDP, newer protocols like QUIC rely on UDP, but are not yet so widespread.

The Constrained Application Protocol (CoAP)

CoAP, the Constrained Application Protocol, jumps into this gap. This is a specialized transfer protocol for limited devices and restricted networks, e. g. with low transmission rates. It is primarily used as a protocol for machine-to-machine communication, and not for interaction at a user interface. CoAP is specified by the Internet Engineering Task Force (IETF) in RFC 7252 (and others).

CoAP vs HTTP: What are the common features, what are the differences?

CoAP has been created close to HTTP in several aspects. It is basically a request/response protocol, i. e. a (device) client makes a request to a (device) server and receives a response. It integrates the URIs known from the web world to name resources. It also adopts some of the query methods (such as “GET”,”PUT”,”POST”, etc.) and defines response codes similar to those of HTTP (e. g. 4.04 “Not found” if a resource could not be found). Web developers who have grown up with HTTP can quickly find their way around CoAP.

Differences result mainly in the amount of data used. In contrast to the plain text format of HTTP, especially the header format, CoAP code words are packed in bytes or individual bits of a binary representation. There are messages that require a response (“Confirmable”), and messages that do not need a response (“Non-Confirmable”). This means that the information about the reliability of a transmission is lifted to the application layer and can be distinguished by the application in individual cases. For example, if a temperature sensor transmits the temperature 1x per minute, a failure of a single message can be merged, it could be mapped as a non-confirmable. If a smart door lock is supposed to lock the door, the message behind it is surely to be classified as “Confirmable”.

Through several of these measures, CoAP achieves a compactness that would hardly be possible with HTTP. Thus, the data communication between devices can be packaged by CoAP into useful control messages with a few bytes.

The asynchronicity of message exchange via UDP also gives CoAP a device the possibility of responding to a request in a resource-saving and time-delayed manner, as well as of transmitting a response in several small (partly “smallest”) blocks. The latter is particularly useful for lossy networks with small packet sizes such as those in the radio range.
Another interesting feature of CoAP is the built-in resource discovery feature. A device can use this mechanism to provide information about the data endpoints it manages. In the example of a Smart-Home weather sensor, such a resource discovery request then returns that there is the current temperature below the endpoint “/temp”, the humidity below “/hum” and the wind direction below “/wdir”. The data format of this discovery response is described in the separate RFC 6690 (“CoRE Link Format”).

Trying CoAP

How can this applicaton protocol be tested? In most cases, it is not exactly known which devices use CoAP as a protocol. This is not a big deal for the first steps in development, since CoAP can also be used by cloud services. And there are test accounts for this.

An easy way is to install an add-on in the Firefox browser. The “Copper” plugin (short CU,[cu]) is an extension that reacts to the protocol identifier coap: // in the browser’s address line and then opens a window with which CoAP messages can be exchanged and thus the protocol can also be tried out. A test access is possible via the URL coap: //coap. me/. In addition, several demo resources of a service can be queried and discovered via Resource Discovery.

If you want to dive deeper into the protocol, you can install the network and protocol analysis tool Wireshark[ws]. Wireshark is able to listen to data on the network interfaces of a PC and break down the protocol details. This makes it easy to visualize protocol settings and access methods (e. g. in conjunction with Copper).
To develop applications that talk CoAP to each other, it is worth taking a look at the[co] website. It compiles the implementation services around CoAP in an appealing way. More than 30 implementations for different languages and runtime environments are listed, so that developers can find something suitable for their own experiments.

Conclusion

In order for the Internet of Things to grow within restricted devices and networks, applications on devices must conserve resources. Lean communication solutions are in demand, and in addition to MQTT as a message-based protocol, developers are starting to use CoAP with its request/response protocol based on the Web or HTTP. Both have specific purposes and strengths, and we look forward to seeing how these application protocols will be used in the future of IoT.