Internet of Things (IoT): Most Challenging Testing Types

by Yulia Lomanova | April 21, 2021 8:23 am

Recent reports state that many people are still “scared” of the Internet of Things. Some of them fear being locked out of their smart home, predict that self-driving cars would cause fatalities and wouldn’t trust IoT devices to administer their medication.

Indeed, glitches in the IoT are dangerous and would bring about serious consequences. To prevent them, IoT application testing is indispensable. In the article, we present the basics and taxonomy of IoT apps, and discuss particular testing aspects, such as the most challenging testing types.

What is Internet of Things? 

The Internet of Things is defined as the connectivity of virtual and physical – objects and devices – that opens up new dimensions such as smart cities, smart homes, etc. Equipped with mini-computers, smart objects and locations can take in information about their environment and communicate with the Internet and other smart things.

Structurally, IoT applications are composed of several layers. Let’s explain what we mean, using the example of a smart LED light bulb.

• Layer 1: Physical Entity.

The physical element, a LED light bulb, in this case, supplies the direct, physical benefit to the user. 

• Layer 2: Sensor/Actuator. 

The physical entity is equipped with a mini-computer that includes sensor technology and actuating elements.  The sensor integrated into the light bulb continuously measures whether people are present in the room, while the actuator turns the light on automatically when human presence is detected.

• Layer 3: Connectivity.

The sensor technology and the actuator can be connected through an embedded radio module; the light bulb can also transmit its status to the owner anywhere in the world.

• Layer 4: Analytics. 

Sensor data is collected, stored, checked and classified. As for the light bulb, the operating hours are recorded and motion patterns are discerned.

• Layer 5: Digital Service. 

Finally, all the layers mentioned are packaged in a suitable form – for instance, as a mobile application. 

Taxonomy of IoT applications

The principal areas of application of the Internet of Things are focused on healthcare, smart cities, the environment, commercial, industrial and infrastructural fields. They are represented schematically in the following infographic:

Most Challenging Testing Types in IoT

There are several aspects that characterize IoT applications. First of all, they involve a combination of hardware, sensors, connectors, gateways and application software in a single system. Additionally, they provide real-time stream analytics and complex event processing, as well as complex data support. Obviously, it makes testing of IoT applications quite a challenging task. Let’s take a closer look at some of the required testing types:

Interoperability testing

Interoperability testing verifies whether the software is compatible with diverse platforms and provides cross-use functionality. 

Given the immaturity of IoT technology, its lack of interoperability brings about numerous challenges. 

Thus, integration issues might occur with devices made by different manufacturers, purchased at different times or managed by different operating systems. What’s more, we have to deal with different connectors, frameworks and standards for communication protocol. Not to mention the overall lack of programmability. 

Adding to the existing challenges, companies like Google and Apple are creating a vertical stack of IoT products instead of horizontal infrastructure to differentiate their brands, thus making the Internet of Things more fragmented.

Nevertheless, proper interoperability testing is crucial to ensure the seamless function of the IoT system. It covers the following scenarios:

• Testing systems made by different manufacturers;
• Testing the entire system and subsystems;
• Testing IoT devices with WLAN and LTE compliance.

There are several distinctive characteristics of interoperability testing. To start with, these are functional tests meaning they should reflect the user behavior. Secondly, test boundaries are defined by the equipment under test (EUT) and qualified equipment (QE). EUT and QE must be from different vendors and product lines.

[1]

Performance testing

Performance testing ensures that IoT devices can work seamlessly without any interruptions while continuously generating a huge amount of data. It is a complex process that covers network communication, internal computation, timing analysis, load testing, as well as data volume, velocity, variety, accuracy and scalability.

Testing the system stability, sensitivity to network connectivity issues, devices, operating systems and high loads is crucial as these factors affect user experience.  

IoT performance testing differs from the traditional approach. It involves the following:

1. Performance across multiple components and endpoints, such as mobile devices, drug dispensers, cameras, sensors;
2. The request-response time between the IoT device and the server; 
3. IoT protocols, such as AllJoyn, CoAP, IoTivity, MQTT, XMPP, Z-Wave, Zigbee;
4. Gateways, routers;
5. On-premise and cloud servers;
6. Applications running in different browsers/operating systems;
7. REST API services; 
8. User interface performance, network performance during high load, performance for high data volume.

The important thing to consider is choosing the right testing tool. It needs to support the appropriate protocol on which the IoT system is built, handle test data management, provide integration with various modules and support network simulation scenarios.

Given the fact that IoT applications are often being developed for a specific device, the product team should provide QA engineers with the specific native devices to make sure no bugs are missed. One should bear in mind that testing on similar devices or emulators doesn’t guarantee the same results on the native device.

Security testing

Security remains the top priority in the IoT testing process as all the devices have their own IP addresses and can transfer data over the network. The spread of the IoT has been a boon for hackers who can easily target these devices. Hence, it is crucial to test all the devices to eliminate vulnerabilities. IoT security testing focuses on data protection, device identity and authentication, data encryption/decryption and data storage in the cloud.

There are three testing strategies implemented to check the validation of IoT security requirements:

1. Security Functional testing covers compliance with the functional specifications to ensure that the security functions are implemented correctly.
2. Vulnerability testing makes use of security patterns to analyze risk-based vulnerabilities and identify possible threats.
3. Security robustness testing aims to examine invalid messages to deal with unpredicted behavior of the system in case security attacks occur.

How QA Addresses Principal Challenges in IoT

QATestLab IoT testing services

QATestLab has developed a comprehensive QA strategy to handle unique requirements and challenges associated with testing IoT applications[2], using a combination of real devices, tools and frameworks. 

Get comprehensive reports on what should be improved for stellar quality, enhance product robustness, reliability and security, improve quality monitoring and automate testing processes. 

Related Posts:

• No Related Posts

Source URL: https://blog.qatestlab.com/2021/04/21/iot-most-challenging-testing-types/

---