Real Case: How to Test AI Technology for Auto Generation of Video Preview?
The specifics of video content and the scarcity of available digital environments affect the performance of the technologies delivered for video optimization and engagement. In this article, we are going to share our experience of testing the technology for automatic generation of the video preview. Besides, the technology is based on AI (Artificial Intelligence) and ML (Machine Learning). Hold on your hat, we are starting!
Technology under test
Our QA team verified the video preview technology developed for automatic generation of the trailer instead of static video thumbnails. The technology processes and integrates certain parameters from different metrics to select different video layers and combine them into preview. The duration of preview can be different depending on settings – from 2 up to 10 seconds.
The platform also includes a Control Panel for video content optimization and video analytics – CTR (Click-Through-Rate) and statistics on video engagement.
Details of QA process
At the development stage, our QA team detected several technical issues. They triggered the implementation of new features and redesign of already realized functionality. During testing, our QA team founded the following issues:
- load and performance bottlenecks (mobile);
- configuration difficulties;
- GUI (Graphical User Interface) problems;
- issues with video slider.
During testing the platform on mobile devices, our QA team detected poor performance and low loading speed of video preview on mobile devices, especially the ones with low capacities. For example, preview non-reproducibility and its freezing on mobile platforms, poor quality of displayed trailers, long loading of the configuration file, etc. During mobile testing, we use Android devices with OS versions upper 6.x and iOS devices (iPhone, iPad) with iOS version upper 10.x.
A video slider, added to the Main page of the website, increased the loading time for the whole web page as it was uploading simultaneously with other page elements. In case of poor Internet connection, server issues, or slow webpage loading, advertisements were not displayed. Our team verified the preview platform on Windows 7/10 (Chrome, Firefox, IE, Edge), and Mac OS X (Safari).
AI and ML Testing: data verification
Our QA team verified the data workflow accepted and generated by the AI technology on the basis of the provided information. The QA Engineers sent the data (clicks and views) to be analyzed by AI. After the analysis, AI returned the correspondent result, which depended on the initial data. The aim of testing was to verify whether the AI returned valid results. We used Postman for sending requests to AI. Testing was performed using a Windows console and special jar file prepared by the development team.
During ML testing, our QA team prepared data in JSON format. Analyzing the provided data, the system studied to distinguish the files and provided us with its own variant. We checked the generated file and edited it when necessary. Basing on the edits, ML technology calibrated the data.
Customization Testing
The preview platform required the customization for every publisher. During testing of the customized version, our QA team has faced several challenges individual for every publisher. For example, the type of player used by publisher affected the operability of the video preview platform. In case of JW player used, our QA team gathered separate analytics using Postman.
Testing results
During QA activities, we reported more than 5500 bugs. The number includes issues detected during web and mobile testing. The average number of bugs for a large publisher from 90 – 110 up was reduced to 20 – 30 issues. Do you remember bugs with system capacities? Performance of the platform was enhanced in 5 times.
P.S. The platform is still being updated and new versions are released. We continue the cooperation with the Client by providing software testing services on verification of customized versions of the video preview platform.
Learn more from QATestLab
Related Posts:
About Article Author
view more articles
No Comments Yet!
You can be the one to start a conversation.