Manufacturers often increase the number of product variants or offer uniquely configured products to serve individual customer needs and create a competitive advantage. This advantage, however, comes with a trade-off: increasing complexity and costs on all steps of the manufacturing process, from engineering to planning and production to service. To eliminate this increased complexity and rise in costs, manufacturers must transform their operations into agile automation that can quickly respond to changes within the planning and production environment.
An example of how this trade-off can be eliminated is a machine learning-based video quality assurance solution deployed at Foxconn’s production lines for HPE systems at the Foxconn site in Kutna Hora, Czech Republic.
The complexity trade-off in IT system manufacturing
The production of IT systems is particularly prone to the complexity trade-off. One reason for this is the number of potential product variants. For example, a specific server model can theoretically be equipped with between two and sixteen memory modules, which in turn can have 16, 32, 64 or 128 gigabytes. Thus, the memory options allow for hundreds of product variants – and these, in turn, must be multiplied by the number of options available to the processors, fans, disk drives, etc. Moreover, the IT industry is known for rapid innovation, with short product refresh cycles and frequent product updates as a result – adding to the variability of the production environment.
Quality assurance of these products not only requires assessment of the number and position of such system components, but also of their correct implementation. For example, a cable must not only be positioned in front of the correct port, but also checked to make sure it was actually plugged in successfully during manufacturing and assembly. Further, other product defects such as scratches on the surface of the server chassis must be accounted for.
Considering the above circumstances, these types of complex IT system quality assurance inspections can take a human several minutes each. For a manufacturer like Foxconn, whose factories produce tens or hundreds of thousands of IT devices every day, this adds a significant amount of cost and time to the manufacturing process.
- Automating quality assurance with machine-learning based video analytics
A solution to this problem is to deploy video analytics for quality assurance to automate the process: cameras capture high-resolution images of products on a conveyor belt and stream them to an embedded or attached IT system where the images are analyzed by a video analytics application using machine-learning (ML) algorithms. Just as humans would, ML compares the image of the actual product with reference images that display accurate and defective implementations. Thus, the machine learns if a cable is properly plugged into a port, if a memory module is properly inserted into its socket, or if there is a scratch on the chassis.
- quality assurance
- video analytics
- ml-based video analytics
- hpe edgeline converged edge systems
- conveyor belt
But also for ML-based video analytics, product variety is a challenge. To teach the analytics application to detect defects in configuration, implementation or any other damage of the product, it could require thousands of reference images that the analytics application can
compare with the product image from the conveyor belt. This comes with two key disadvantages. First, it can take weeks to train the ML algorithms to enable accurate and reliable detection of product defects. Second, this approach is inflexible, requiring a new training cycle for every new configuration, product refresh or update.
Together with Relimetrics, a company specialized in smart quality audits for Industry 4.0, HPE Pointnext implemented a solution to solve these problems. One initial action included the disaggregation of product images. The solution does not store reference images of complete servers, for example, but of components like memory modules in the proper slots, the processor socket with its fan, the hard disk, etc. For each product on the conveyor belt, the Manufacturing Execution System (MES) provides a list of materials to the analytics application, enabling it to assemble the relevant reference image components into a complete reference picture.
This approach has two key effects. First,
the ML algorithms aggregate learning much faster and more efficiently because reference image components are frequently reused. At Foxconn’s Kutna Hora factory, HPE Pointnext was able to train the ML model for new server quality assurance with roughly 1,000 configuration variants in just two days, completely automating the fault detection process. Second, this approach enables increased flexibility by combining image components according to the actual product configuration provided by the bill of materials.
- Edge analytics for real-time quality management
When using ML for managing a production process like quality assurance, another challenge is the amount of data produced by the video cameras. The implementation at Foxconn’s server production in its Kutna Hora factory, for example, employs 10 video cameras for each conveyor belt in order to capture extremely detailed
images of each server and part. These cameras generate 3 gigabytes of image data per hour. Transmitting large amounts of data to ML-based video analytics systems quickly is critical for high-producing factories. It would be impractical to transfer that data via internal or external networks to be processed on remote servers – because the latency would be too high, networks would be overloaded with these data volumes, and production systems would grind to a halt during network outages.
Therefore, HPE Pointnext deployed the
ML-based video analytics solution on HPE Edgeline Converged Edge Systems – rugged, compact systems delivering enterprise-grade IT capabilities at the edge. These systems are designed with manufacturing environments in mind and also integrate operational technology (OT) like data acquisition systems, control systems and industrial networks to enable seamless bi-directional and deterministic communication and control of OT systems like video cameras, production machines or conveyor belts .
With this solution, the constant video camera stream is first pre-processed on HPE Edgeline Converged Edge Systems running in close proximity to the conveyor belt. The solution extracts images of the actual product and next analyzes the data in real-time using ML algorithms to detect defects. Only a subset of the analyzed images are transferred through the network to be archived for traceability and compliance.
To eliminate the complexity and cost trade-offs of production variability, manufacturers must transform their operations from static to agile automation. Key enablers to achieve that include artificial-intelligence methods like machine learning. However, as the example of machine-learning based video quality assurance confirmed, realizing this promise requires smart solutions that take into account the specifics of the individual production process. Moreover, it requires manufacturers to establish a new edge infrastructure to turn video streams into intelligence and action.
According to Gartner, by 2025, as a result of digital business projects, 75 per cent of enterprise-generated data will be created and processed outside the traditional, centralized data center or cloud, up from less than 10 per cent in 2018. Video analytics quality assurance is just one of many examples that show why this prediction is valid.