Beat the edge device data computing challenges – 1
Overview
To manage sensor data for various devices in an embedded system, such as a smart gateway, data must be first processed, transferred, or stored. Advancements in networking technologies, like 5G wireless, have solved data transfer challenges whenever connectivity becomes available. But what about when there is poor or no connectivity? How should data computing continue?
A data orchestrator is responsible for keeping track of the embedded system’s data and managing the flow of information between deployed devices. The orchestrator decides how to take inaccessible and inconsistent data and make it organized and usable. In addition, effective data computing orchestration requires that devices are capable of continuously streaming from sensors, processing, and unification for the entire data captured.
The orchestrator dynamically assigns edge data computing tasks to each device, and those assignments can change over time. There are edge systems that may benefit from a powerful central device to maximize data computing.
An embedded edge data platform provides the ability for each device to dynamically capture, query, analyze and store data. It is also capable of continuing, with no interruption, to perform data computing when a new device or sensor is added to or removed from the system. In addition, when an orchestrator starts a data collection campaign, the live queries deployed to various devices for analysis can be adjusted and updated. At the end of a data campaign, the results stored data on the devices can be exported and queries removed.
Data orchestration workflow and edge computing
Building high-performance devices that stream and process real-time data is not trivial. Whether you are building or upgrading applications to become compatible with the era of IoT, taking on machine learning (ML) tasks, or doing anything else related to artificial intelligence (AI), the data must be managed carefully. Otherwise, data will be fragmented, difficult to correlate, and cause performance bottlenecks.
Therefore, architects and developers seek possible software solutions to coordinate and manage sensors connected to multiple devices. They also prefer devices to be able to share a portion of the data with each other.
The main requirements for such an embedded system include streamlining and optimizing the real-time data with the philosophy of first processing input and then storing what is important. These devices typically perform various intelligent tasks, and automating the orchestration of data management across all of these devices would be extremely cost-effective, reliable, and efficient.
The key strategy for edge data computing
Enabling embedded systems applications to benefit from streaming, processing, and sharing real-time data with each other, or a central device, is one of the main characteristics of ITTIA DB. With ITTIA DB, manufacturers can choose what data to share between devices with high granularity. Devices can register any tables, live sensors, or real-time query results for other devices to consume.
For example, suppose an i.MX8 device advertises the result of its processing assignment, stored in table-A, another i.MX 8 device advertises device parameters in table-B, and a third i.MX 8 device advertises its raw sensors as data stream-C. Finally, the central S32G device can choose to receive any or all of the advertisements provided by these devices.
In the processing assignment, the database software on the STM32U5 device streams real-time data that is processed, filtered, and aggregated with certain defined criteria. As a result, only refined information is replicated between devices. For example, the device can downsample from 1000 data points per second to replicate only one sample per second. This journey of processing and storing can continue on the second, third, and all other devices inside the embedded system.
An embeddable HTAP database optimized for IoT data
ITTIA DB is an HTAP database for edge computing. What is HTAP? Hybrid Transaction/Analytical Processing (HTAP) is an emerging architecture (already available on the cloud) that breaks the wall between transaction processing and analytics. It enables more informed and real-time decision-making for embedded systems. HTAP empowers devices embedded into the system to configure multiple tasks that they all need to process and produce results for the central unit device which monitors the workflow process for each device.
As a result, each device is capable of performing analytics and data storage on its own. HTAP is also an ideal choice for embedded AI data computing.
An HTAP database, like ITTIA DB, offers each individual device to scale and benefit from the infrastructure that is empowered to receive information from all device data sources, including events, alarms, failures, etc. Data for various purposes, including self-health monitoring, self-decision-making, and self-predictive maintenance via real-time data management, is now possible.
Artificial intelligence and edge data computing
As the central unit continuously has access to the most real-time updated data for the entire embedded system, it can be the right candidate for AI integration and involvement with consolidation, governance, and maintenance, and can help to automate many of the tasks originating from different devices.
This central unit can also store data, identify patterns and trends within the entire system, and uncover insights for various purposes, including self-training. For example, the combination of data and AI applications may identify trends and errors that can be used to optimize data campaigns for various tasks, including autonomous-oriented tasks.Stay tuned, as we are working on Beat the edge device data computing challenges – 2.
