Smart City - 4D Digital Twin
This document consist description of AM-X platform implementation and how it enables building a 4D Digital Twin of operational environments.
Purpose of the contract is to give clear understanding of the AM-X platform implementation and benefits for clients ecosystems.
AM-X is an IOT/EDGE-AI solution that has been developed for efficient data annotation, AI training and system deployment. AM-X is hardware independent, modular part of client’s data ecosystem and enables building a 4D Digital Twin of real world operational environment.
With EDGE-processing, AM-X is able to process large sets of data in real-time and on-site - without constant mobile network connection. AM-X solves the bottle-neck problem of mobile networks by providing only the relevant information for client’s operations. AM-X is hardware independent and can be cost-efficiently adjusted to any new hardware, which makes it adaptable for rapid development of sensor technology.
Relevant information can provide added value such as EDGE-anonymization, obstacle detection, 4D digital twin, asset management, predictive maintenance and improvement of logistics operations for client’s ecosystem. In addition, the information and sensor fusion can be used for robotization of on-site operations.
In this paper, we provide example use cases and describe benefits of AM-X. In addition, information on implementation process and cost of implementation of AM-X is described in this document.
“Recon AI” - Recon AI Oy, Business ID 2827294-8 ;
“AM-X” - Recon AI platform that provides position information of components (object of interest) from sensor data. AM-X platform is adapted for needs of different industries.
“Relevant information” - Information that the client specifies to be relevant for their application.
“Object of interest” - Object that sensors are receiving information from and client specifies as object of interest.
“Measurement accuracy” - Accuracy with which the device is measuring dimensions, position or distances of objects of interest.
“Measurement interval” - Interval between which objects are measured.
“Position database” - Database where objects position information is saved to.
“Output information” - Information that AM-X provides for the client. Usually object of interest related to a structural drawing, point-cloud information of position and orientation of the component and information on environmental conditions.
“4D Digital Twin” - Digital model of client’s operational environment consisting dimensional and time information. Model usually includes other IOT measurement information such as temperature, pressure, structural drawing with metadata, depending on client’s operations.
System infrastructure description
IOT Sensors - Sensors that are used in client’s ecosystem. These sensors can be any sensors providing relevant information from operational environment.
EDGE AI Node (part of AM-X) - First calculation node after the sensor. One node can be used for analyzing several different IOT Sensors data and pre-processing of data is done on this node. Node analyses relevant data from sensors and provides a) loop-back to on-site robotic or assisting information systems and b) Ecosystem relevant information to Ecosystem node. In addition, node handles machinery specific sensor fusion operations.
Ecosystem Node (part of AM-X) - Second calculation node after the EDGE AI node can be used for post-analyzing relevant information provided by EDGE AI nodes and and provides a) loop-back to on-site fleet-control operations and b) History relevant information to databases storing historically relevant information. In addition, node handles ecosystem specific data fusion operations.
Cloud Database - Cloud database that is used for storing relevant data and providing this data to any 3rd party operational systems wanted. Possible feedback loop from 3rd party systems
3rd party services - Cloud database that is used for storing relevant data and providing this data to any 3rd party operational systems if wanted.
AM-X Security Overview
All parts of the system handle only required data without unwanted personal information. Access to data is limited according to use cases.
Save the valuable data without unwanted data. Processing data on EDGE prepares it before saving to meet, for example, General Data Protection Regulation (GDPR) requirements and privacy legislation compliance. AM-X can anonymize data already on-site. For instance, AM-X can blur faces from video on EDGE. As a result, we can make sure that personal information is not saved to databases if it is not needed.
Access control. AM-X provides data to through APIs with authentication and reveals only the needed information. For example, AM-X can provide HTTP REST API access point to selected data and to selected user. API provides a limited view to data and requires user authentication before the access.
EDGE AI node enrich and cleans the data: valuable AI detections are added and unwanted data is removed. As a result, we do not need to save unwanted data to cloud. Connection between EDGE AI Node and Ecosystem node is either local or secure connection over non-private network (SSH, VPN).
Ecosystem node can be accessed through SSH or VPN which provide encryption and authentication. Primary this is for the database connection.
3rd party systems can access cloud databases through APIs which provide limited
Detection of objects both for the tracking and the digital twin will be implemented using AM-X platform. AM-X platform is based on neural network algorithms, enabling object detection and localization on edge devices on-site without excessive network load or latency.
Neural networks enable transferring the computationally heavy part of the analysis to the off-site a priori training process which, given sufficient data, enables computationally lightweight analysis on an edge device. The data requirements are further mitigated by various augmentation techniques as well as utilizing synthetic data.
Three cornerstones for digitizing Smart Cities
1. Data Anonymity
By providing data anonymization already on EDGE AI Node, raw sensor information can be provided to markets and more machine vision solutions can be implemented to ecosystem. AM-X Platform can provide pre anonymized data which makes it possible to provide needed raw-data samples openly for the markets and is in compliance on privacy and data storing legistelation.
2. Economical scaling
By moving from ERP centric software ecosystems to OpenData based ecosystems will provide better possibilities to access data for new market operators. With high-quality and open relevant data available in markets without or with low cost structure, market operators are able to provide better services to the field and so forth improve productivity. Such services can be implementing all the way from sensor fusion in edge nodes to big data analytics based on open data information.
3. 4D Digital Twin
AM-X can recognize location of components from city structures. This information can be used for building 4D Digital Twin of the whole city. By applying sensor fusion and using already existing and new sensors in the ecosystem, location of each component can be measured in short time intervals, which will provide time dimension to already existing or new 3D Virtual Models. 4D modelling provides possibility for usage of AR-systems for real-time virtual tour of whole city and same information can be used as part of building information modelling, moving from preventive maintenance model to predictive maintenance models, providing asset management information, quality-, fleet-, logistic control, and robotic systems.
AM-X Implementation description
Phase 1 - 3D Digital Twin. The client provides a high fidelity measurement of the environment. Based on this, Recon AI will build a framework for the synthetic data and implement a measurement simulation algorithm.
Phase 2 - Detection of immovable objects. Recon AI trains and implements an object detection and localization system for the immovable objects.
Phase 3 - Tracking algorithm. Based on the reference of the aforementioned immovable objects and 3D environment, Recon AI develops and implements a tracking algorithm for the measurement vehicle.
Phase 4 - Detection of movable objects. Recon AI trains and implements an object detection and localization system for the movable objects, and implements it to the clients edge device.
Phase 5 - 4D Digital Twin. Recon AI finalizes the implementation of the 4D digital twin, that is continuously updated with respect to the movable objects.
We estimate the first implementation circle to take 8-14 months. Training new object of interest to AM-X platform is estimated to take three weeks on average.
An optimised neural network makes it possible for AM-X to analyse large data sets in real-time. Real-time analysis makes it possible to run AM-X without large data storages.
After training the neural network on cloud, the optimised neural network makes it possible for analyse data on-site with low calculation power requirements. These features makes EDGE processing possible and solves problems caused by mobile network bandwidth restrictions.
AM-X is hardware independent solution that can be trained for any existing and new hardware in client’s ecosystems. Using data from existing and new sensors, benefits of sensor fusion can be maximised.
True to the AM-X ability of processing data on EDGE. AM-X can anonymize data already on-site. On-site anonymization decreases cyber security requirements for the rest of the information ecosystem.
Due to optimised calculation requirement of neural network, AM-X decreases energy consumption of processing and utilises unused processors in EDGE/IOT devices. Decreased power consumption and utilisation of processors in IOT-devices smaller ecological footprint can be achieved.
Recon AI is continuously working on optimising the training pipeline of new components to new sensor data. Optimised training pipeline makes it possible for our clients to apply AM-X on scale that benefits their ecosystem in full.
Transportation and automotive industry is making huge efforts for making their machinery safer and autonomous - AM-X can track the motion of vehicles and estimate routes in real-time, which makes it possible for using the data for obstacle detection purposes.
4D DIGITAL TWIN
Building 4D (real-time) digital twin of city is one of the key elements for making cities true smart cities. AM-X can provide real-time location information of different components of interests. By applying sensor fusion the platform provides 4D information for building information modelling.
AUTOMATED ASSET MANAGEMENT
By relating real-time location information of components of interest to structural drawings such as CAD, AM-X can provide up-to date information of your assets in buildings and infrastructure. This information can be used as baseline for designing operations and making 5D BIM modelling reality.
Predictive maintenance makes it possible for our clients to allocate their resources during maintenance operations. By training AM-X to recognise faulty component and applying measurement abilities of the platform, operators can move from preemptive maintenance models to predictive maintenance models.
During all operations inefficient logistics is one of the major cost-drivers. By applying route tracking abilities of AM-X platform, all on-site logistic operations can be surveilled and optimised.
AUTOMATION & ROBOTICS
Automated solutions often requires applying huge datasets. Analysing these datasets with traditional methods requires huge calculation power. AM-X real-time, on-site processing makes it possible for training AM-X for guiding robotic systems and machines autonomously on-site with low requirements for processors in use.