As sensor technology continues to advance, the potential for sensor fusion to revolutionise a diverse range of industries will only grow. Future developments will bring even more sophisticated algorithms, lower power consumption, and cost-effective solutions to further expand its benefits.
The integration of sensor fusion technology into all industries will be a game-changer and is due to unlock unprecedented levels of efficiency, accuracy, and innovation.
“The art of combining data from an array of sensor technologies to create a unified and enhanced understanding is not just a technological advancement – it’s a transformative approach enabling significant changes that will span everything from industrial and consumer products through to every kind of service,” says Jamie Jeffs, Head of Industrial Instrumentation at 42T.
“The rapid advancements in sensor technology have made electronic sensors more affordable, compact, and versatile, embedding them into our everyday lives. But the real game-changer comes when we combine these sensors, unlocking possibilities that go beyond individual capabilities. This fusion will lead to enhanced performance, cost reductions, greater accuracy, and the creation of entirely new applications.”
What is sensor fusion?
Sensor fusion combines data from multiple sensors to provide a more comprehensive and accurate understanding of an environment or system. This process integrates sensory data from disparate sources, such as cameras, radar, lidar, accelerometers, and gyroscopes, to create a unified picture.
The core idea behind sensor fusion is that each sensor has its strengths and weaknesses, and by fusing their outputs, it is possible to overcome individual limitations and improve the overall system performance.
Sensor fusion plays a crucial role in various applications, including autonomous vehicles, machine vision, medical devices, and smart home systems, significantly enhancing their capabilities and efficiency.
Sensor fusion is found everywhere
Sensor fusion is widely applied across numerous industries. A notable example is in smartphones, where sensor fusion enhances user experience and device functionality.
Smartphones typically integrate accelerometers, gyroscopes, magnetometers, and GPS sensors to provide features such as screen orientation adjustment, augmented reality applications, step counting, and location services. By fusing data from these sensors, smartphones can deliver smooth and precise performance in various tasks, from navigation to fitness tracking.
Paul Bearpark, Head of Electronics & Software at 42T, cites an example: “One of the most common applications of sensor fusion is GPS and INS (Inertial Navigation System) in aeronautical navigation systems. The INS can provide much faster updates to the aircraft control system than GPS but its instruments drift over time. The drift-free position provided by GPS can be used to calibrate or reset the INS. In high dynamic situations the GPS signal can be lost but the INS continues to compute position and angle.”
“In a novel extension to the GPS/INS navigation system, 42 Technology has been contributing to a project integrating a UWB (Ultra Wide Band) sensing system into the GPS/INS to enhance the landing capabilities of an autonomous drone. The UWB system will augment the GPS/INS system to enhance accuracy and provide the drone with commercial jet airliner level of landing reliability but at the fraction of the cost.”
Paul continues, “Another commonly deployed example of sensor fusion is multimode intrusion detection systems which are combinations of both infrared and microwave sensors. By deploying two different types of sensor, they have a high immunity to false alarms because the detectors use different sensing modalities and are therefore susceptible to different false alarm mechanisms that are unlikely to occur at the same time.”
Distributed sensing systems and how they are employed
Distributed sensing systems consist of multiple sensors placed in different locations or devices that communicate and collaborate to achieve a common goal. These systems are widely used in environmental monitoring, industrial automation, and smart cities.
For instance, in environmental monitoring, distributed sensing systems can be deployed to measure parameters like temperature, humidity, air quality, and water levels across a large area. The collected data is then fused and analysed to provide a comprehensive view of environmental conditions which can inform decisions on pollution control, resource management, and disaster response.
In industrial automation, distributed sensing systems enhance manufacturing processes by monitoring equipment status, product quality, and environmental conditions in real-time. Sensors distributed across a factory floor can detect anomalies, predict maintenance needs, and optimize production efficiency. This collaborative approach helps in minimizing downtime, reducing waste, and improving overall productivity.
Smart cities employ distributed sensing systems to manage infrastructure and services such as traffic control, public safety, and energy consumption. For example, traffic sensors placed at intersections can gather data on vehicle flow, which is then fused with information from cameras and GPS devices to optimize traffic light timings, reduce congestion, and enhance road safety.
Often sensing systems can be distributed to provide enhanced performance or even create new applications.
Example: Distributed sensing is employed in many different ways. The Square Kilometre Array (SKA) radio telescope to be constructed over the next few years will employ many highly distributed radio receivers to create the world’s largest radio telescope using interferometry techniques to combine the signals. A similar distributed system combines hundreds or sometimes thousands of seismic sensors to create a large array that is capable of creating a 3-D image of geological formations to identify deposits of coal, oil, groundwater and minerals.
A simpler type of distributed sensing approach is used to regulate our driving speeds on many roads by employing ANPR (automatic number plate recognition cameras) to determine our average speed between them.
Distributed sensing requires communication between the sensors and/or communications with a central processing unit. The communications system can be employed for timing and synchronisation, data transfer, and monitoring and control. The SKA system will be transmitting data at an aggregate average rate of 8 Terabits per second!
Methods of combining sensors
There are several methods for combining sensor data, each with its own advantages and suitable applications. The three main methods are data-level fusion, feature-level fusion, and decision-level fusion.
Data-level fusion
Data-level fusion involves combining raw data from multiple sensors before any processing occurs. This method is beneficial when the sensors provide complementary data that enhances the overall information quality. For instance, in autonomous vehicles, data-level fusion might integrate raw data from radar and lidar to create a detailed and accurate map of the surroundings.
Feature-level fusion
Feature-level fusion entails extracting relevant features from sensor data and then combining these features for further processing. This method is useful when the sensors provide different types of information that need to be interpreted together. In robotics, feature-level fusion might involve combining visual features from a camera with motion features from an accelerometer to improve object recognition and tracking.
Decision-level fusion
Decision-level fusion involves making individual decisions based on data from each sensor and then combining these decisions to form a final output.
The integration of sensors can happen at various levels, including combining raw data directly from the sensors. This often requires high-bandwidth communication, especially in distributed systems where data might need to travel over long distances. For instance, the SKA radio telescope leverages optical fibre communications to transmit large amounts of data and timing information to a central signal processor. Similarly, seismic acquisition systems also demand robust communication channels to handle the transmission of raw data and timing information.
Transmitting features instead of raw data typically requires much lower communication speeds. For example, average speed detectors send timing details along with the vehicle’s registration number.
At their simplest level, sensors might only send out a simple decision or trigger alert. For instance, a PIR sensor used for intrusion detection could send a notification that it has been activated. Pairing it with a second PIR sensor can increase confidence that there’s a real intruder rather than a false alarm, especially if both sensors are triggered.
If decision making can be made at the sensor, this can considerably reduce the communications requirements resulting in lower Capex and Opex costs. Timing and synchronisation can still be important, so communications latency must often be considered.
Managing power consumption of fusing sensors
Managing power consumption is a critical consideration in sensor fusion, especially in battery-operated devices and systems. Several strategies can be employed to optimize power usage while maintaining sensor performance.
One approach is to implement adaptive sampling techniques, where sensors dynamically adjust their sampling rates based on the context and requirements. For example, a wearable health monitor might reduce its sampling rate during periods of inactivity to conserve battery life, while increasing it during physical activity to ensure accurate monitoring.
Another strategy is to use low-power sensor nodes in distributed sensing systems. These nodes can perform local processing and data fusion to reduce the amount of data transmitted to a central unit, thereby saving power. Additionally, techniques such as duty cycling, where sensors alternate between active and sleep modes, can significantly extend battery life.
In energy-efficient sensor fusion algorithms, prioritising the use of sensors with lower power consumption and selectively activating higher-power sensors only when necessary can also help manage power consumption. For instance, a smartphone might rely on its accelerometer for basic motion detection and only activate the GPS sensor for precise location tracking when needed.
Paul Bearpark, Head of Electronics & Software at 42T, says: “Camera traps are battery powered devices that can be deployed remotely for long periods on a relatively small battery. They employ low-power PIR sensors that trigger the camera to wake up and start recording for a pre-defined period greatly reducing the power consumption compared with a permanently active video sensor. Networks of portable low power seismic sensors have been deployed for long periods to wake up communications systems to alert security personnel to intrusion. Often these have been integrated with battery powered camera systems so that the alert is accompanied by real time video of the site providing security personnel with situational awareness. When they are comfortable they no longer need the video feed they can remotely return the camera to its sleep state.”
Reducing sensor system cost
Reducing the cost of sensor systems is essential for widespread adoption and scalability. Sensor fusion can contribute to cost reduction in several ways.
By integrating multiple functions into a single system, sensor fusion can eliminate the need for redundant sensors and reduce hardware complexity. For example, in an automotive system, a single multi-functional sensor unit that combines radar, camera, and ultrasonic capabilities can be more cost-effective than using separate sensors for each function.
Moreover, sensor fusion can improve the performance of lower-cost sensors by compensating for their limitations with data from higher-quality sensors. This approach allows manufacturers to use a mix of inexpensive and premium sensors without compromising overall system accuracy and reliability.
Additionally, advances in sensor fusion algorithms and software can reduce the need for expensive processing hardware. Efficient algorithms that optimize data processing and fusion can be implemented on less powerful and more cost-effective hardware platforms, further reducing system costs.
Paul says, “People often think that adding more sensors means higher costs for the system, but that’s not always true. Sometimes, using a combination of cheaper sensors can be more cost-effective than relying on a single expensive one. For example, a network of microphones could be more effective and less costly for detecting drones than a single radar. But, combining microphones with radar might actually give the best overall performance.”
Example: 42T came up with a really innovative solution for water leak detection. While acoustic technology is commonly used for this, it can be expensive and consume a lot of power due to the need for continuous computation to differentiate between leaks and background noise.
Instead, we used two temperature sensors: one to measure the temperature of the pipe and the other for the ambient temperature. In cooler climates, the temperature of incoming water is usually lower than the ambient air. Even a small leak can keep the pipe temperature lower than the ambient temperature, preventing them from fully converging. This clever approach resulted in a low-cost, very low-power leak detection system.
Enhancing capability of fusing sensors
Fusing sensors can provide capabilities and enable applications that would often not be possible with a single sensor.
In robotics, sensor fusion enhances spatial awareness and navigation capabilities. Robots equipped with multiple sensors, such as cameras, lidar, and sonar, can navigate complex environments, avoid obstacles, and perform tasks with higher precision. This capability is crucial for applications ranging from industrial automation to search and rescue missions.
Paul says, “Returning to the intrusion detection application example mentioned previously, a 360 degree PTZ (pan-tilt-zoom) camera can be enhanced by a network of intrusion sensors. These could be PIR or seismic sensors positioned around the camera.
By setting the camera pre-sets to the positions of the sensors, the camera can automatically pan, tilt and zoom into the area around the sensor when it is triggered. This has the benefits of not requiring an ‘overwatch’ camera, alerting security personnel of an intruder and automatically setting the camera to provide the best view of the intruder.
An enhancement to this would be to add video analytics to automatically confirm the intrusion being a person rather than an animal since the original trigger could have been a false alarm.”
Making sensor fusion work is transformative
By integrating data from various sensors, sensor fusion overcomes the limitations of individual sensors, providing more accurate, reliable, and comprehensive information.
This technology enhances the capabilities of systems across diverse industries whether it be in smart systems, manufacturing, consumer, healthcare, and beyond, leading to improved safety, efficiency, and performance.
Jamie sums up, “The implications and benefits of sensor fusion are vast and transformative across multiple industries. 42T’s ability to identify and implement pragmatic sensing solutions has enabled a number of our clients to establish a more comprehensive understanding of how their processes are performing, leading to opportunities to implement significant improvements in process performance.
The continued innovation and integration of sensor fusion will undoubtedly lead to smarter, safer, and more efficient systems that improve our daily lives and address complex challenges in diverse industries.
From simple to complex systems, sensor fusion has already proven its worth across various fields. Yet, we’re just scratching the surface of what’s possible. The potential for innovation in this space is immense, limited only by our imagination and the ability to develop groundbreaking sensor systems.”
As we continue to explore the limits of sensor technology, the future holds endless possibilities. The next big leap lies in our willingness to experiment and push boundaries, paving the way for novel and exciting applications that could redefine industries and everyday life.