Empowering robots with environmental perception and mobility makes them safer and more mobile. Combining artificial intelligence (AI) technology will make them smarter and more practical, and make their future development extremely imaginative. This paper will explore the development of robotic perception and AI technology, as well as related solutions introduced by many manufacturers.
Sensors enable the robot to sense the world
Robotic perception technology enables robots to "see" and "hear" the world. Through AI technology, robots can respond to perceived environments. Robotic perception is crucial for robots to make decisions, plans, and operations in real-world environments. Relevant applications involve sensory data and artificial intelligence/machine learning (AI/ML) technologies. Examples of such applications include obstacle detection, object detection and identification, 3D environment representation, gesture and speech recognition, scenario understanding, road detection, vehicle detection, pedestrian detection, target tracking, human detection, environmental change detection, activity recognition, semantic place classification, object modeling, etc.
Today, most robotic perception systems use machine learning (ML) techniques, from the classical method to deep learning methods. Data from sensors are key elements of robotic perception, regardless of the ML method considered. Data can come from a single sensor or sensors, usually mounted on a robot, but can also come from infrastructure or another robot (for example, cameras mounted on nearby flying drones).
Multiple sensor technologies, including visual (ToF), radar wireless links, complete positioning in the form of an inertial measurement unit (IMU), and gyroscopes and all their associated applications, software and firmware, are required for safe and comprehensive navigation control of mobile robots.
The information provided by robotic sensors can be used to infer information about the world, the robot, and its location in the world. A sensor can provide simple information (e.g., a simple collision sensor on a vacuum robot) or rich, highly complex data (e.g., an RGB-D camera, which provides color images and estimates of the depth of each pixel). Ideally, sensing processes can be modeled using the laws of physics (for example, how light reflects from the surface of an object) or sometimes using simple geometry (for example, stereoscopic vision relies on the use of simple triangulation measurements to determine the distance to an object).
Provide a complete solution for robot development
The perception capabilities of robots are primarily applied to localization and navigation, and the sensors used vary depending on the environment. For example, in indoor settings, most indoor robots encounter mostly regular and flat surfaces on the ground. However, for outdoor robots, the terrain is often irregular. Additionally, outdoor robots must also deal with weather conditions and variations in light intensity and spectrum. Therefore, the required sensors will also differ.
3D time-of-flight (ToF) technology is required in order for robots to “see” things. 3D ToF is a scanner-less LIDAR (light detection and ranging) technology that captures the depth of the scenario (usually at short distances) by emitting a nanosecond-level high-power optical pulse.
ADI has invested years in 3D ToF technology to deliver industry-leading products and solutions that directly implement and upgrade advanced ToF systems and camera capabilities, including high-resolution CMOS imaging chips (1 Megapixel), in-depth computing and processing, laser drives, power management, and development tools and software/firmware to help fast-track ToF solutions.
ADI's 3D ToF products are fairly complete, including ADSD3100, a 1 MP CMOS that supports ToF backside Illumination sensors, ADDI9036, a CCD TOF signal processor with programmable timing and V-driver. In addition, ADI provides a 3D ToF development platform, a modular ToF solution based on the industry standard 96Boards platform, which measures X, Y, and Z axis data for objects. ADI’s prototyping 3D ToF platform can be used for software and algorithm development, which can be used for hardware design to accelerate product design assessment and implementation in industry and automotive sectors
ADI also launched AD-FXTOF1-EBZ, a 3D ToF development kit, which is a depth sensing solution well suited for developing 3D computer vision systems. Using VGA CCD, it supports capturing 640×480 depth mapping scenarios with 30 frames per second, providing a four-fold higher resolution than many other TOF systems on the market. ADI’s another AD-3D SMARTCAM1-PRZ is a 2D and 3D machine visual solution with AI edge processing capabilities that can be used to implement advanced machine visual applications in different industrial segments, including logistics, robotics, agriculture, and people activity monitoring.
Collaborative robots to coexist and work with people
In the past, robots in factories used to operate independently, and workers had to maintain a safe distance from them to avoid collisions and injuries. However, with the rapid development of robot perception and AI technologies, robots have become increasingly intelligent. Intelligent machines and systems have been able to operate autonomously, including areas of work that previously relied on manpower. However, even today, when robotics are highly developed, the assembly and inspection processes still rely on manpower, requiring the use of collaborative robots to coexist and work with people.
Murata has a variety of product lines for the robotic design domain, including a variety of sensors, such as gyroscopes, accelerometers, vibration sensors, temperature sensors, piezoelectric film sensors that can provide a variety of parts necessary for robotic design.
Taking Murata's vibration sensor as an example, it can be used for predictive maintenance of machinery and equipment, and can detect minor differences in the vibration of rotating equipment by using the wide detection frequency band characteristic of ceramic sensors. Monitoring these changes in vibration data allows early detection of equipment anomalies for predictive maintenance. Vibration sensors support a wide frequency range and Z-direction sensing, with driving circuits, differential output to reduce noise, and temperature sensing, with a size of only 5.0×5.0×t3.5 mm and seven terminals structurally, packaged in an SMD, integrating an acceleration sensor and thermistor.
Murata's UWB (Ultra Wide Band) connection module can also be applied to robotic products. UWB technology can be used for secure and precise distance measurement, indoor location tracking and access control applications. There are two models: Type 2BP and Type 2AB.
Type 2BP uses NXP Trimension SR150 chipsets, supports UWB channels 5 and 9, has an SPI interface and support for 3 antennas (3D AoA or 2D AoA support), has an embedded reference clock and sleep clock, the integrated BPF (band pass filter) can be used for regulatory certification, uses a resin mold combined with conformal shield package, and meets RoHS compliant。
Type 2AB uses a Quorvo DW3110/3120 chipset with an integrated Nordic IC (nRF52840) host-less module and BLE feature to wake up the UWB and update firmware. It supports USB, UART, SPI, I2C interfaces, integrates 3-axis sensors to save battery power, uses CR2032 batteries with embedded reference clocks, is packaged with a resin mold, and meets RoHS compliant.
A complete solution for industrial autonomous mobile robots
Traditional industrial robots were mostly confined to fixed positions for their operations. However, with the development of intelligent robot perception and AI technologies, robots have gained the ability to autonomously move. This advancement has expanded the application of robots beyond the industrial sector and into the realms of commercial and household use.
The robotics industry is a rapidly rising industry with enormous advantages, but there are also many potential traps that must be avoided in order to successfully harness this rapidly changing and largely unexplored field. The electronic parts to be used are quite diverse, and Rohm has introduced a variety of related solutions for robotic applications.
Like LiDAR and SLAM technology, it enables autonomous mobile robots (AMRs) and drones to navigate effectively in dynamic industrial environments. Therefore, the correct selection of high-power laser diodes is critical to the accurate performance of the LiDAR/SLAM system.
Rohm high-power laser diode solutions for AMR and robotics meet modern LiDAR/SLAM specifications and limitations. These solutions include RLD90QZW5905 nm, 25W visible pulsed laser diodes with a launch width of 70 µm (equivalent LD emission width of 200 µm).
In power applications, Rohm innovates and leads competitors in releasing breakthrough products and solutions, from pioneering SiC power modules and discrete devices to the next generation of GaN and proprietary efficient power technologies
In addition, Rohm has a variety of features that can be applied to the robotics field, including the KX13x accelerometer. Rohm's KX132-1211 and KX134-1211 triaxial accelerometers are triaxial 16-bit accelerators that support up to ±16 g (KX132)/±64 g (KX134) and integrate a user configurable 3-stage ADP (Advanced Data Path) with a low-pass filter, low- /high-pass filter, and RMS calculation engine. Other features include 2×2×0.9 mm LGA12 pin package, broad temperature range (−40°C to +105°C), 0.5 µA standby current, and 0.53-239 µA operating current.
Rohm’s optical sensor lineup is quite abundant, including high sensitivity, high reliability phototransistors, infrared light emitting diodes, and transmission/reflective type photointerrupters. There are also ambient light sensors and proximity sensors RPR-0521RS, as well as transmission type photointerrupters, reflective type photosensors, infrared LED, phototransistors, and ambient light sensors IC.
In addition to standard-type motor drivers, Rohm also provides application-specific motor drivers with excellent reliability and efficiency across a range of voltage, current, pin configuration and package types, including protection circuits to further improve reliability and performance.
Robotics supports rapid industrial automation development
Industrial automation relies on sensors, remote control, and robotics. Combining AI and IoT technologies will be a key driving force for industrial automation, enabling continuous monitoring and improvement of efficiency and productivity in the connected factories. Industry 4.0 envisions factories capable of mass-producing more personalized and differentiated products, with the ability to quickly change production processes. Connectivity is an indispensable part of achieving this goal, and key application requirements include Mesh networking, Industry-grade security standards, Enhanced range, High performance data processing, Bluetooth Low Energy, Thread and Zigbee, and other communication protocols.
In the application of Industry 4.0, sensors play a crucial role in acquiring system information. The more information we have about the systems and processes, the higher their operational efficiency and competitiveness. Today, miniaturized, cost-effective wireless sensors are helping manufacturers adapt to the new era of Industrial IoT. These sensor-based systems contribute to making informed real-time decisions and optimizing the entire manufacturing supply chain. Sensors using technologies such as Bluetooth and other mesh networking techniques can be installed anywhere due to their small size and low power requirements.
Furthermore, accurate positioning using the world' s most popular Bluetooth wireless technology is the current new technology. Bluetooth direction-finding is based on two key concepts: the angle of arrival (AoA) and the angle of departure (AoD). Using the angular phase shift occurring when the antenna receives (AoA) or transmit (AoD) RF signals, it is possible to determine the phase shift data by using an array of antennas on either side of the communication link and compute the location accordingly. Nordic’s nRF5340, nRF52833, nRF52820, and nRF52811 SoC are ideal for supporting Bluetooth direction-finding.
Solutions to meet the needs of autonomous mobile robots
Machine vision is the key solution so that the machine can see, and its solution is image sensors. Having most image sensor products from VGA to 45MP, onsemi provides suitable sensors for machine vision applications.
Consider the AR0234CS image sensor, a 1/2.6-inch, 2 Mp CMOS digital image sensor with an active pixel array of 1920(H) x 1200(V). The new innovative global shutter pixel design is designed to optimize accurately and rapidly capture mobile scenarios with a total resolution of 120 frames per second, and the sensor generates clear, low-noise images in both low-light and bright scenarios.
Autonomous Mobile Robots (AMRs) are a type of mobile robot that can autonomously navigate in both structured and unstructured environments without continuous human guidance or intervention. They are equipped with sensors and onboard computers that enable them to perceive the surroundings and make decisions based on that information. AMRs can perform a wide range of tasks such as logistics, inventory management, quality control, or be involved in medical applications.
AMR can perform quality control checks autonomously to reduce the need for manual intervention. They can use sensors such as cameras, laser radars, and ultrasonic sensors to detect defects or anomalies in the product. onsemi provides a range of imaging sensors, analog front-end (AFE) ICs, and power management ICs that allow AMRs to capture high-quality images and perform accurate measurements.
AMR can also be equipped with sensors to monitor environmental conditions, such as temperature, humidity, and air quality. They can collect this data independently and transmit it to a central database for analysis. onsemi provides a range of sensors, such as temperature sensors, humidity sensors, and gas sensors, that allow AMR to accurately and reliably monitor ambient conditions.
AMR is also increasingly used for various tasks in the healthcare industry, such as delivering medical supplies, transporting patients, and assisting in surgery. These robots depend on a variety of sensors, actuators, and other electronic parts for normal operation, and onsemi is one of the leading suppliers of such parts in the AMR industry.
Conclusion
The robot’s intelligent perception enables the robot to safely move and perform comprehensive navigation control, requiring the deployment of multiple sensor technologies, combined with AI algorithms and all related applications, software, and firmware to enable the robot to achieve real intelligence and work with humans. The robotic application solutions of various manufacturers described herein can only be briefly described due to limited space. To learn more about the features of these products, please contact Arrow Electronics for more details.
