Establishing Standards for Embodied AI

Embodied Artificial Intelligence (EAI) involves embedding artificial intelligence into tangible entities, such as robots, equipping them with the capacity to perceive, learn from, and engage dynamically with their surroundings. In this post, we discuss establishing standards for EAI systems to ensure safety, reliability, performance, and usability of EAI systems.

1.  The Need for an EAI Standards Framework

Figure 1: EAI Systems
Credit: Shaoshan Liu

Figure 1 illustrates an EAI robotic system with various critical components, highlighting the complexity and integration required for these systems. This complexity underscores the imperative need for a standards framework to ensure safety, reliability, performance, and usability.

Safety is paramount in EAI systems because they often operate in close proximity to humans, where any malfunction or unintended behavior can result in serious injury or harm. Ensuring safety involves preventing accidents through reliable hardware and software integration, implementing fail-safes and emergency stop mechanisms, and adhering to stringent safety standards.

Reliability of EAI systems concerns the durability of physical components, ensuring they can withstand regular use without frequent breakdowns or failures. It also involves maintaining consistent performance, where the system accurately and efficiently performs tasks across various operating conditions. Software stability is crucial, requiring the embedded AI algorithms and software to operate without crashes, bugs, or significant errors that could disrupt functionality. Additionally, fault tolerance is essential, allowing the system to handle and recover from faults or errors with minimal impact on performance and safety.

Performance standards for EAI systems address the accuracy of tasks performed, such as precise movements and correct responses to inputs. They also cover the speed at which the system can complete tasks, ensuring timely and responsive operation. Efficiency in resource usage, including power consumption and computational resources, is another critical area, ensuring that the system can perform its functions without excessive energy or processing power use.

Usability standards ensure EAI systems are user-friendly, accessible, and effective in their interactions with human users. Ease of use is crucial, ensuring that the system can be operated intuitively by users with varying levels of technical expertise. Accessibility ensures the system is usable by individuals with diverse abilities and disabilities.

Despite the importance of these standards, the EAI industry is still in its infancy, and comprehensive standards frameworks to ensure the safety, reliability, performance, and usability of EAI systems are currently lacking. Developing and implementing these standards is essential to facilitate the widespread adoption and effective deployment of EAI technologies.

2.  Lessons from the Intelligent Electric Vehicles Industry

EAI systems and intelligent electric vehicles (IEVs) share similarities in terms of complexity and safety requirements. Both systems rely on sophisticated sensors, actuators, and control algorithms to perform complex tasks in dynamic environments, often in close proximity to humans. Therefore, we can borrow concepts from the IEV industry to develop a standards framework for EAI systems.

The standards framework for IEVs consists of over 50 standards that constitute a comprehensive range of guidelines and regulations from the components level to the full systems level. At the components level, standards certify the quality and compatibility of individual parts such as batteries, sensors, and electric motors. For example, ISO 6469 and IEC 62133 focus on battery safety, ensuring power sources are robust and reliable under various operating conditions. Similarly, ISO 26262 addresses the functional safety of electrical and electronic systems, which is critical for the safe operation of autonomous and semi-autonomous driving features.

At the full systems level, the integration of these components into a cohesive vehicle necessitates additional standards. For instance, ISO 21434 focuses on cybersecurity for road vehicles, ensuring the complex software systems controlling modern electric vehicles are secure from hacking and other cyber threats. ISO 26262 provides a framework for identifying and mitigating risks associated with electrical and electronic systems in automobiles. These standards ensure all aspects of the vehicle, from individual components to the integrated system, meet stringent safety and performance criteria.

3.  A Standards Framework for EAI Systems

Based on the IEV standards framework, we propose a standards framework for EAI systems, as shown in Table 1. While the IEV standards framework is divided into the components level and the full systems level, the EAI systems standards framework is divided into four levels: the components level, the full systems level, the usage scenarios level, and the swarm intelligence level.

At the components level, standards are required to ensure the safety, reliability and performance of various components.  Only the components that meet these basic requirements shall be integrated into the full EAI systems. Similar IEV standards can be applied to EAI systems as they share similar sensors and even some mechanical components. Some of the standards that can be borrowed from IEV standards include ISO 6469 and IEC 62133, which ensure the quality and safety of batteries, sensors, and electric motors, focusing on component-level standards.

At the systems level, after integrating all components, it is crucial to apply standards to verify the entire system’s safety, reliability, performance, and usability. This process involves generating a comprehensive system score to quantify the overall quality of the system. For example, standards similar to ISO 26262 can be used to assess the functional safety of EAI systems, while standards akin to ISO 21434 can enforce cybersecurity measures. Unlike IEVs, usability is a particularly critical metric for EAI systems due to their frequent interactions with humans. The ISO 9241 series of standards, which provides guidance on human-centered design processes for interactive systems, ensures usability throughout the design and development lifecycle. However, the field of usability standards for EAI systems is currently underdeveloped and requires rapid advancements to meet the growing needs of this technology.

Once all these standards have been established, a system score is necessary to summarize the EAI systems’ safety, reliability, performance, and usability into an easy-to-understand metric. This score would facilitate the comparison of different products, aiding consumers and stakeholders in making informed decisions based on the overall quality of the systems.

Unlike IEVs, EAI systems can be applied to a wide range of usage scenarios. For instance, a humanoid robot can function as a worker in a manufacturing plant or as a home service robot. Therefore, it is crucial to define a set of usage scenarios and evaluate different robots’ performance within these contexts. A viable metric for this evaluation could be the task completion rate of a particular robot in a specific usage scenario. An example in this domain is ISO 18497, which specifies safety and performance requirements for agricultural automated machinery, including robots used in farming operations. This approach ensures robots are assessed based on their effectiveness in real-world applications, providing a comprehensive understanding of their capabilities and limitations.

Table 1: Standards Framework for EAI Systems

Credit: Shaoshan Liu

In addition, different EAI systems can collaborate to complete complex tasks. Swarm intelligence refers to the collective behavior of multiple EAI systems working together to achieve common goals. Inspired by natural systems like ant colonies, bird flocks, and fish schools, swarm intelligence involves individual agents following simple rules and local interactions to create complex, coordinated actions. In embodied AI, swarm intelligence enables robots to collaborate, share information, and adapt to dynamic environments efficiently. This approach enhances the robustness, scalability, and flexibility of robotic systems, allowing them to perform tasks such as search and rescue, environmental monitoring, and collective transportation more effectively than individual robots working alone.

Therefore, a cooperation score standard is essential for evaluating how well multiple EAI systems work together. It measures the effectiveness of their communication, coordination, adaptability, and overall contribution, ensuring the swarm operates efficiently and effectively in achieving collective goals. This standard helps optimize swarm performance and ensures successful collaboration in dynamic environments. However, standards specifically addressing swarm intelligence are currently lacking and require rapid development.

4.  Summary

A standards framework for EAI systems is urgent and imperative because these systems closely interact with humans and operate in complex, dynamic environments. Ensuring safety, reliability, performance, and usability through standardized guidelines is essential to prevent accidents, enhance system effectiveness, and facilitate widespread adoption and trust in these advanced technologies. EAI systems are very similar to IEVs, as both rely on sophisticated sensors, actuators, and control algorithms to perform complex tasks safely and reliably in dynamic environments, often in close proximity to humans. Therefore, we can build upon the foundation of IEV standards to establish standards for EAI systems.

However, EAI systems have unique traits that differentiate them from IEVs. Firstly, EAI systems often interact with humans, making usability a key area for standards development. Secondly, the same EAI systems can be utilized in different scenarios, necessitating scenario-specific performance standards. Thirdly, swarm intelligence is a unique feature of EAI system deployments, making it essential to develop a cooperation score to evaluate how well different EAI systems work together. To conclude, we summarize some existing standards that could potentially be utilized for EAI systems in Figure 2.

Figure 2: Existing Standards that Could be Applied to EAI Systems
Credit: Shaoshan Liu

Shaoshan Liu is currently a member of the ACM U.S. Technology Policy Committee, and a member of the U.S. National Academy of Public Administration’s Technology Leadership Panel Advisory Group. His educational background includes a Ph.D. in Computer Engineering from U.C. Irvine, and a Master’s degree in Public Administration (MPA) from the Harvard Kennedy School.