Artificial Intelligence (AI) is renovating the fast-growing Internet of Things (IoT) by migrating AI innovations, including deep neural networks, Generative AI, and large language models (LLMs) from power-hungry datacenters to the low-power Artificial Intelligence of Things (AIoT). Located at the network’s edge, there are already billions of connected devices today, plus a predicted trillion more connected devices by 2035 (according to Arm, which licenses many of their processors).
The emerging details of this AIoT development period got a boost from ACM Transactions on Sensor Networks, which recently accepted for publication “Artificial Intelligence of Things: A Survey,” a paper authored by Mi Zhang of Ohio State University and collaborators at Michigan State University, the University of Southern California, and the University of California, Los Angeles. The survey is an in-depth reference to the latest AIoT research.
“This survey comprehensively explores all aspects of AIoT, from sensing and computing to communication. It highlights the latest advancements, explains the role of AI in enhancing IoT, details how AI can be integrated, and showcases the new applications this AI-IoT synergy enables. It is a must-read for those who want to work in the AIoT domain,” said Longfei Shangguan, an assistant professor in the Department of Computer Science at the University of Pittsburgh, an AIoT expert who did not contribute to the survey.
The survey addresses the subject of AIoT with AI-empowered sensing modalities including motion, wireless, vision, acoustic, multi-modal, ear-bud, and GenAI-assisted sensing. The computing section covers on-device inference engines, on-device learning, methods of training by partitioning workloads among heterogeneous accelerators, offloading privacy functions, federated learning that distributes workloads while preserving anonymity, integration with LLMs, and AI-empowered agents. Connection technologies discussed include Internet over Wi-Fi and over cellular/mobile networks, visible light communication systems, LoRa (long-range chirp spread-spectrum connections), and wide-area networks.
A sampling of domain-specific AIoTs reviewed in the survey include AIoT systems for healthcare and well-being, for smart speakers, for video streaming, for video analytics, for autonomous driving, for drones, for satellites, for agriculture, for biology, and for artificial reality, virtual reality, and mixed reality.
A device is categorized as belonging to the AIoT if it contains sensors, a micro-computer running AI algorithms with or without an accelerator, and either a direct actuator connection or a communications channel to other nearby AIoTs and/or a cloud aggregator.
Credit: “Artificial Intelligence of Things: A Survey”
AIoTs equipped as AI agents have the ability to combine different sorts of sensor data from separate sources into fused perceptions. AI agents also perform complicated multi-step tasks, make high-level plans, and reason-out complicated decisions. For instance, the Octopus v3 multimodal AI Agent is an AIoT device using the authentication, authorization, and secure storage of tokens. LLMs interpret user instructions from which Octopus identifies and locates useful elements of apps, then plans and executes the user’s tasks by autonomously navigating these multiple apps.
Likewise, the AutoDroid LLM-powered task automation system uses AIoT Agents to execute multiple-step tasks automatically by combining dynamic app analysis with LLM task specification. The two-step process starts with an initial offline stage, where agents catalog how the user-interfaces (UIs) accomplish simulated tasks; during a second execution stage, it defines the order of a set of LLM-specified tasks from its previously acquired UI knowledge, which its agents then perform in sequence to accomplish a complicated task.
Complimenting the 70-page survey is a GitHub repository that archives the 353 papers on AIoT included in the survey, reviews them, summarizes them, and sorts them by application. Zhang also claims to have commitments from his co-authors, as well as outside experts in AIoT, to keep the GitHub repository actively updated with new AIoT results as they are peer-reviewed and published in the future.
“The genius of combining artificial intelligence with connected devices is that it fundamentally reshapes what’s possible in networked systems” said Yaxiong Xie, an assistant professor in the Department of Computer Science and Engineering at the University at Buffalo, NY, who was not involved in the survey. “We’re witnessing a shift from simple data collection to intelligent decision-making right where the information originates. This new architecture doesn’t just save bandwidth; it creates an entirely new class of system that can understand its environment, make real-time decisions, and continuously improve its performance. I believe this fusion of intelligence and connectivity will define the next era of technology—opening doors to applications we haven’t even imagined yet.”
IoT began its migration to AIoT when it was discovered that inference engines could be slimmed down using low-power small-integer neural networks that consume a fraction of the power required by power-hungry floating-point deep neural networks used in datacenters, and thus can be run at the network edge (see the 2018 conference paper “Training and Inference with Integers in Deep Neural Networks”). Since then, researchers also have invented ways to update these tiny neural networks in the field (see “Design Principles for Lifelong Learning AI Accelerators”), greatly expanding their applications.
“As AI gets built into more and more IoT devices, they are addressing more diverse applications—eventually empowering billions of people with the latest breakthroughs brought to them by next-generation AI inventions,” said Zhang. “For instance, today even the latest Gen AI and LLM inventions can be accessible as services provided by AIoT,” as described in detail in the survey.
According to Bo Yuan, an associate professor in the Department of Electrical and Computer Engineering at Rutgers University, “Professor Zhang [et al.’s] work is a comprehensive review, analysis, and summary of the milestones and future projections for the research activities in AIoT. This survey also discusses the new opportunities for AIoT, making it very timely and important.”
AIoT streamlines the collection and categorization of myriad sensor readings, images, videos, audio clips, text files, physiological signals, and environmental measurements, according to Zhang, while IoT began with simple individual sensor monitoring, which passed its raw sensor readings to datacenters. Adding AI to IoTs expanded the landscape of AIoT technological innovations. Said Zhang, “Now I deeply believe that eventually all future IoTs will be AIoTs, since together the two technologies more efficiently process the world’s streams of raw data.”
“In order to interact with the physical world and serve a purpose in any real-world application, any IoT system must have the ability to sense and make decisions. AI represents the most effective and efficient means to enhance and integrate these capabilities into an IoT. Indeed, I believe that all future IoT systems will incorporate AI,” said Xiaoxi He, an assistant professor at the University of Macau. “Zhang [et.al.’s] survey provides a detailed taxonomy of the ways and paradigms with which modern AI can aid IoT systems. More importantly, it highlights how AI has transformed IoT systems, enabling new applications and purposes that were previously unattainable.”
The synergy between AI and IoT in AIoT devices already is fundamentally transforming how users perceive and interact with the world, according to Zhang. “AIoT enhances decision making, facilitates a new level of time efficiency, and improves every aspect of human-machine interactions—such as transforming a simple step-counting IoT into a personal-fitness AIoT, essentially a virtual ‘fitness coach’ which not only makes intelligent recommendations relevant to your physical needs, but enables you to become more aware of your own physical fitness. And as more AI breakthroughs like LLMs and Gen AI are made, not only will portable AIoTs become more useful, but they will also be better able to protect the privacy of their users.”
R. Colin Johnson is a Kyoto Prize Fellow who has worked as a technology journalist for two decades.
Join the Discussion (0)
Become a Member or Sign In to Post a Comment