Computing Without Looking Away: Embedded AI Architectures for Always-On Smart AR Glasses

Smart AR glasses such as SPECS create a new design point for embedded AI. They are embodied, see-through computers: they must sense continuously, perceive, render, and interact with the physical world in real time, while operating within strict limits on power, thermal envelope, latency, memory bandwidth, and privacy. They are coupled to the user’s perception, attention, movement, and social presence. At the compute-system level, the central challenge is to sustain useful, context-aware perception and spatial interaction within a live human situation. A practical solution must move beyond peak throughput and exploit sparsity, temporal redundancy, event-driven execution, data locality, and co-design across models, compilers, runtimes, and silicon. Architectural choices will also shape what kind of computing smart AR glasses become. On-device perception can improve social connection, supporting accessibility, translation, memory recall, and contextual assistance without forcing users to look down into a private screen. But it can also enable concealed mediation, emotional analysis, manipulation, and the erosion of trust and privacy. This ambiguity makes restraint a key system requirement: what is computed locally, what is shared, when the system intervenes, and when it remains silent. Drawing on our experience designing AI acceleration for smart glasses, this talk will examine how low-power local inference, sparse and event-driven architectures, temporal locality, runtime scheduling, and model/compiler/hardware co-design can enable AI systems that are always available, context-aware, and designed for presence rather than distraction.