LiteRT

LiteRT (formerly TensorFlow Lite) is Google's next-generation, high-performance runtime for deploying ML and GenAI models directly on edge devices.

LiteRT (Lite Runtime) is the core on-device framework from Google AI Edge: it's the evolution of TensorFlow Lite, trusted by 100K+ applications on billions of devices. This runtime delivers high-performance ML and Generative AI deployment across Android, iOS, Web, and IoT platforms. We leverage purpose-specific accelerators (GPUs and NPUs) to maximize efficiency, achieving up to 25x faster inference and 5x power reduction compared to CPU processing. The latest 2.x release introduces the CompiledModel API for state-of-the-art performance, featuring MLDrift for best-in-class GPU acceleration and unified NPU support (co-developed with MediaTek and Qualcomm). LiteRT streamlines the workflow: convert models from PyTorch, JAX, or TensorFlow, optimize them, and deploy them with simplified hardware acceleration.

https://ai.google.dev/edge/litert