Our Latent AI Efficient Inference Platform (LEIP) is an SDK for deep learning training and inference. LEIP provides APIs to train, quantize, and deploy edge AI models from many deep learning frameworks. It then generates optimized runtime engines deployable in embedded devices and infrastructure edge devices.
Latent AI’s LEIP SDK enables you to deploy deep learning for a wide range of applications, such as natural language processing, audio, and image/video analysis. The LEIP SDK consists of modular software toolsets:
- LEIP Compress – state-of-the-art quantization optimizer that supports a broad set of quantization algorithms for compressing neural networks.
- LEIP Compile – automated compiler framework that optimizes neural network performance for hardware processor targets.
- LEIP Zoo – a broad collection of neural network models as examples of how to use the LEIP SDK.
The LEIP SDK is available as a Docker container so that you can use it in your own developer environment. Your data does not leave the premises at all, and the SDK is built to integrate with standard workflow.
The LEIP SDK is a unique toolset that is model and hardware agnostic. Any AI model can be compressed, including convolutional neural networks (e.g., ResNet, Inception, VGG, MobileNet) or recurrent neural networks (e.g., RNN, LSTM). You can also compile and generate inference runtime engines for any hardware (e.g., x86, ARM).
With the flexibility and powerful optimizations afforded by the LEIP SDK, you can now quickly explore the design space to reduce power, size, and cost (e.g., bill of materials). With the reduced bit-precision and efficient memory use, many state-of-art edge applications can be designed and brought to market in record time.