This project provides an efficient approximation of the SiLU activation function optimized for quantized inference on embedded devices, targeting the ESP32-P4 and ESP32-S3 platforms.
A detailed explanation of the method and implementation is available here:
🔗 ESP32-P4 Deep Learning Pipeline: Approximating SiLU for Efficient Quantization
The sigmoid approximation used in this project is based on:
🔗 Computationally Efficient Approximations of S-Shape Functions (Page 20)
- Fast bitwise approximation of SiLU using a quadratic sigmoid function
- Optimized for MCUs with efficient shift-based computation
- ESP32-S3 SIMD implementation (available in
QSiLUApprox_espS3_b1subfile) for enhanced vectorized operations - Maintains high accuracy in the key range [-4, 4] while preserving expected SiLU behavior
Clone the repository and follow the usage instructions in the blog post:
git clone https://github.com/BoumedineBillal/silu_quantizer.git
cd silu_quantizerThe ESP32-S3 SIMD implementation can be found in the QSiLUApprox_espS3_b1 directory. To use it:
- Open the project in VSCode with the ESP-IDF extension
- Build the project using the ESP-IDF build system
- Flash the resulting binary to your ESP32-S3 device
This SIMD implementation can be easily integrated into any deep learning inference engine running on ESP32-S3 platforms to accelerate models that use the SiLU activation function.
Contributions are welcome! Feel free to open an issue or submit a pull request.
📩 Questions? Reach out via GitHub Issues.