We held our next tinyML Talks webcast. Vijay Janapa Reddi from Harvard University has presented tinyMLPerf: Deep Learning Benchmarks for Embedded Devices on March 16, 2021.
Tiny machine learning (ML) is poised to drive enormous growth within the IoT hardware and software industry. Measuring the performance of these rapidly proliferating systems and comparing them in a meaningful way presents a considerable challenge; the complexity and dynamicity of the field obscure the measurement of progress and make embedded ML application and system design and deployment intractable. To enable more systematic development while fostering innovation, we need a fair, replicable, and robust method of evaluating tinyML systems. The talk introduces TinyMLPerf, a machine learning benchmark for ultra-low-power systems that measures both performance and energy. TinyMLPerf extends the existing MLPerf benchmark suite from MLCommons (mlcommons.org) to include tinyML systems. The benchmark suite consists of four ML tasks: small vocabulary keyword spotting, binary image classification, small image classification, and detecting anomalies in machine operating sounds. Thanks to a collaboration with EEMBC, it also features power measurement, which is critical in evaluating tinyML systems. TinyMLPerf will enable device makers and researchers to choose the best hardware for their use cases and provides hardware and software vendors to showcase their offerings. Additional details can be found here: GitHub - mlcommons/tiny: TinyMLPerf is a benchmark suite for extremely low-power systems such as microcontrollers.
Vijay Janapa Reddi is an Associate Professor at Harvard University, Inference Co-chair for MLPerf, and a founding member of MLCommons, a nonprofit ML organization that aims to accelerate ML innovation. He also serves on the MLCommons board of directors.
Watch on YouTube:
Vijay Janapa Reddi
Download presentation slide:
Vijay Janapa Reddi
Feel free to ask your questions on this thread and keep the conversation going!