tinyML Talks on April 5, 2022 “Constrained Object Detection on Microcontrollers with FOMO” by Shawn Hymel

We held our next tinyML Talks webcast. Shawn Hymel from Edge Impulse presented Constrained Object Detection on Microcontrollers with FOMO on April 5, 2022.

April 5 global forum

Image classification has been a core focus of deep learning for many years. However, many computer vision applications require knowing where objects are in an image and the ability to count the number of objects, which goes far beyond simple image classification. This is where object detection comes in.

Object detection models are capable of finding objects of interest in an image and provide us details about those objects, such as their classification, location, size, relative distance from the camera, etc. A handful of object detection models, such as MobileNet V2 SSD and YOLOv5, are optimized for low-power systems, including smartphones and single board computers. However, most microcontrollers are still incapable of running such models due to their processing and memory limitations.

Edge Impulse has developed a new technique named “Faster Objects, More Objects” (FOMO) that performs constrained object detection on low-power devices, such as microcontrollers. FOMO provides the location of target objects in an image, but it does not give arbitrary bounding box information about the size or distance of objects. As a result, it requires up to 30x less processing power and memory than MobileNet V2 SSD or YOLOv5.

In this talk, we will describe object detection, how FOMO works, and provide a live demonstration of constrained object detection on a microcontroller.

Shawn is a machine learning DevRel engineer, instructor, and university program manager at Edge Impulse. He creates compelling technical videos, courses, and blog posts around edge machine learning and embedded systems that inspire and teach engineers of all skill levels. Shawn is an advocate for enriching education through STEM and believes that the best marketing comes from teaching. He can be found giving talks, running workshops, and swing dancing in his free time.

=========================

Watch on YouTube:
Shawn Hymel

Download presentation slides:
Shawn Hymel

Feel free to ask your questions on this thread and keep the conversation going!

Hi, this video is not yet available in Tinyml YouTube channel.

Hi Ramkumarkoppu,

Please find the video recording below:
tinyML Talks: Constrained Object Detection on Microcontrollers with FOMO - YouTube

Can it be used for PoseNet or something similar?

PoseNet is a little different in that it is looking for joint positions on a human body. It might be possible to train FOMO to look for joint points, but it would still be limited to the grid locations of the feature maps.

Does it support RISC-V 32bit MCU?

As long as you have a C++ compiler for your architecture, you can compile the Edge Impulse inference library (including FOMO) for your device. You will also want to make sure that you have enough RAM and flash on your device. Note that this inference library supports CMSIS-NN extensions, which makes most neural network architectures (including FOMO) run faster on supported Arm processors. While you can certainly create DSP and NN hardware acceleration on a custom RISC-V core, we do not support such acceleration in our inference library out of the box. However, as the inference library is open source, you could theoretically change it to support whatever extensions you desired.

Thanks for this interesting information!