My objective is to create a "FindFaces" recognizer for honeybees that -- when supplied with a video of bees on a honey frame -- Mathematica will highlight the position of the queen bee in the moving mass of bees so that new beekeepers can learn how to "see" the queen by watching the behavior of her movements and the way in which worker bees interact with her. (Or, with an augmented reality headset to highlight the queen in real-time during a beehive inspection).

I do not have an image base of 10,000 bees to use as a training base. But I do have relatively high quality videos of a beehive frame with many hundreds of worker bees moving about, and a queen there as well. After 5 seconds of video, I would have around a set of 10,000 images -- so, I do have a image training dataset, except I need a smart boot-strapping video tool to help me start small, and end by guessing correctly where to find the queen.

My feature request is this : a video annotation tool wherein I markup -- frame by frame initially -- rectangles of specific areas of the video that I may tag as {"type" -> Queen | Worker | Drone }.

I begin by tagging the very first frame of the video with rectangles & tags.

Then, the video annotation tool moves to the next frame and presents the same rectangles overlaying. If I click on a rectangle, it is for the purpose of moving and/or resizing the rectangle to ensure the focus remains on the specimen.

After a few image frames of this hand work, I want the ML Image Classifier to begin predicting where it "sees" similar images on successive frames. At this point, if I click on a rectangle, it is to correct the ML Image Classifier's guess as to the type indicated and the image captured -- and in doing so, the ML Image Classifier restarts its training to incorporate my corrections to successive frames. Soon, I would expect the Image Classifier is tracking the queen reliably, and that my small set of rectangles tagged {"type" -> Worker} blossoms with hundreds of rectangles tracking the movement of the many hundreds of workers actively moving on the beehive frame.

Nice to have features of said video annotation tool would include:

1) Opacity of the rectangle represents the confidence level. That way, when I click on rectangles with a low opacity -- but a correctly identified tag -- my click confirms it guessed correctly and the ML Image Classifier again recomputes from the start of the video and confirms it lands on the same image rectangle with more confidence it got it right;

2) Said video annotation tool needs a good strategy to highlight guesses, and to contrast colors so that as my primary focus is on the queen, if the ML Image Classifier mistakenly identifies one of the workers as a queen, I will see it so as to correct the incrementally growing training set.

3) When training for this frame is complete, I want to apply different annotation tools tracking the queen against a time line (e.g., maybe a flashing red circle for her position every 2 seconds, or a box around her all the time, or cross hairs.