DeepMeerkat
Machine learning for video-based biodiversity surveys.
B.G Weinstein. Scene-specific convolutional neural networks for video-based biodiversity detection. Methods in Ecology and Evolution. * Accepted.
|
|
|
|
DeepMeerkat was supported by a Segment Open Data Fellowship. Read more!
Getting Started
With the publication of the article in Methods in Ecology and Evolution, I hope there will be many users hoping to train their own models. I was just contacted by Bette Loiselle and her collaborators studying manakins in Central America. Manakins are communal lekking species with males performing elaborate displays!
The goal of using DeepMeerkat is to find the relatively rare periods of time when manakins are present. Let's go through the process of training a new model. I'm going to try to present this as much as possible from a new user's perspective. I'll add in some thoughts about decisions specific to a project.
- Download and Install DeepMeerkat. Either use the binaries provided above, or get the latest build from github.
- Collect training videos. What kind of videos are good for use? Typical ones! Don't bias the training in a way that will distort the kind of images it expects in future videos. Just pick your typical videos. Try to pick a diverse bunch, some with lots background motion, some with little motion. We can customize our target classes as we continue.
- Open DeepMeerkat and run in training mode.
- This will return cropped frames that have been detected by the motion detection algorithm.
- Now its time to sort the output frames. Make two folders, one called Training and one called Testing. Within each folder make a Positive and Negative folder.
- Place images of your object of interest in the Positives folder. Place images of the background into the Negatives folder. The training data is used to predict the testing data. I recommend a minimum of 10:1 ratio of training to testing data. For example, in my paper I had 14,000 positives and negatives and 500 training samples for each class.
- But what counts as a positive image? This depends on the use case and there is no general rule. Think of it as what kind of image would I want to see returned from a future video. In our manakin example, we are just trying to find the periods when a manakin is present. Researchers will probably go back for future review to the original video. It can be tricky to decide and I recommend creating a folder of secondary positives to test whether including them helps prediction. For now, let's call images like the top image a Positive, and we will delete the second image, and not use it as positive or negative. In the second photo, we might be able to tell there is a male manakin there, but I wouldn't want to use it as data.
- Most applications of image-based machine learning use tens of thousands of examples, even millions of sample images. Ecology cannot afford such luxuries. The drawback is that we must be constantly vigiliant of overfitting, where our model sees and image and will only look for extremely similiar images. DeepMeerkat uses statistical functions to try to reduce overfitting. But its not magic. If we want manakins in many different poses, depths and conditions, we need to keep training our model. For example, if we want images of both male and female manakins, we need to train on images of females as well.
Advanced Training: For most users, the above strategy will work well to generate training frames. For those ecologists accustomed to the command line, I wanted to take a brief second to show how we can manipulate a couple arguments that might be useful in special cases.
- DeepMeerkat lives in the application folder, and the entry point is called main. We can directly access the main application in terminal (or cmd for windows users). We can then run the command line version and change any of the default arguments. In case, I thought it was useful to reduce the minimum size to 0.0001 to make sure any motion, no matter the number of pixels, will be returned.
Next steps: Now that we have images scored as Positive and Negative, its time to train our model. Let's head on over to github to look at cloud-based training (recommended) and local training.
