Honey, there are deer at the bird feeder again! ( Keras, OAK-1 and a RaspberryPi to detect deer)

3 min readFeb 2, 2022

OAK-1 running on RPI using a Keras model to detect presence of deer

( Please take no offense to the use of PyImageSearch Raspberry Pi books to elevate the camera. These books are handy on my bookshelf as I have been reading through them and I highly recommend them. These books are a big factor in my understanding of how to apply computer vision using the Raspberry Pi. )

If you have been following my posts on using the OpenCV AI Kit (OAK-1) camera, I am going to build upon those articles and create a deer detector to alert me when the deer have eaten all of the food from the bird feeder so we can feed the birds.

In this article I will take you through the steps necessary to create a deer detector system that runs on a RaspberryPi.

From finding the necessary datasets, creating training, validation and testing datasets, using transfer learning to train a model to detect the presence of deer to evaluation to deployment.

You can find all of the code and write up on my github repo.

Step 1: Downloading a dataset of Deer images and Landscape images from Kaggle

Two Kaggle datasets were used; deer dataset and a landscape background dataset.

Step 2: Create a script to currate a train/validation/test dataset of the Kaggle images

From the raw dataset, create a new dataset of train/validation/test images and create the supporting folder structure.

See the script, make_datasets.py

Step3: Using transfer learning, train a Keras model to detect the presence of deer.

See the script, model_train.py

We start with a MobileNetV3 model, using the imagenet weights and remove the top fully connected portion. We create a new fully connected dense top model and retrain the top section.

Step 4: Evaluate the model on the test data

See the script, evaluate_model.py

In this script, we use the test data that we created in Step 2, along with images and video collected from the backyard bird feeder. I purposefully did not use images from the actual backyard because I wanted to see if I could train the model and have it generalize to the actual location without seeing images of the actual location.

Normally you want to train with data that is as close to your ‘production’ environment as possible. Again, I used this as an experiment.

Step 5: Test with video

See the script, detect_video.py — video-file datasets/backyard/movies/test_deer.MOV

So far all of this has been executed on my MacBook Pro — but I really want to deploy to the OAK-1 device and run it from my Raspberry Pi.

When you run this detect_video.py script locally, make a mental note of the performance. You will see running it on the OAK-1 is much faster.

Step 6: OAK-1 and Raspberry Pi

We need to install three additional libraries

pip install openvino-dev
pip install blobconverter
pip install depthai

See the script, create_openvino.sh

In my previous article, I covered the details of how to convert a Keras model to an OpenVINO format to run on the OAK-1.

When deploying to the Raspberry Pi. If you try to run the OAK-1 and the RaspberryPi does not recognize the device. Try running the following:

echo 'SUBSYSTEM=="usb", ATTRS{idVendor}=="03e7", MODE="0666"' | sudo tee /etc/udev/rules.d/80-movidius.rules
sudo udevadm control --reload-rules && sudo udevadm trigger

You can find information on this issue on the Luxonis Troubleshooting page

Wrap up

I am going to call this project a success. While I would now pull images from the backyard, and from the OAK-1 from where I would have it setup to better train on ‘production’ like images, I thought the performance of the model was acceptable and the performance of the system using the OAK-1 was great.

You can find a more detailed writeup, the source code and all of training data on my Github repo.

Now how to automatically fill the deer, umm I mean, bird feeder…..