The OpenCV AI Kit ( OAK ) super simple video streamer starter
I know the OAK has been out for a little while now, but I am just starting to really dig into this device from Luxonis. If you dont know what it is, you can find it and buy it here.
The best place to start, is with the great documentation from Luxonis.
The programming metaphor is one of a processing pipeline. Each step in the pipeline is a ‘node’ and there are a number of ‘node’ types that are part of the API. You can find documentation on the different types of Nodes here.
The image below, from the Luxonis documentation, represents the high level architecture. The OAK devices handles all of the video and image processing, including running computer vision models and then send resulting information to the host computer to act upon.
In our simple example we need two nodes:
ColorCamera Node
This node will handle reading high resolution frames from the OAK camera.
The ColorCamera node has 2 inputs, and a number of outputs. For this simple example, we will only need the output labeled ‘preview’. This output is suitable for smaller preview sized images.
XLinkOut
This node will handle consuming messages from nodes on the OAK device, and sending them to the host computer to an internal queue.
The application / script running on the host computer pulls data from the internal queues. In our case, the internal queue will hold image frames.
Video Stream Pipeline
Where is the code?
You can find the code on my Github repo.
Assuming you have a Python virtual environment setup, all you have to do to get started is the following:
git clone https://github.com/youngsoul/depthai-video-stream.git
cd depthAI-video-stream
pip install -r requirements.txt
python videostream.pyPipeline Setup
The important lines of code are:
Line 2 - creates the Pipeline.
Line 6 - creates the ColorCamera node
Line 11 - creates the XLinkOut
Line 13 - connects the camera node to the xlinkout node.
Processing Loop
The loop to process messages from the internal queue looks like above.
The important lines of code are:
Line 1 - create a Device context manager which represents the software twin of the physical OAK device.
Line 2 - get a reference to the internal queue where frames are being sent from the OAK
Line 6 - read a frame from the queue
Line 8 - read the frame as an OpenCV frame
Line 10 - show the frame in an OpenCV windown
That is almost the entire file, but for those are would like to see the entire file, here it is.
Entire Script
Wrap up
While this is a really simple starter, it did help me get a better understanding of the mental model used by the OAK devices. With this understanding I am now ready to tackle the more complex and interesting computer vision tasks.
