Recently, I have had a couple projects come my way that involve capturing video. One is a 3D imaging project where we want to use five 1080p video feeds to capture a one-minute rotation of an item on a platform, giving us an incredible amount of data to train up some image recognition software.
The other project is pretty exciting since we will have a special guest in our lab and I was trying to figure out how we could get the community involved in some of the things we are going to be building. Why not stream the entire lab live to our YouTube channel so people can see what we are working on and interact with us?
Raspberry Pi USB Video Capture
These are both fun projects, and since we had a bunch of Raspberry Pi 3’s on hand, I decided to use one and see if I could make it capture a bunch of video feeds or turn it into an effective streaming device so I didn’t have to kidnap our video guy’s more powerful laptop. If I could get this little single board computer to perform these tasks it would really free up more powerful machines, plus it would be a cool demo, but I was wondering if the board really had the horsepower to pull it off.
Initially, I was just researching everything I could regarding video capture and settled on trying to work with a USB webcam I had on hand, the Logitech C920, as well as trying to implement the NoIR Pi Camera that we had used in our Fridge Defender project. While the Raspberry Pi comes with some software built in for capturing video like Streamer, I chose to go down a different path and work to utilize FFmpeg, which has a wide community following and a lot of options for different encoders, giving me flexibility and a ton of examples to get started from.
Using FFMPEG to Capture Video
My initial results were not encouraging. Using ffmpeg to capture video from the USB webcam to an AVI file had poor results – the Raspberry Pi would hit about 50% cpu usage and had no hope of capturing 1080p video at a reasonable 30 frames per second. When I went and looked at the results, I was getting about 6.6 frames per second, resulting in choppy and unusable video for streaming our lab or doing 3D object creation. I scaled back to 720p and saw some improvement to 14 frames per second, but this was still not worthwhile. Moving to WVGA format (which is 854x480), I finally got the consistent 30 frames per second I was looking for, but the video quality left a lot to be desired, and for image recognition, the more pixels the better. I was disappointed in my little workhorse Pi but was not ready to give up completely.
Recompile FFMPEG for Raspberry Pi 3
At this point, I could choose to move to a different computer with extra power or dive deeper. I picked ‘dive deeper’ because I like the challenge and I thought the Pi would be able to do more. I began recompiling ffmpeg with different codecs for audio and video to test. Compiling ffmpeg on the Raspberry Pi has typically been discouraged since compiling code can take a lot of work, but from my experience and when you take into account the four core processor of the Raspberry Pi 3, it is certainly reasonable to compile ffmpeg locally with many options. If you do choose to compile locally, it won’t take hours, but it is enough time to go and get a cup of coffee and have a chat with someone. Make sure to use the ‘make -j4’ command to fully utilize the Pi’s four cores or you will be waiting quite a bit longer.
The C920 Raspberry Pi Solution
My eventual ‘eureka’ moment came late on a Thursday night fueled by caffeine and good music. The biggest issue I was having on the Pi was limited power being dedicated to trying to transcode my incoming video stream to something I felt was more usable. The camera was outputting a raw video feed that the Pi was then turning into mpeg4, which is more common. This transcoding was taxing the power of the processor and the bandwidth for storage and memory. There is a way around the problem of raw input with the C920. The C920 has an onboard dedicated encoder capable of outputting an h.264 encoded stream. H.264 is another common video format, and while it brings me a lot closer to what I want, transcoding would still kill my video frames per second.
After figuring out that the camera would give me h.264 without extra work, I also found out that for the lab stream, YouTube is capable of taking in an h.264 stream without me needing to manipulate it first. I still needed to modify the audio a bit, but that didn’t tax the pi. After I was able to just pass through the h,264 stream, I was looking at myself on YouTube in 1080p at a steady 30 frames per second, and I was ecstatic.
My final set command set into ffmpeg to give you a starting point for your own adventures is here below.
ffmpeg -ar 44100 -ac 2 -f alsa -i hw:1,0 -f v4l2 -codec:v h264 -framerate 30 -video_size 1920x1080 -itsoffset 0.5 -i /dev/video0 -copyinkf -codec:v copy -codec:a aac -ab 128k -g 10 -f flv rtmp://a.rtmp.youtube.com/live2/(Your Stream Key Here)
The above command gives me a 1080p 30fps stream with audio from my USB webcam to YouTube. There is still a lot of polish necessary to make it perfect as YouTube still complains about my keyframes not being correct, but it has been hard to argue with the results and it does work.
Raspberry Pi Camera Stream: HD 1080p Video Capture
Now that I can effectively capture 1080p 30fps video, I think that I can at least get a few of the feeds I need for our 3D imaging project. The h.264 video can be converted to MPEG4 after being captured using MP4Box on the Pi, giving our analysis software something it can break down into individual frames for analysis.
So there you have it, the Raspberry Pi 3 has enough horsepower to effectively stream high definition video to YouTube – just make sure that the camera you are wanting to use will give you an encoded stream. The Pi does not quite have what is needed to transcode 1080p video in real time, but with the right choices and some digging it can still get a lot done. I look forward to seeing what you put together with these insights and hope you can all catch our lab stream with special guests as we create our next cool projects.