View MJPEG Stream in Flex

I get to find a useful article about viewing MJPEG streams in Flex. Keep on reading if you are interested.

So, I'm learning Flex at long last. The reason for me finally taking up this challenge is that I finally found a project I believed it well suited to. You see, my family has lots of digital photos. About 9000 at last count. We keep these stored on a NAS in my office. All of our computers are set to show these photos as our screen savers. However, I've wanted a nice wireless digital picture frame that would randomly pull from this pool of images. Well, to make a long story short, I was unable to find a single wireless picture frame that could read from a Samba share. (And trust me, I've tried a lot of things.)

So, finally I became frustrated enough to take this into my own hands. I plan make my own ideal digital picture frame. I'm currently working on an Air application that will run in full screen and provide access to a range of picture frame components. Obviously, a part of this will display pictures. Other components will display weather information, stats on Alagad, and pretty much anything else I want to wire into my picture frame. Eventually, I'll buy a nice, thin, touch screen tablet PC, configure it to run my air app on load, get it professionally framed and hang it on the wall.

Now, one of the components I want this picture frame to show is a video stream from my daughters baby monitor camera. Actually, it's not really a baby monitor camera it's a more-or-less generic wireless surveillance camera. I don't have the model information any more and it's not branded at all. But, from research I've done there are a lot of these cameras which run the same software.

The camera runs a basic web server so I can pull up the camera and see either a still photo or a video stream in a Java Applet or ActiveX Control. I did a little playing around with the camera and discovered that the still photo is returned from a file called "image.jpg". Any time you request this file you get the latest image from the camera. So, my first attempt a hooking into the video camera was simply to request this photo over and over again and use that as the source for an Image object. This worked to an extent, but the Image object wouldn't refresh quickly enough and would as a result flicker. Beyond that, I had to make a complete HTTP request for each frame of the video and it had a very slow refresh rate. I was able to solve the flickering problem by having two images on my canvas and alternate setting the source of each image and hiding the other image.

I was happy to have accomplished that, but it just wasn't very nice. So I decided to do a little more research. The Java Applet on the camera was called xplug.class. Googling that revealed a few people who had decompiled the class and used that to discover that the camera actually streams video from a file named mjpeg.cgi. If you access this file directly you'll simply get a long stream of binary data which makes up the video feed from the camera. Obviously the Java Applet and Active X Control could read this stream and use it to display the video feed.

A little more research revealed that, sadly, mjpeg is not a standard but more of a technique. Mjpeg really is just a concatenated stream of jpeg images, one after another. I'm familiar with the JPEG format (at least to some degree) from my work on the Image Component. At least, I knew enough to know the first few bytes that mark a JPEG image (FF D8). So, I figured I'd look at the data in the video stream to see if I could figure it out. I captured a few seconds of the video to disk using Curl and opened it up in a hex editor. The first thing I noticed was that each frame of the video was denoted by an ascii string "--video boundary--". Looking a bit further into the file I discovered that I could find the markers that start an image. I tested cutting the bytes from the beginning of where I thought the JPEG would start to the last byte before the video boundary into a new file and saved it as a jpeg, which I was subsequently able to read!

I knew that if I could read the feed by simply starting at the top of the feed, find the start of an image and the end, cutting out that image, displaying it and repeating the process forever.

Simeon Bateman filled in the last little piece of data I needed to know to make this work. Namely, that you can simply provide binary data to the source property of an Image object and Flex will display the image.

So, after figuring out how to make a socket connection from Flex I was able to parse the video feed and display the video feed! I did still have the flickering problem but I solved it in a similar manner.

Attached to this blog entry is a final Flex Component which connects to the camera by URL and streams the feed. Anyone who has a use or it may use it, assuming your video feed is the same format as mine. This is my first real bit of AS3/Flex development though so don't be offended if I'm not doing things in the best way possible.

Here's an example usage of the component:

<webcam:webcamImage left="0" top="0" right="0" bottom="0" host="mycamera.com" port="80" />

You can also supply username and password properties to access password protected cameras.

If you have the MJPEG on hand, you can also convert MJPEG to Flash quickly. It allows simple editing and custom players.

Source: http://www.alagad.com/go/blog-entry/viewing-mjpeg-streams-in-flex