Optical Flow Based Pan/Tilt/Zoom Tracking Using a Kalman Filter Predictor

We are developing software to control pan/tilt/zoom cameras in target tracking applications. Our goal is to produce a system that will be able to use smooth movements in order to keep the specified target centered within the view of the camera.

Group Members

Michael Hopkins, Yaw Owusu-Boamah, Joey Makar

March 2009
Implemented HTTP controls

April 2009
Completed prototype using feedback controller and linear Kalman filter predictor

Setup

Our setup consists of a Canon VB-C60 network pan/tilt/zoom camera interfaced to a PC running Microsoft Visual Basic 2003. The camera and PC communicate with each other via an ethernet connection which allows for monitoring and control of the camera's rotation and zoom. The VB-C60 camera is ideal for tracking applications because it can pan and tilt with speed and accuracy which is vital to making corrections when following a target's position. We extract information from the sequence of images retrieved from the camera to estimate the relative position and velocity of the target. The PC then relays messages to the camera controller, supplying the necessary angular velocity vector to keep the target centered.

Control

HTTP protocol commands are issued to the camera to control its motion and ro request images. HTTP is a widely-used request-response standard in which a client initiates a request that a server responds to, its popularity stems from its extensive use on the Internet. Standard can be issued to control the camera's pan and tilt velocities. This allows us to control both the camera's direction and speed as it rotates about the base.

Canon VB-C60

Pan Tilt Zoom Image Format FPS
340˚ 115˚ 40x optical JPEG 30

Color images are sent to the computer in JPEG format and are sampled at 30 frames per second. The sampling period limits the resolution of the video with respect to time. This means that faster moving segments are more difficult to track. As for the camera's focus, an onboard microprocessor automatically adjusts the lense to keep the picture sharp.