BAYESIAN VIDEO OBJECT TRACKING By Dehong Ma A Dissertation Submitted in Partial Fulfilment of the Requirements for the Degree of Doctor of Philosophy In Engineering at The University of Wisconsin-Milwaukee December 2006 UMI Number: 3244154 INFORMATION TO USERS The quality of this reproduction is dependent upon the quality of the copy submitted. Broken or indistinct print, colored or poor quality illustrations and photographs, print bleed-through, substandard margins, and improper alignment can adversely affect reproduction. In the unlikely event that the author did not send a complete manuscript and there are missing pages, these will be noted. Also, if unauthorized copyright material had to be removed, a note will indicate the deletion.
® UMI UMI Microform 3244154 Copyright 2007 by ProQuest Information and Learning Company. All rights reserved. This microform edition is protected against unauthorized copying under Title 17, United States Code. ProQuest Information and Learning Company 300 North Zeeb Road P.
Box 1346 Ann Arbor, MI 48106-1346 BAYESIAN VIDEO OBJECT TRACKING By Dehong Ma A Dissertation Submitted in Partial Fulfilment of the Requirements for the Degree of Doctor of Philosophy In Engineering at The University of Wisconsin-Milwaukee December 2006 ose /2 16 ener Major-Prefessor—— mĩ ate i) Graduate School Approval. Date 11 BAYESIAN VIDEO OBJECT TRACKING By Dehong Ma The University of Wisconsin-Milwaukee, 2006 Under the Supervision of Professor Jun Zhang This is a study of tracking moving objects robustly and efficiently from a video stream. Video object tracking aids higher level image analysis by providing richer image information more: efficiently. It can find many applications in smart video surveillance, video conferencing, human computer interface, traffic measurement, image stabilization, video compression, etc.
However, it is still an open problem due to the difficulties from over/under segmentations, the different views of a moving obj ect, the morphology of non- rigid objects, the occlusions of multiple moving objects, lighting changes, shadows, reflections, etc. These difficulties often result in frequent object loss, high false alarm ratio, and other problems. In this thesis we describe a Bayesian framework, combining with a novel model and an efficient particle filtering implementation, to combat those difficulties. The Bayesian 11 approach is optimal in that it gives the minimum square error (MSE) estimation of the object’s states being tracked.
Because the key issue of this approach is modelling, a unique model was proposed and studied in this research. It consists of a compact representation of a moving object, a unique state vector formed from robust shape and colour features in addition to the often-used kinetic features, observations from motion detection, and their relationships described by the state dynamic and observation equations. This model has the advantages of being able to track non-rigid or shape- changing objects and being robust with lighting variations. In order to track multiple objects simultaneously, we studied the typical multiple hypothesis tracking algorithms and proposed a variation of the joint probability data association (JPDA) algorithm to solve the data association problem in video object tracking.
For-fully automatic tracking, a new technique based on sequential likelihood test is combined into the system for the object initialisation and deletion. This technique can greatly reduce the false-alarm ratio and track-loss frequency. The robustness and efficacy of the system are demonstrated by tracking various objects in a variety of applications, such as video based parking-plot surveillance, and video tracking from Unmanned Aerial Vehicles (UAVs). a _Major Professor _/2zl4 lo Date 1V Acknowledgments My foremost gratitude goes to Professor Jun Zhang, my advisor, for his academic and financial support throughout my Ph.
This thesis could not have been completed without his guidance, insightful instructions, and constructive criticism. He is a model of a teacher and a supervisor besides a perfect academic advisor. Over the years, I have learned from him not only effective research approaches and good study habits, but also academic writing and an optimistic, positive attitude. My appreciation also goes to Professor Gilbert G.
Walter, one of my committee members, for his help on my study and research, and Professor Chiu Tai Law, also a member of my-program committee, for all his help in my Ph. study in UWM. ! would like to express my thanks to Professor D. Hosseini, and Professor IG.
Lauko for all their help in my doctoral study and being members of my program committee. I would like to give my thanks to my colleagues and dear friends Jianbo Gao, Xiao Zhang, Weisong Liu and Yirong Wu, Xin Sheng, Wen Hu, Chuan Zhou, Sun Tong, Jieping Xu and all other friends for your help and support in the period of my doctorate study. Weisong deserves my special thanks for his help in the coding of the thesis project. Finally, I must give my deepest gratitude and love to my wife, Ruiyun Wang, who supported me with all her efforts during the hard times in my doctoral study.
Without her endless love and assistance to the family, it would be impossible for me to finish this thesis. Hence, I would like to dedicate it to her, and my dear daughter Jenny, my parents Yiju Ma and Nianxiu Li, and all my family for ever-lasting encouragement, inspiration, and love! Milwaukee, December 2006 Dehong Ma vi Table of Contents Acknowledgements V 1.2 Definition of Video Object TTAaCKINE. HH ee sees kh, 3 1.3 Difficulties of Video Object Tracking and Objectives of Research.4 Outline b09:(-~*tta|iIÍIAIỌỤỌIAIiẢ. Previous Work on Video Object Tracking 8 2.
¬ eee ne ene EERE EOE READE EES HEEEEE EO DES ED EDP EE DESO SEA ee ESOS EEE EES EEOE EES 8 2. nh nà erent nh nà ki nà hệt 9 2. cu HH nh nh nh kh re 10 2.2 Motion Def€CtIOn. cQQn n nnHn nH nh nh nh hy, 12 2.
Representation, Modelling, and Methods -of Filtering.4 Data Association Techmiques. HQ nh nh nhe 18 2.5 Objectives of this Research.c con ng ng HH kh hs. Object Detection Based on Non-linear Prediction 23 EM si Ji gdaiiaaẳaiiiiiiiiiiiiii.2 Gaussian Mixture Model and Its Optimal Prediction.1 Multivariate Gaussian ï nn "¬""— 24 3.2 Optimal Prediction of Multivariate Gaussian Mixture. Small Object Detection Based on Non-linear Optimal Prediction.4 Experimental Results and Comparison with AR model.
Bayesian Single Video Object Tracking 4. ni eee ene teen nh nh nh nà nh bà 4. A Robust Model for Video Object Tracking.1 System State and System Dynamic Mode.2 Observation and Observation Model. ng nà sees eaeaeneeeeeeenes 4.
vet eceeeee pe eneeee eee teeeseeeaene tenes 4. Experimental Results and Conclision. Bayesian Multiple Video Object Tracking 5. ccc cece eee neee cere eneee teen eneeeeeeneeeeneneeteeeeaenees 5.2 Bayesian Filter for Multiple Object Tracking .1 The three steps of Multiple Hypothesis Tracking.3 Hypothesis Likelihood Evaluation and Models.4 Bayesian Multiple Hypothesis Tracking Formulations 5.
Sequential Likelihood Ratio Test for Object Maintenance."— eden serene e neces ene eea ene na enes 5.cc cc cn ee eee eeeneeeseeeaenaeseenens Vill 5.4 Data Association for Multiple Video Object Tracking.1 Multiple Hypothesis Tracking (MHT).2 Joint Probabilistic Data Association Filter (TPDAF). Probabilistic Multiple Hypothesis Tracking (PMHT).5 Experiment Results and Conclusion. cà nền nha 85 5.1 Simulation Experiments and Comparison of MHT, JPDA, and PMHT. Experiments on Multiple Video Object Tracking Using MHT, JPDA, and 6.
Summary 93 1X List of Tables Table 3. Implementation/Control pararnef€TS. MSEEs from MHT, JPDAF, and PMHT on simulation. 88 List of Figures Figure 1.
The states (x) and observations (z) in a tracking system. A typical video tracking SySf€T.- Ác HH ng ng m 9 Figure 2. The grouped observations are from objects. Isolated are from false alarms.
con HH ng HH ee ene ence eee km vàn 20 Figure 3.Right: one-dimension vector. An Object in Synthetic Background. 00sec cee e eee e eee rere eee ee ee 29 Figure 3. An Object in a Brodatz Texture Background.
A Real World Ímage.--- ng nh hy 33 Figure 3. Visualizing the Optimal Nonlinear and Linear Predictors - A Simple ExampÏÌ©. eee eee eee cee eee sen cee tee een eee keo se «uc Figure 4. Illustration of the state of a moving obJecf.
A typical moving region detection example by our technique. Here the images were taken from a MOVING CAMETA.ccceceeee ee eee eee ee eees 51 Figure 4. Illustration of the observation data. Tracking a single person using a PTZ camera on pre-recorded video clip.
Red ellipse: our result; Green rectangle: template matching; Blue rectangle: histogram matching (mean-shIfẨ).-- cà taeeeeaeens 61 Figure 4. Tracking a single person in real-time using a PTZ camera. Red ellipse: tracking result; Blue ellipse: partIcles.1: Illustration of hypothesis generating. Simulation experiments on MHT, JPDAF, and PMHT.
Real video experiments using MHÍT. sành he 90 XI Figure 5. Real video experiments using JPDAF. eee eee ee ee eed Figure 5.
Real video experiments using PMHT ee ee ry xii Chapter 1 Introduction 1.1 Background “Vision is knowing what is where by looking,” D. Marr said in his book Vision [1]. The ultimate goal of computer vision, or machine vision, is to make a computer/machine be able to “see” the world. Computer vision is one of the most important aspects of machine intelligence.
The research on computer vision has evolved from laboratories to the real world by finding many applications in a variety of areas, such as in medical systems [2]-[6], factory automation [7]-[10], remote sensing [11]-[16], bio-identification in security and law enforcement [17]-[25], smart vehicles [26]-[30], etc. Almost all of the modern intelligent machines resort to machine vision to acquire outside information as a smart input device. In order to provide richer image information more efficiently, video object tracking, especially active object tracking by controlling sensor parameters [31]-[33], is a critical process in a computer vision system. A typical vision perception process begins with motion detection, also called focus invoke, then it has a fixation or tracking process to b2 acquire more information about the moving object until an assessment or a further action has been made, which ends a perception circle.
In addition to acquiring better image information of the object of interest, video object tracking itself finds many direct applications. In smart surveillance or virtual guard systems it is used for event detection and compression [35]-[38], and in teleconferencing for auto-recording [39]-[40]. It is also used for traffic analysis [41]-[42], human-computer interface [43]-[45], image stabilization [46]-[48], automatic target tracking in UAV [49]- [51], and image registration in medical imaging [6] [52]. In recent years the need for smart surveillance for homeland security, crime prevention and verification has strengthened.
Usually a smart surveillance system should be able to detect potential criminal activities in a public area and to obtain close-up video recordings and give instant warnings when suspicious behaviours have been detected. Currently these tasks are mostly done with human participation, which is labour-intensive and stressful, and also creates a variability between different operators (due to experience and work ethic). Different times of the day also can cause inconsistent results. All these applications call for reliable, robust, efficient and fully automatic video object tracking techniques, which are still an open research area due to a variety of difficulties remaining unresolved.2 Definition of Video Object Tracking Computationally, the tracking process is to estimate the state (position, velocity, etc.) of a moving object from its observations or measurements over time [53].
The video object tracking is to estimate a moving object’s state through its video observations. The observations are usually “noisy,” which can be caused by observation inaccuracy or measurement errors. Also, there is uncertainty about the state itself at any given time, so the state can be viewed as a random process over time. Hence, tracking is essentially a statistical estimation process of the state of a random process [54].
A tracking system mainly consists of two elements: system state and system observation. The system state refers to the state of the object of being tracked, which usually consists of a set of state variables. They are also called the system state-vector. Some often-used state variables are an object’s kinetic parameters.