VIETNAM NATIONAL UNIVERSITY HO CHI MINH CITY HO CHI MINH UNIVERSITY OF TECHNOLOGY FACULTY OF COMPUTER SCIENCE AND ENGINEERING GRADUATION THESIS Vehicle detection in surveillance videos Major: Computer Science THESIS COMMITTEE: Computer Science and Engineering SUPERVISOR: Assoc. Nguyễn Thanh Bình REVIEWER: Ph.D Phan Trọng Nhân Student 1: Ôn Quân An 1852221 Student 2: Hoàng Phương Nam 1852595 HO CHI MINH CITY, 10/2022 2 3 4 COMMITMENT We promise that the thesis : “Vehicle detection in surveillance videos” is conducted by us, under the supervision of Assoc. Nguyễn Thanh Bình. All of the references and related works are cited in Section References.
The contents of this thesis have never been published under any format. If there were any violations, we will take full responsibility before the committee and presidents of the institution. Group members 5 ACKNOWLEDGEMENT First and foremost, we would like to thank our research supervisors Assoc. Nguyễn Thanh Bình - lecturer of Computer Science and Engineering Department (Ho Chi Minh City University of Technology - University of Vietnam National University, Ho Chi Minh City).
Without his assistance and dedicated involvement in every step throughout the process, this paper could have been accomplished. I am extremely grateful to you for supporting and understanding me during the past time. We would also like to show gratitude to my committee of Ho Chi Minh City University of Technology, lecturers of the Computer Science and Engineering Department for getting us through our dissertation required with academic support, and I have many, many people to thank for listening. Most importantly, none of this could have happened without our family.
To our guardians – it would be an understatement to say that, as a family, we have experienced some ups and downs in the past three years. This dissertation stands as a testament to your unconditional love and encouragement. Group members 6 APPENDICES COMMITMENT 2 ACKNOWLEDGEMENT 3 1. Structure of Thesis 7 2.
Background & Related Works 8 2. Two-stage detection algorithms 10 2. One-stage detection algorithms 11 2. Detection in adverse environment 16 3.
Experiments & Results Evaluation 29 4. Computing Resource & Dataset 29 4. DAWN dataset image tests 32 4. Video-based tests 37 4.
Conclusion and Future Work 40 5. Future Work 41 References 42 8 1. Thesis Introduction With the recent trend of development in the field of technology, especially in many countries, the traffic surveillance system is deployed and widely used to monitor traffic conditions to ensure road safety as it is a major concern recently. However, news about the road accidents validating the inefficiency of the current surveillance systems while statistics show that there is a large increase in the rate of road accidents every year.
So an accurate real-time traffic information collecting technique is necessary and essential for intelligent traffic surveillance and autonomous driving systems. Particularly crucial is vision-based technology that uses real-time video cameras to gather road photos and automatically extract traffic data from those images. Due to the wide range of conditions, lighting, weather, vehicle position, and scale patterns, traffic surveillance camera images are quite complex. Vehicle detection in surveillance videos is the most important and challenging stage of traffic surveillance using computer vision techniques.
The vehicle detection system includes detection of moving vehicles, counting the number of vehicles and the classification of the detected vehicles. This thesis plans to create a lightweight and real-time vehicle detection system by implementing an appropriate algorithm on a video input taken from surveillance cameras. Missions Our team aims to apply the computer vision algorithms as it simply needs video footage from the surveillance camera to collect data of traffic. Those data can be used to analyze and find a solution to lower the accident rate in the future.
Our vehicle detection system using surveillance’s video includes the following features: - Detection vehicles. - Classification of its type. - Counting the number of vehicles. Goals In this thesis, we will investigate the background knowledge and related researchs.
While doing the thesis, we will collect and enrich the dataset. And for illustration, we will run a 9 demonstration on a few labeled images in Viet Nam. In the future, we hope our data of traffic can be used as a dataset for various applications like congestion analysis, accident detection. Limitations Our vehicle detection system aims to work normally on good weather and light condition videos (clear sky, sunny day) and on extreme weather and low light images (rain, fog, blizzard day or low light condition).
It also can operate when the vehicle at the front is not blocking more than 50% of the vehicle behind it. Finally, the inference speed is at least real time with 24 frames per second similar to industrial standard. Structure of Thesis The structure of this thesis is as below: - Chapter 2 contains some background knowledges and related researches that are used in this thesis - Chapter 3 restates the problem and introduces the algorithms and methodologies that we use to build the system. - Chapter 4 illustrates the results of the vehicle detection system and evaluates its performance.
- Chapter 5 summarizes the work and compares the pros and cons of the suggested method and proposes future development. Background & Related Works 2. Object detection In the past few years, we can see a huge leap in many fields of information technology. One of the most notable developments is the development of Object Detection algorithms and implementations.
Object detection has progressed from detecting simple objects to human faces, animals, vehicles, etc. With the recent development of technology and society, people have implemented it in many industries like mass manufacturing, autonomous driving, traffic monitoring.1: Object detection development [15] Object detection is a computer vision technique for detecting instances of objects in pictures or videos. Object detection algorithms usually integrate with machine learning or deep learning to produce meaningful results. People can quickly identify and pinpoint objects of interest when viewing photos or videos.
The goal of object detection is to simulate this intelligence using a computer. Deep learning object detection algorithms are data-driven algorithms for feature extraction and object detection, as opposed to feature engineering methods. Deep learning methods extract features from image data statistics and automatically learn object appearance features. Furthermore, CNN features have more representative characteristics, and the feature learning process is similar to that of the human visual mechanism.
Deep learning object detection algorithms are divided into two categories: two-stage object detection and one-stage object detection, based on network structure and feature learning progress. R-CNN Feature extraction Using the Caffe implementation of the CNN, the algorithm extracts a 4096-dimensional feature vector from each region proposal. Forward propagation of a mean-subtracted 227x227 RGB image through five convolutional layers and two fully connected layers is used to compute features [3]. To compute features for a region proposal, the image data must first be converted in that region into a format that the CNN can understand (the CNN's architecture requires inputs of a fixed size of 227x227 pixels).
R-CNN chose the simplest of the many possible transformations for the arbitrary-shaped regions, all pixels are wrapped in a tight bounding box around the candidate region to the required size, regardless of its size or aspect ratio. It dilates the tight bounding box before wrapping so that there are exactly p pixels of warped image context around the original box at the warped size (p = 16). Run-time analysis There are two things that make R-CNN detection efficient. First, all CNN parameters are common to all categories.
Second, when compared to other common approaches, such as spatial pyramids with bag-of-visual-word encodings, the feature vectors computed by the CNN are low-dimensional. The UVA detection system, for example, has features that are two orders of magnitude larger. The time spent computing region proposals and features (13s/image on a GPU or 53s/image on a CPU) is amortized across all classes as a result of this sharing. The only computations that are class-specific are dot products between features and SVM weights, as well as non-maximum suppression.
In practice, all of an image's dot products are combined into a single matrix-matrix product. In most cases, the feature matrix is 20004096, and the SVM weight matrix is 4096N, where N is the number of classes. R-CNN can scale to thousands of object classes without using approximate techniques like hashing, according to this analysis. On a modern multi-core CPU, matrix multiplication takes only 10 seconds even if there are 100k classes.
This efficiency is not solely due to the use of shared features and region proposals. Due to its high-dimensional features, the UVA system would be two orders of magnitude slower, 12 requiring 134 GB of memory just to store 100k linear predictors, versus only 1.5GB for these lower-dimensional features. Fast R-CNN The approach is similar to the R-CNN algorithm. But, instead an entire image and a set of object proposals are fed into a Fast R-CNN network.
To create a convolutional feature map, the network first processes the entire image with several convolutional and max pooling layers. A region of interest (RoI) pooling layer extracts a fixed-length feature vector from the feature map for each object proposal. Each feature vector is fed into a series of fully connected layers, which branch out into two sibling output layers: one that generates softmax probability estimates for K object classes plus a catch-all "background" class, and another that generates four real-valued numbers for each of the K object classes. The refined bounding-box positions for one of the K classes are encoded by each set of four values.
The Region of Interest pooling layer The RoI pooling layer employs max pooling to convert features within any valid region of interest into a small feature map with a fixed spatial extent of H W, where H and W are layer hyper-parameters that are independent of any specific RoI [4]. An ROI is a rectangular window into a conv feature map in this paper. Each RoI has a four-tuple (r, c, h, w) that specifies the top-left corner (r, c), as well as the height and width (h, w). RoI max pooling divides the h w RoI window into a H W grid of sub-windows with approximate sizes of h/H w/W, and then max-pools the values in each sub-window into the corresponding output grid cell.
As with standard max pooling, each feature map channel is pooled separately. The RoI layer is simply a one-level version of the spatial pyramid pooling layer used in SPPnets. Two-stage detection algorithms The two-stage detection algorithm divides the detection process into two stages: generating candidate regions (region proposals) and classifying them (generally with location refinements). To begin, most algorithms use a selective search [2] to generate proposal regions of interest (ROI).
Second, employ the SVM classifier to determine the most accurate region and type of object. R-CNN is a classic object detection algorithm that uses Convolutional Neural Networks to extract vehicle features (CNN). The detection task is completed in two stages using this method. Fast RCNN is an improved version of R-CNN that includes the ROI pooling layer, 13 which eliminates duplicate computation in R-CNN.
Furthermore, Fast R-CNN improved detection results by using a softmax layer instead of an SVM classifier in R-CNN. Faster R-CNN is an improved version of Fast R-CNN that uses RPN networks instead of the selective search methods used by R-CNN and Fast R-CNN. The loss function smooths the training progress and improves the detection results, which is another Faster R-CNN innovation. The region-based fully convolutional network (R-FCN) aims to improve detection performance by combining Faster R-CNN and FCN.
The main contributions are: the introduction of FCN to achieve more network parameters and feature sharing (compared to Faster R-CNN) and solving the problem of fully convolutional networks' location sensitivity deficiencies (using position-sensitive score maps). The two-stage detection algorithms produce accurate vehicle detection results. However, it cannot achieve real-time inference.