VIETNAM NATIONAL UNIVERSITY HA NOI UNIVERSITY OF ENGINEERING AND TECHNOLOGY PHAM VAN THANH DEVELOPMENT OF REAL-TIME SYSTEMS TO DETECT AND TRACK ON-DUTY INJURED FIREFIGHTERS USING ADVANCED SIGNAL PROCESSING TECHNIQUES DOCTORAL THESIS IN ELECCTRONICS AND TELECOMMUNICATIONS ENGINEERING Ha Noi - 2022 VIETNAM NATIONAL UNIVERSITY HA NOI PHAM VAN THANH DEVELOPMENT OF REAL-TIME SYSTEMS TO DETECT AND TRACK ON-DUTY INJURED FIREFIGHTERS USING ADVANCED SIGNAL PROCESSING TECHNIQUES Major: Electronic Engineering Code: 9 52 02 03.01 DOCTORAL THESIS IN ELECCTRONICS AND TELECOMMUNICATIONS ENGINEERING Supervised by: Assoc. Tran Duc Tan Ha Noi - 2022 DECLARATION “T hereby declare that the work contained in this thesis is of my own, and it has been written by myself under the supervison of Professor Tran Duc Tan at Faculty of Electrical and Electronic Engineering, Phenikaa University, Hanoi, Vietnam during the period from September 2017 to August 2021. The thesis has not been previously submitted for a degree or diploma at this or any other higher education institution. I have duly acknowledged all the sources of information which have been used in the thesis.
The thesis content has been partly published in my list of publications as below: 1. Pham Van Thanh, Tuan Khai Nguyen, Duc Anh Nguyen, Nhu Dinh Dang, Huu Tue Huynh, Duc-Tan Tran*, “Adaptive Step Length Estimation Support Indoor Positioning System using Low-Cost Inertial Measurement Units’”’, 2020 IEEE Eighth International Conference on Communications and Electronics, pp. Pham Van Thanh, Le Quang Bon, Nguyen Duc Anh, Dang Nhu Dinh, Huynh Huu Tue, Tran Duc Tan, “Mulfi-Sensor Data Fusion in A Real-Time Support System for On-Duty Firefighters", Sensors 2019 (ISSN: 1424-8220 — SCIE). Van Thanh Pham, Duc Anh Nguyen, Nhu Dinh Dang, Hong Hai Pham, Van An Tran, Kumbesan Sandrasegaran and Duc-Tan Tran, “Highly Accurate Step Counting at Various Walking Speeds Using Low-Cost Inertial Measurement Unit Support Indoor Positioning System ”, Sensors.
Pham Van Thanh, Duc-Tan Tran, Dinh-Chinh Nguyen, Nguyen Duc Anh, Dang Nhu Dinh, S. El-Rabaie and Kumbesan Sandrasegaran, “Development of a Real-time, Simple and High-Accurate Fall Detection System for Elderly Using 3- DOF Accelerometers”, Arabian Journal for Science and Engineering. Pham Van Thanh, Anh-Dao Nguyen Thi, Quynh Tran Thi Thuy, Dung Chu Thi Phuong, Viet Ho Mau and Duc-Tan Tran, “A Novel Step Counter Supporting For Indoor Positioning Based On Inertial Measurement Unit”, 7th international conference on Integrated Circuit, Design, and Verification ICDV), IEEE, pp. Nguyen Van Duong, Pham Van Thanh, Tran Van An, Nguyen Tuan Khai, Duong Thi Thuy Hang, Hoang The Hop and Tran Duc Tan, “Elevator Motion States Recognition Using Barometer Support Indoor Positioning System”, The 7th International Conference in Vietnam on the Development of Biomedical Engineering, IFMBE Proceedings, Springer, pp.
The Hop Hoang, Van Thanh Pham, Thuy Quynh Tran Thi, Huu An Nguyen, Tuan Khai Nguyen and Tan Tran-Duc, “Xdy đựng hệ thong xác định độ cao bên trong nhà và công trình sử dụng đa cảm biến áp suất”, Hội nghị Quốc gia lần thứ XXI về Điện tử, Truyền thông và Công nghệ Thông tin (The 21st National Conference on Electronics, Communications and Information Technology), 2018, pp. Ha Noi, I6" January, 2022 Author ACKNOWLEDGEMENT I would like to express my sincere thanks to my advisor Assoc. Tran Duc Tan, Faculty of Electrical and Electronic Engineering, Phenikaa University for the guidance and support throughout the completion of my thesis. My thanks go to all lecturers and people in Faculaty of Electronics and Telecommunications, Universisty of Engineering and Technology, Viet Nam National University Hanoi for their teaching and useful help.
I give my special thanks to leaders, colleagues and Mr. Tran Van An at University of Fire Prevention and Fighting for their help, guidance and financial support in the entire thesis completion process. I am grateful to all people in MEMS lab as well as my students in Universisty of Engineering and Technology, Vietnam National University Hanoi and University of Fire Prevention and Fighting for their contribution. I would also like to greatly thank the Vingroup Innovation Foundation (VINIF) - Vingroup Big Data Institute (VinBigdata) for their grant support and encouragement, which help me overcome financial problems and difficulties.
Last but the most important, I would like to thank my parents, my brother, my sister-in-law because their comfort and support are the power for me going to success. “This work was supported by the Domestic Master/ PhD Scholarship Programme of Vingroup Innovation Foundation”. Ha Noi, 16" January, 2022 CONTENTS 0 0)60500 10117. 4 LIST OF ABBREVIATIONS.LnH* HS HH HH HH TH HH key 7 LIST OF FIGURES.-- Ánh HH HH nọ TT HH Hà HH nh nh n 12 INTRODUCCTTON.
The purpose Of th€S1S. -- - c1 TH TH HH ng 19 3. Objectives and Scope of the 'TH€S1S. Scientific significance and Contributions of the Thesis.
G0 HT HH Ti hệt 23 CHAPTER 1. OVERVIEW OF THE RESEARCH. HH HT HH HH HH HH, 25 1. Related Studies on Injured DetectIOIi.
Related Studies on Indoor Positioning. The challenges in study on injury detection and indoor positioning. SYSTEM DESCRIPTION, SENSOR ERRORS ELIMINATION AND MAP PROCESSING. -- G1 TH Họ Họ ng 31 2.
Sensors Errors ElimInatiO.-- ---- s + xxx vn ng ng iệt 33 2. The 3-DOF AccelerOImef€T. The Magnetic S€nSOF. S123 SH HH ng net 35 2.
Map PTOC€SSITE.- -- G1 HH HH ni 38 2. HH HH HH HT HH HH HH nh gà 43 "“ hoam a. DEVELOPMENT OF A METHOD TO DETECT INJURED FIREEIGHITTERS. Fall Detection Method.
so HH ng Hết 46 3. Fall Detection Module. Post-fall Recognition Module. The Posture Recognition EstimatiOn.
_ The Vertical Velocity EstimatiOn. Injury Detection for On-Duty Firefighters. | The Proposed Fall Detection Algorithm for FEirefighter. The Proposed Loss of Physical Performance Detection Algorithm for 2i 0.
The CO Detection Algorithm. Result and IDDISCUSSIOH. HH HH nh ệt 61 3. The Experimental ResuÏ(§.
Fall Detection ResuÏtS. Loss of Physical Performance Detection. The High CO Level Alerting Algorithm. The Comparison on the Experimental Data.
The Comparison on Public Datasets .- --- 5+ 5<<<<<<se+sss 71 Kha. DEVELOPMENT OF A METHOD TO TRACK ON- DUTY INJURED FIREFIGHTERS. The Step Counting Method.-- ---c s11 1v vn vn kg kg rưy 80 4. Step Length ESfImafIOT.
_ The Proposed Method. Results and DISCuSSIOH. Turning Time and Direction ESfImafIOT. Turning Time Estimation.
Turning Direction Estimation. Vertical Position Estimation. G1 HH HH tp 124 CHAPTER 5. INDOOR FIREFIGHTER POSITIONING AND TRACKING USING MULTI-SENSOR DATA FUSION AND MAP MATCHING ALGOIRITHM.
Combining Data Fusion and Map Matching to Detect Indoor Position. The Scenarios T€S(Ing.-- Sn St HH ri, 126 5. 135 CONCLUSIONS AND FUTURE WOIRK. 65G cĂS se ĂSsSSesee 136 LIST OF PUBLICA TIONNS.0 0600080080906 138 THE RELATED PUBLICATIONS .0009 000800600906 141 LIST OF ABBREVIATIONS 3-DOF | Three Degrees Of Freedom | _ADLs| Activities ‘of Daily Living mm „CO _ “Carbon Monoxide 7777777777777.
_CO; - 'Carbon Dioxide 77mm _COHb|BloodCarboxyhemoglobin | -—FFT _ ‘FastFourier Transform mm GP |Global Positioning System CCé;SCsS ~ 1OS _ | iPhoneOperating System (Apple) = sti‘; ;CSCSTM*” cóPC Vinter-Integrated Circuit 777mm "IC. |IncidentCommander CC a IMU [Inerial Measurement Unit 00) ~ MEMS |Micro-Electro-Mechanical systems | ~ NFPA|National Fire Protection Association ị SỈOADs _ “On-Duty Activities T777. sócPAA _ “Piecewise Aggregate Approximation 77m SỐPASS _ ‘Personal Alert Safety System 7777777777777. _ ppm = |PartsPerMillions = 77C | aRMS _ “Root Mean Square nnn "0 receiver operating characteristic curve curve ị a SAX |S ymbolic Aggregate approximation 7777 _ SCBA|Self-Contained Breathing Apparatuses CS cócSpO; |SaturationofPeripheralOxygn | soSVM_ “Support Vector Machine ns wUFPE | University of Fire Prevention and Fighting =—=—Ss=~<CS SỐ|| AAAaAaaAaä vs U.
Occupational Safety and Health Administration Code of Federal | Regulations | _ Acc | Accuracy as —_FN - “False Negative 00s FP |FalsePositive = HHaaaaaäa _ Sen _ ‘Sensitivity T777. "TN - “HN Negative CỐ —_ TP ITRWPSHW LIST OF SYMBOLS AND THEIR MEANINGS SYMBOLS MEANINGS mm. Ô The Acceleration Of Gravity On The Surface Of The | Earth At Sea Level | |_——---------- nnn nnn nn ed The Accelerations Measure In Positive (a*) And| Negative (a~) Of Ax, Ay And Az Axes |_——-—-—--—- nnn nnn ed The Accelerations Measure In Positive (a*) And | Negative (a~) Of Ax, Ay And Az Axes After Multiple | With K Factor. ÔÔÔ The Root Mean Square (Rms) Of Acceleration Along | Ax, Ay, And Az Axes At Time t |~~~~~~~~~~~~~~~~~=~~~~~=~~~~~~~~~~=~~~~~~~~=~=~~~~~~~~=~=~=~~~~~==~=~=~===~====~=~=~=z========~=~=====================~===~e.ÌÔ Threshold Until It Exceeds The UFT Threshold.
LFT Lower Fall Threshold ị -~' UFT |UpprFalThehold | 6|The Angle Between Ay AndGravity tS v |Vertical Velocity | — | The Threshold To DistinguishBetween“The Rest And ehreshold The Active States | UnS. "The Upper Threshoid ~——=S*~<“<~*~*~<“<~S~S~S UeThe Upper Threshold to check the post-fall condition Ly The Lower Threshold to check the post fall condition ee The Theta Angle ~=~=~=~*SC*CS*~*~“C~<—S~S — YPRÍThe Yaw, Pitch,AndRoll Angles sts—S ~The UpperThreshold To Check“Loss” OF Physical: “mov Performance Condition | The Lower Threshold To“Check Loss OF Physical. “man Performance Condition | xaAattitudeC The Altitude Variations =S=SOS*~=C“—~*~=*~“—~*~*~‘—~s~s~*~S Ay 7 |[ThAtuteChngEe ~ Ef | The Minimal Number Of Samples tisị Mmxwwin_size — - ‘The WindowSie Ts The Average Time Period To Perform Each Step — Averp |The Average DynamicThresholding = - Th) | Fluctuated Value~=~SC*~C~<“<—s~<—S~S ' Den | The Difference Between Acc(j) And The Gravity ` Acceleration Tp The Time Range Between TwoNeighboringPeak "treakix, ANd tpeax, | The Time Of Peak(I+l)AndPeak() | 8 The Similarity Between Two Neighboring Peaks ””” Kg Kilogam i(ai‘“‘:OCOO!OOOO 10 E The Number Of The Estimated Steps T The Number Of Reference Steps 11 LIST OF FIGURES Figure 0. US firefighter injuries by type of duty during 2015 [45].
The block diagram of the proposed sySfem. The recorded data with and without using the simple Kalman filter. The Magnetic fields along Ax, Ay and Az before and after using the simple Kalman filter. The signal from the magnetic sensor before and after calibration.
The altitude signal comparison between with and without using the simple €1 0ï: 1. The position of CO sensor on the masK.---- «+ +sss++s<++sss++sss+ 38 Figure 2. The binary image of a floor used in experimental testing. The map simplification aÏgOr1thim.
The Erosion and Dilation operations [17] .-«---««++<<++see++es++ 41 Figure 2. The result of applying Erosion and Dilation with structuring element size of (7 7) 0i00‹ 0i i0. The result of applying Erosion and Dilation with structuring element size of (4 x 4) for keeping windows and S{21TS.- - c1 vn rey 42 Figure 2. The Map simplification achieved after using Erosion and Dilation OPCTALIONS 2277777.
The floor size detected after applying the flood fill algorithm. The unclosed floor Structure. The floor size detected after applying the dilation operation and flood FUN ANGOLA. The proposed fall detection algorIthm.-- --- «+ ++-ss+++se++seexsssss 47 Figure 3.
An example of a fall event and the UFT, LFT and tFE thresholds. The time cycle of 6 walking Steps .-- - 5 vs set 49 Figure 3. The different states of the user along three axes Ax, Ay and Àz. The values of acceleration, Velocity and Theta angle of a volunteer when transiting between activities: standing — walking — sitting — walking — lyIng.
The injury detection algorithm for firefiglhf€TS. The proposed fall detection algOr1thim. -- «<< £+se+se+sessesses 56 Figure 3. Firefighters move through the narrow paths or SpaC€S.
The proposed loss of physical performance detection algorithm. The high CO level alerting algOr1thim. 5 5s ss>+xesseese 60 Figure 3. The volunteer is carrying the support device in his trouser pocket in the crawling state viewed from a side (a) and from above (Đ).
(a) The RMS of acceleration of a fall forward from standing, first impact on knees; (b) the theta angle; (c) the pitch and roll angÌes. The loss of physical performance because of the accident (crawling then falling); (a) the RMS of accelerometer data; (b) the barometric data. The loss of physical performance because of moving up in an elevator; (a) the RMS of accelerometer data; (b) the barometric datfa. a) Testing and measuring the CO level in the fire; b) the measured CO MU 4.