VIETNAM NATIONAL UNIVERSITY HOCHIMINH CITY UNIVERSITY OF INFORMATION TECHNOLOGY ADVANCED PROGRAM IN INFORMATION SYSTEMS TANG QUOC HUNG - 19521583 VO NGUYEN DANG KHOA - 19521704 BACHELOR OF ENGINEERING IN INFORMATION SYSTEMS THESIS ADVISOR Ph. LE KIM HUNG HO CHI MINH CITY, 2023 ASSESSMENT COMMITTEE The Assessment Committee is established under the Decision ¬—. by Rector of the University of Information Technology. ACKNOWLEDGMENTS For this graduate thesis, first and foremost, we would like to express our most profound appreciation to our supervisor, Ph.
Le Kim Hung, who generously took his precious time to guide and pass on his experiences to me. His immense knowledge and plentiful experience have encouraged us all the time in my academic research and daily life. We feel very fortunate to have had the opportunity to enrich our knowledge with him, his guidance, and suggestions have provided us with much-needed insight into this thesis. Without his tremendous support and advice, it would be impossible for us to complete this thesis.
Additionally, we would also like to thank all the lecturers and teaching assistants at the University of Information Technology — Viet Nam National University — Ho Chi Minh City, especially the lecturers in the Faculty of Information Systems who have imparted invaluable knowledge to us during these four years here; and MSc. Thai Huy Tan, who has supported and imparted invaluable knowledge to us throughout the thesis period. Our appreciation also goes out to our family and friends for their advice and support throughout our studies. This is a significant turning point in our education and development, in our opinion.
We will do our best to make the best use of the abilities and knowledge we've gained in the future, and we'll keep working to get better. We want to express our sincere gratitude to everyone for their assistance over the years once more. TABLE OF CONTENTS œ4Í-Ìk› ACKNOWLEDGMENTTS. --- << S2 HH Họ SH i LIST OF FIGURES.
Họ Họ iv LIST OF TABLES. Họ in vi ABSTRACTT. HHTH IE vii Chapter 1 IntrOdUCÏOT.-- << Ăn TH ng ng 1 IS. Scope of the StUỦY.- Sàn TH TT TH TH TH TH HT HT HH nghi 3 1.
-s- 5< tk 19v HH TH HH gu4 1.- ó5 6 11391 E91E 1 91 1931 1 1n HH ng ung5 Chapter 2 Literature r@VÏ@W.------ 5c nu nh my 7 2.1 Evolution of Rainfall Prediction Model s.2 Identified Gaps and Research ÏNeedS.- ----- 6+ x2 9 2 1 xe 8 Chapter 3 Theoretical backgrOUnid .1 Meteorological Data and Rainfall PredICtiOn.1 Significance of Meteorological ÏDatia.- «sex kh 1t nung ri, 10 3.2 Exploring Traditional Rainfall Prediction Models .2 Statistical Methods in Rainfall PrediCfIOï.1 Regression Analysis: Unveiling Model Nuance .2 Temporal Insights: Time Series Analysis. Advanced Numerical Weather Prediction Models.1 Dynamic Simulation Models: Atmospheric Dynamic Simulafion.2 Challenges and ÍnnOVafIOTNS. ¿+ + + xxx 191v vn TT HH nh net 11 3.4 Empirical Models: Learning from Historical ModelÌs.1 Analyzing Historical RelationslhIpS.- --- c5 65x Sx 1# EE* kg ggnrin 11 3.2 Integration with Machine Learning.5 Machine learning and Deep neural netWorkS .-- -- + xxx vn HT ng HH net 12 3.2 Deep neural n€fWOTKS.--s- «vn HT HH TH TH HH ng rà 16 3.6 Technical Frar€WOTKS. HH ng TH HH ng 24 3.
«kh TH TH TH TH HT HH TH ri 24 3. 25 il Chapter 4 Proposal apprOaCh.--- << HH HH HỌC HH HH HH HH 28 4.2 Rain Detection model based on numerical dafaSet. Rain Detection model based on satellite image daftaset.------ 5+ <<s2 32 Chapter 5 Setup and r©SUIÏ. --- -- -- Ă 5S sọ ng vn 37 h2.
nh ốc ae .1 Rain detection đafaSeK. - hàn TH HH HH Hy 5.3 Experimental ITI€ẨTIC.v TH TH TH cư 5.2 Experimental result hố.1 Classification performance on numerical dataset .2 Classification performance on satellite image dafaS€(.-- 5-5555 £+x+erseexe 48 Chapter 6 ConCÏUSỈOIN. -- «5 Ho họ HH nh 51 6.--- SG Q nọ họ Họ 54 APPENDICES. On H51 c5 00c nh TH nh Hà nh TH 11 0 TH 10m 57 A.
Set up OpenWeatherMap APÏ,. Set up Sentinel Hub OAuth clients .---- + + 65s +£+E+vEsEssseseerseekrsrrke 62 11 LIST OF FIGURES cs Le Figure 3.1 Machine learning process flow [7] .2 Decision Trees Architecture [ Ï 3] .--- ¿5xx x+kEsk+kEvkekekEskekrekrkrkereeree 15 Figure 3.3 Random Forest Architecture [ Í S],.4 Artificial neural networks and Deep neural networks [18] .5 Multi-Layer Perceptron [20].6 Convert Feed-Forward Neural Network into Recurrent Neural Network [23] —-.7 How Recurrent Neural Network Work [23] .9 One to Many RÌNN. -Ặ LH HH HT HT HH TH re 20 Figure 3.10 Many to One RÌNN. 112112111 HH 21T H1 HH HT HH HH 20 Figure 3.11 Many to Many RNN .- cà SH HH HH HH TH HH 20 Figure 3.12 Simple 1D CNN architecture with two convolutional layers [26] .14 VGG Neural Network Architecture [3 Í ].- --- Set 921191 91 911 111 1x gu ngư, 23 Figure 3.2 OpenWeather Main Site .3 Sentinel Hub logo [36 ].4 Sentinel Hub Main S1f€.- ----- +5 1x19 91T 91g HH ng ngư 27 Figure 4.1 The architecture of the rain detection dataset framewOrk.1 CNN-MLP model ArChIf€CfUT,.
(G625 1921191 91 91 111191 vn re, 29 Figure 4.1 Vgg16 model architecture. eee ee eseseesesseeeseeeeseseeseseeecseeesaeessesseesaeeees 34 Figure 5.1 The first 27 rows of the numerical dataset .2 Data collection on Onedrive.3 Each time we collect there will be 76 IiagS.-- ----- 555 5+++e>+c+esec+eseres 42 Figure 5.4 Folder of input 1InaS .- --- + 5s 1 vn vn TT HT HT TH HH 44 Figure A.I OpenWeather Sign-in S1(€.---- chàng HH TH TH HH 57 Figure A.2 OpenWeather Sign-up SI. --- «vn HT HT TH TH TH TH HH ng 57 Figure A.3 OpenWeather Account COnẨirTatiOT.5 OpenWeatherMap Menu Ba[. --- - 5 5< S4 xxx 1T HH HH1 như 59 1V Figure A.6 OpenWeather Subscribed S€TVIC€S.
--- -- 5 + ke ST TH TH nh re 59 Figure A.7 OpenWeather API KeyS. ee cceseccsseseeseesceceeecseeseeaceceeeeseeseeceeeeseeseeseeaeeeseeneeaeeaes 59 Figure A.8 OpenWeather Purchased Payments .9 OpenWeather Profile ÏnfOrImatIOI.10 OpenWeather Student SponsorshIp.11 OpenWeather Service limitation. --- 5 6 + S323 9E vn re, 60 Figure A.12 OpenWeather Current weather and forecasts collection price .13 OpenWeather Historical weather collection price .14 OpenWeather API key in COdINg. ee eceeeeeeseeceseseeeeseeeesceeeecseeeeaeeesacseeesseeeeaeeees 62 Figure B.1 Sentinel Hub Sign-in S1(€.
- --- -- +5 1v vn vn TH ngư 62 Figure B.2 Sentinel Hub Sign-up Site oe .3 Sentinel Hub Account ConfirmatiOTn.- ----- + + + + Sex re 64 Figure B.4 Sentinel Hub Dashboard S1te.---- +2 + 6+ k x3 vn ngư 64 Figure B.5 Sentinel Hub Usage SI(G.¿- -- Ác 1 911 121111111911 111111 T1 ng nh Hàng, 65 Figure B.6 Sentinel Hub OAuth clients 00.7 Sentinel Hub NoR Sponsored Accounts And Data Collections .8 Sentinel Hub Sponsorship Approved .9 Sentinel Hub Processing DTIC€.--- - - + 5+3 *E*k SE vn n rhnnrrrư 67 Figure B.10 Sentinel Hub Planet Monitoring & Processing Tice .11 Sentinel Hub Imagery price .12 Sentinel Hub expired ©TmaIÏ.- --- + + + + *+xE#EeE+sEeeEeEekreerekeseekeekreerexee 69 Figure B.13 Sentinel Hub API key in COdIng.- -- c5 22 *+tEsE+EvEekeEskekreerkreereeree 69 LIST OF TABLES cs Le Table 5.1 71 provinces and cities Of Viet Naim .2 Numerical Dataset before and after Pre-DrOC€SSInE.3 The first 10 rows of the satellite image table.4 Satelite image dataset before and after pre-prOC€SSINE.5 Server Specifications are used for data colÏ€CfIOH. --- ¿5+ 55+ +s£+sx+sx+xsxs+ 46 Table 5.6 Server Specifications are used for model training and evaluation.1 Model performance comparison on numerical đafaSet .2 Model performance comparison on satellite images dataset .-- 49 vi ABSTRACT Weather forecasting has played a vital role in human life since ancient times, guiding decisions for human daily activities, agriculture, outdoor events, and various plans. The higher the accuracy of weather predictions is, the more easily individuals make informed choices. To enhance the accessibility and precision of weather forecasting, there is a significant contribution of the Application Programming Interfaces (APIs) by enabling seamless communication between applications.
This graduate thesis focuses on leveraging two prominent weather APIs: OpenWeatherMap and Sentinel Hub. OpenWeatherMap offers real-time access to comprehensive weather data, while Sentinel Hub provides access to raw satellite images captured in real-time. The integration of these APIs serves as the foundation for developing a rain prediction model. The primary objective of this research is to harness the capabilities of OpenWeatherMap and Sentinel Hub APIs to enhance rain prediction accuracy.
The model, developed through this integration, aims to provide reliable forecasts that empower individuals and organizations to plan and adapt their activities. The study explores the synergies between these APIs and their collective contribution to the improvement of rain prediction models. The methodology involves the extraction and analysis of real-time weather data from OpenWeatherMap, combined with the utilization of high-resolution satellite images from Sentinel Hub. The integration of these diverse datasets aims to enhance the model's ability to predict rain patterns with a higher accuracy level.
The study's outcomes are expected to advance the field of weather forecasting and contribute valuable insights for applications in diverse sectors. In conclusion, the combination of OpenWeatherMap and Sentinel Hub APIs in the development of a rain prediction model represents a significant step towards more precise and reliable weather forecasts. The research contributes to the broader goal of harnessing technological advancements to enhance our understanding and anticipation of weather patterns, ultimately benefiting individuals and industries alike. Vii Chapter 1 Introduction 1.1 Problem statement The precise and timely forecasting of rainfall is integral to several critical sectors such as agriculture, water resource management, and disaster prevention.
Traditional rainfall prediction models often rely on a singular data source, limiting their accuracy and reliability. In the contemporary era of abundant data availability, there is a growing imperative to explore the potential benefits of amalgamating diverse data sources to enhance the precision of rainfall predictions. The current approach heavily depends on data streams from sources like the OpenWeatherMap API and Sentinel Hub API. However, there is a concern that this approach may not fully harness the wealth of information available from various sources, including weather stations, satellites, and radar systems.
This raises a fundamental question: Can the integration of these diverse data sources significantly elevate the accuracy of rainfall predictions? The limitations of existing models underscore the importance of adopting an approach that leverages synergies among different data sources. Bridging this gap is crucial for the development of a dependable rainfall prediction model that provides real-time information. The overarching objective is to transform decision-making processes in agriculture, optimize water resource management practices, and fortify disaster prevention measures. This research project seeks to thoroughly examine the extent to which the integration of data from various sources can create a more comprehensive and accurate model for predicting rainfall.
By addressing the shortcomings of conventional methods and exploring the synergies among different data streams, this study aims to provide real-world solutions that can significantly mitigate the challenges associated with traditional rainfall forecasting methods. The outcomes of this research have far- reaching implications, offering valuable insights that can revolutionize current practices in fields critical to societal well-being.2 Objectives This research project encompasses several key objectives aimed at advancing the field of rainfall prediction through the integration of multiple weather sources, machine learning, and deep neural networks. The specific objectives are as follows: Enhanced Rain Prediction - Develop arobust rainfall prediction model by integrating data from diverse weather sources, including but not limited to the OpenWeatherMap API and Sentinel Hub API. - Investigate the synergies among various data streams to improve the accuracy and reliability of rain forecasts.
Machine Learning Integration - Implement machine learning techniques to analyze historical weather data and identify patterns that contribute to more accurate rainfall predictions. - Explore the potential of machine learning algorithms in adapting and improving the model's performance over time. Deep Neural Networks (DNN) - Investigate the application of deep neural networks to enhance the model's capacity for learning intricate relationships within complex weather datasets.