VIETNAM NATIONAL UNIVERSITY HOCHIMINH CITY UNIVERSITY OF INFORMATION TECHNOLOGY ADVANCED PROGRAM IN INFORMATION SYSTEMS NGUYEN TUAN ANH GRADUATION THESIS BUILDING A MOBILE APPLICATION FOR DETECTING AND RECOGNIZING INFORMATION OF DRUGS BACHELOR OF ENGINEERING IN INFORMATION SYSTEMS NATIONAL UNIVERSITY HOCHIMINH CITY UNIVERSITY OF INFORMATION TECHNOLOGY ADVANCED PROGRAM IN INFORMATION SYSTEMS NGUYEN TUAN ANH - 18520465 GRADUATION THESIS BUILDING A MOBILE APPLICATION FOR DETECTING AND RECOGNIZING INFORMATION OF DRUGS BACHELOR OF ENGINEERING IN INFORMATION SYSTEMS THESIS ADVISOR Dr. PHAN XUAN THIEN ASSESSMENT COMMITTEE The Assessment Committee is established under the Decision. , date by Rector of the University of Information Technology. - Member ACKNOWLEDGMENTS I would like to thank my family all of them .to my mother the greatest woman Iknow, my brothers and sisters for their love and support for their wishes and prayers.
All gratitude to my thesis advisor Prof. Phan Xuan Thien of the information system faculty at University Of Information Technology. He was there whenI ever need, for his wise directions and his full support and encouragement. Words cannot express my gratitude to my professor and chair of my committee for his invaluable patience and feedback.
I also could not have undertaken this journey without my defense committee, who generously provided knowledge and expertise. And i would to thank all instructor in IS department especially Associate Professor PhD. Nguyen Dinh Thuan.Thanks should also go to the librarians, research assistants, and study participants from the university, who impacted and inspired me. Finally, I must express my very profound gratitude to my friends for providing me with unfailing support and continuous encouragement throughout my years of study and through the process of researching and writing this thesis.
This accomplishment would not have been possible without them. Thank from bottom of my heart! TABLE OF CONTENTS caLeo Chapter 1 INTRODUCTION cL Background .3 Aim of the Study.1 Study ObJ€C(IV€S. cành HH HH Hit 4 4 Significance of the Study.- ¿+ St 1 St 2101 HH1 1 He.6 Overview of the Thesis. ¿xxx St E1 HH0 te.
6 Chapter 2 LITERATURE REV HE ẨW.1 Design and Development of Mobile based Medication classification. Chapter 3 THEORETICAL FRAMEWORK.2 Java Programming Language .- se sesesreteeeeerrirrrrrrer LZ 3.5 Convolutional Neural Networks (CNN).1 Region Proposal Networks (RPN).1 Mask R-CNN Based Pill Inspection Model .10 TensorFlow Object Detection API. - ¿5c S2 SE E1 1101 1g ườn 37 43 Model Development. -- - 5: + 5e St St2kE*‡*SEk‡keEEEEkEkErkrkerrrkrkrrree 38 44 Model Training.
4D Chapter 5 SYSTEM IMPLEMENTATION.5 Near By Hospital places .6 Scan QR Barcode 1 .7 Create Bill Page .--- ¿sen rrrerrrrerc OO.8 bình ÔÔÔÔÔÔÒÔÔỎ Chapter 6 CONCLUSION AND FUTURE WORK 6. e6 Set kề HH HH1 ng ườn 6. 6 St TH KH HH1 HH1 hờn REFERENCES LIST OF FIGURES œ4LHlg» Figure 1-1 Gantt chart of the SÍHUẢY. 55-55 S+S+‡SteStt+Et+E+rtErterererertertrrrrrrrrrrrrrrrrrrrrre 7 Figure 3-1: Android Studio user interface .ccceccsecssecsssscesssssssseseseesessesesseessesessssesseseseeseseenes 11 Figure 3-2: Mobile development framework (Kulathumani, 2015)J18].--- 13 Figure 3-3: : Architechture of Neural Networks [19].
Figure 3-4: Architechture of Convolution Neural Networks [23]. Figure 3-5: Diagram fo Faster R-CNN [26] Figure 3-6: Faster R-CNN Architecture of faster_rcnn_test.-----------+ 19 Figure 3-7: Anchor generation [29] .cscscsssssevesessesssssessesvecneeneeseessesesssssssassseeeneeneensensene 21 Figure 3-8: RPN data ÏOWV.- S55 HH ri 22 Figure 3-9: Rol Pooling Layer. Figure 3-10: Examples of image segmentation and object detection: (a) Input image; (b) Semantic segmentation; (c) Object detection; (d) Instance segmentation Figure 3-11: Mask R-CNN Architechture Figure 3-12: Structure of RolAlign Figure 3-13: Process of the proposed Method .cccccccssessessesvessesvseveesessssssssesssseseeneeneeneeneene 26 Figure 3-14: Result of the pill area detection: (a) Detection result image; (b) Cropped image of instance segmentation. Outer rectangle is a bounding box and inner solid line indicates a detected pill area; (c) Cropped image of detection information consisting of the number of pill, detection scores, and bounding box POSITIONS.
Ăn re Figure 3-15: Process of data labeling and JavaScript Object Notation file creation: (a) Process of data labeling; (b) Structure ofJavaScript Object Notation,. 28 Figure 3-16: Training process of pill detection using mask region-based convolutional neural network. Figure 3-17: Mechanism of Object Detection in TensorFlow Figure 4-1: Dataset Image of Alaxan, Bactidol, Bioflu, Biogesic, DayZinc, Decolgen, Fish Oil, Kremil S, Medicol, and Neozep. Figure 4-4: Architecture of CNN Model for Drugs Classification.----------+ 38 Figure 4-5: Setting Procedure of the SO[ÏWđTE.
- tt êc 40 Figure 4-6: Model Training. HH HH HH HH HH HH he Figure 4-7: Training Process Figure 4-8: Graph Training and validation Accuracy/Loss Figure 4-9: Uploading the model and result Figure 5-1: Some main Function of this app Figure 5-2: Mainmenu Figure 5-3: View of Drug detail on the List of drug On Aatabase. 47 Figure 1-5: Click nagative button Figure 1-6: View of open camera. Figure 1-7: View Of Open VDI 000088666.
48 Figure 5-5: Near By Hospital places.- - ST TH TH HH it 49 Figure 5-6: Scan QR barcode SCF€€H. 55t S£‡ct‡E‡EEEErErertertrrtrrrrrrrrrrrrrrrrrrrrrree 50 Figure 5-7: Bill Page ŠCT€€H. - - kề TT HH HH HH ghe 51 Figure 5-8: The Statistic Screen LIST OF TABLES caLeo Table 1: Project schedule Table 2: Drug datasets, training and testing data Of PTOj€CI. --- ¿+55 c+c+<++35 ABSTRACT To ensure the safe administration of medications to patients, accurate pill identifi- cation is crucial.
The foundation of patient safety is the proper prescriptions being written. It is challenging to prescribe and load medications without a support system, which increases the risk of prescribing errors for patients. This is a significant issue that demands attention in the medical field because it can have a significant impact on the patient's overall health, prolong the healing process, increase medical expenses overall, and even put the patient's life in danger. We provide a solution to that issue that makes use of deep learning to identify pharmaceuticals in order to assist physicians and nurses in appropriately dispensing medications.
This work uses a CNN model baseline deep learning drug identification to explore how identification confusion of similar images by humans arises through the cognitive counterpart of deep learning solutions in the hunt for better image-based solutions to the drug identification problem. We introduce the fundamental ideas behind object recognition models in this study. To find the most effective pill recognition model, we trained each algorithm using a dataset of images of pills and then examined how well the CNN models performed and applied the CNN network model to the drug identification problem on the Android platform. The recent advancement in technology has provided an enabling technique to solve these types of problems by designing and developing an application that can run on smart phones in which patient will find it easy to carry along.
The medication classification application could impact positively on the life of the patient as it will help patients in keeping track of their daily pills as remembering the intake of these prescribed medications could be a matter of life and death. The performance of the model is evaluated by the correct recognition rate and investigated with many different cases. Chapter 1 INTRODUCTION The design, development of the Android application based on the classification of medicine, the study problem description, the study aim, the study objectives, the study significance, and the application limitation are all introduced in this chapter.1 Background Health is riches, according to a well-known proverb. For most people, having excellent health is one of the most important things because poor health can result in a very terrible life (Leonard, 2008).
There are up to 10,000 different medications on the market right now, many of which are LASA medications, and there are constantly more pharmaceuticals entering the market. The US FDA has received over 95,000 reports of drug mistakes since 2000. Drug name confusion resulting from similar looks or readings accounts for about 25% of errors [1]. The Malaysian Ministry of Health also received 5,003 reports of prescription errors in 2011, with LASA medications accounting for around 6% of the incidents.
Most recently for Vietnam, in April 2018, there was an instance when pregnant women were given the incorrect medication at the Health Center of the Tan Phuoc district. Specifically, the pharmacy accidentally gave patients Misoprostol 200mcg for abortion purposes when the doctor had prescribed Miproton 100mg for pregnancy maintenance [2]. Another instance of confusion occurred in the beginning of 2014 when a physician at Binh Chanh Hospital (HCMC) gave a patient Levetiracetam (an anti- epileptic drug) instead of Piracetam (a medication that enhances cell metabolism and supports central nervous system activity) because the two medications are believed to be similar [3]. Drug interactions are not only dangerous for the patient and can even be fatal, but they are also inefficient.
Whether an accident happened or the potential for a harm existed, medication errors are errors in the ordering or delivery of a drug. Adverse drug events can be caused by some prescription errors [4]. A pharmaceutical error is any avoidable circumstance that could result in improper medication use or patient damage. The following recommendation has been accepted as the working definition of medication error by the National Coordinating Council for Medication Error and Prevention (NCCMERP): ".any preventable event that may cause or lead to inappropriate medication use or patient harm, while the medication is in the control of the health care professional, patient, or consumer".
The following activities may be connected to professional practice, healthcare systems, and products: prescribing, order communication, product labeling, packaging, and nomenclature, compounding, dispensing, and distribution [5]. Nevertheless, recent technological advancements have made it possible to solve these kinds of problems in a variety of ways, one of which is by purchasing a robot that is specifically designed to remind the doctor to dispense the right medication for the patient and to help the patient understand how to take the medicine. However, the aforementioned solution appears to be ineffective and expensive (Riehemann et al. Instead, using a mobile application looks to be more efficient because it eliminates the need to purchase a separate device and because the majority of people use smartphones.
The study decided to employ one of the most popular smartphone operating systems, Android, because it is the best in the smartphone industry. However, according to top-tier engineers, Android appears to be quite effective in smartphones (Nosrati, 2012) [7]. The Android operating system was created from the very beginning to enable developers to create compelling mobile applications that fully consider the preferences of each device. Because of this, the suggested mobile application is compatible with smartphones utilizing one of the most widely used mobile operating systems, Android.
Using a CNN model, the program essentially serves to remind doctors or users to take their medications properly and in the proper proportions. Additionally, the suggested method aids in medication differentiation and displays some drug-specific information, such as the drug's name, action, and production date. In order to design, develop, and implement an android-based application for drug classification using Java programming language, CNN model, and some android APIs. The software is made to assist users in getting the most out of their medication while minimizing the chance of forgetting to take a dose or doses at the wrong time.2 Problem Statement For the majority of people, health is one of the most important things because, without it, everything seems to go wrong.
Recently, it has become more common for doctors to prescribe the incorrect medication and for people to utilize medications without being aware of the source. As the number of medications rises year after year, doctors with a limited knowledge base will inevitably become confused regarding color and shape in the absence of product packaging. Medication abuse is a very severe issue because it can impact a patient's general health, delay healing, and raise their overall medical expenses. The CNN model is used to classify drugs into various categories, and several drug APIs are used to provide some information, giving doctors and patients the right source of information to prevent unfortunate confusion.