VNU HO CHI MINH CITY UNIVERSITY OF INFORMATION TECHNOLOGY FACULTY OF COMPUTER ENGINEERING DOAN VAN NGHIA- 18521139 GRADUATE THESIS NGHIEN CUU HE THONG HO TRO DIEU TIET CAM XUC BANG ANH SANG Study on light-based emotion regulation system ENGINEER OF COMPUTER ENGINEERING HO CHI MINH, 2022 VNU HO CHI MINH CITY UNIVERSITY OF INFORMATION TECHNOLOGY FACULTY OF COMPUTER ENGINEERING DOAN VAN NGHIA- 18521139 GRADUATE THESIS NGHIÊN CUU HE THONG HO TRỢ DIEU TIẾT CAM XUC BANG ANH SANG Study on light-based emotion regulation system ENGINEER OF COMPUTER ENGINEERING SUPERVISOR PhD. Doan Duy HO CHI MINH, 2022 THESIS COMMITTEE INFORMATION Thesis committee, established under decision no 528/QD-DHCNTT dated 19/07/2022 of the Rector of the University of Information Technology. ACKNOWLEDGMENT I must first acknowledge and thank my family for their constant support and guidance. Next, to the faculty and staff here at the University of Information Technology - VNUHCM: I sincerely thank Dr.
Doan Duy, my advisor, for guiding me through this thesis at the UIT. You have taught me to consider logical reasoning, and critical thinking as well as other useful skills and knowledge. I thank my team members who have all played a role in the development of our research work: Nguyen Nhat Minh, Tran Thi Duy An and Dang Le Ngoc Tram. In the end, I just want to thank you for everything, for the knowledge, and for the beautiful time at the university.
Thank you! Ho Chi Minh, June 27", 2022 Doan Van Nghia Student at Faculty of Computer Engineering TABLE OF CONTENTS ACKNOWLEDGMENT --------~-----~--~-~-~~~=~~~~~~~~=~~~==~~=~~=~==~==~========~~r I TABLE OF CONTENTS --------~--~--~~~~~~~~~~~=~~=~~=~====~=~~~~=====~=r~==~====~=m~~r II PICTURE LIST --------~----~--~-=~==~==================================r==================~= IV TABLE LLISTT----------~--~--~~=================================r==r==r==r=rr=r=====r===r====mr VI LIST OF ABBREVIATIONS ----------------------------------------- 2-2-2252 2 oo nen ene VII ABSTRACTAm cưrkcơ7c7.7Ắ7Ắớ7ỸŸ7ỶẮĨỗĨÏĨŠẼŸẼF®ễ<ễ<7ỖÖẮÖ7Ð7Ÿ7“Ÿ7Ÿ7Ÿ7Ÿ<Ÿ.ŸƑŸẶŸ7Ặ7ẶŸ7Ặ7Ặ7ỄÈ7ỄÈ7È7ˆ7ˆ7È7È.ợ,,;Xựgg VIII CHAPTER |. OVERVIEW -----~--~---~--~-=~==~==============r====r==r=rr=r==r==r====r=mmr 1 1. Objectives of the topic ------------------------------------------------------------------- 3 1.1 Purpose -------------------------------------------------- 2-9 2-22-00 2-22-22 n n-ne nn 3 1.2 The meaning of the topic --------------------------------------------------------- 3 CHAPTER 2. Human emotions through facial expressions ----------------------------------------- 4 2.4 Fear -----------------------------2 25-22-2252 nnn nnn nnn nnn nnn nnn nnn nnn ncn 9 2.5 Anger----------------------==-=======rrr=rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr=rrrrmr=mer 10 2.
Some factors affect emotions --------------------------------------------------------- 12 2. Light and human emotion ------------------------------------------------------------- 14 2.1 How does light affect humans? ------------------------------------------------ 14 2.2 Regulating human emotion based on light ----------------------------------- 17 2. Emotions and changes in heart rate -------------------------------------------------- 18 CHAPTER 3. SYSTEM ANALYSIS AND DESIGN ---------------------------------- 20 3.
Facial expression recognition --------------------------------------------------------- 20 3.1 Convolutional Neural Networks (CNN) -------------------------------------- 21 3.2 Basic layer of CNN -------------------------------------------------------------- 21 3. Data---------------------------------------------2 2222-2 n nn nnn nnn nn nnn nnnn nnn n nnn nn ncnnee 25 3.1 Data overview ------------------------------------------- 2-92-2022 2 oon nnn n nnn 25 3. System architecture Overview--------------------------------------------------------- 29 3.1 Basic emotion detection layer-------------------------------------------------- 30 3.2 Decision making layer ---------------------------------------------------------- 32 3. Algorihm of the system ---------------------------------------------------------------- 33 3.
Hardware resources -------------------------------------------------------------------- 36 CHAPTER 4.2 Accuracy and Confusion matrix ------------------------------------------------ 38 4.3 Result of the model ---------------------------------------------------------------- 38 4.3 Product Test Scenario ------------------------------------------------------------------ 42 CHAPTER 5.2 The product and application ---------------------------------------------------- 43 5. Future work --------~---~-======~~====================~=====r==r~==r==r==r===r==rr=rrer 44 5. ConcluSIOR------~---~--=====================================r==r======r==r===r==rr==rer 44 REFERENCES------~----~--~~-~~-~~=~~======================z===============r=============me 45 Il PICTURE LIST Fig 1. 1: Rate of depression in V1efTiam.- «+ + + +31 1E vn ng tre 2 Fig 2.
1: Circumplex model of the human emofiOII.-- -- + + ++*£+s£es+seess 5 Fig 2. 2: Happiness CMotions 8ẺẺn.-- cv TH TH TH HH HH TT nh nh nành 6 Fig 2.5 1H HH HH ng nh 7 by.-- G (+13 HH TT HH 10 Fig 2.- G5 5 1190 930 E91 991 90 nh ng ng tr 11 Fig 2. 8: Example circadian rhythms. LH "TH TH HH HH 16 Fig 2.- s1 vn ng Hinh 17 Fig 2.
12: Diagram for changing ©OfIOH. 1: A regular 3-layer Neural NetWOTK. 2: Illustrates the CNN process from input to Output Data. Image is taken from Slide 12, Introduction to Convolutional Neural Networks(Stanford University, 2018).
esccersees isc saees TIO cv essccesscnssenscsnsensevsessnesensonsseaseeneranevssssneonasenes 22 Fig 3. 3: Example feature extraction-edge detecfION. 4: Example of the kernel and convolutional operation. 5: Example of Max Pooling and Average Pooling.
6: FER-2013 train afS€K. c1 vn TH HH net 26 Fig 3. 8: Vietnamese dataset OV€TVICW. HH HH TH HH HH 27 Fig 3.9: Example of Vietnam dafS€K.--- - G HH ng nh 28 Fig 3.
11: Circumplex model for classes Of emOfIOI.---5 5555 +++++sse++sss+ 30 Fig 3. 12: Diagram of the basic emotion detection ÏaY€T.-----« «<< s++ss++ 30 IV Fig 3. 13: Diagram emotion €ÌaSSIÍCAfIOTI. IS An) ca.
15: Diagram initializing pÌaS€.- 5 (+13 1 91191 91 vn ng ng nếp 34 Fig 3. 16: Diagram detecting emotion phase. 17: Diagram control light and sound phase .--- -- «5555 «s++<<++se+ssess 35 Fig 3. ng HH 39 Fig 4.
2: Accuracy of model train on FEjR20 13. -- + £sx£sveseeeseeseeseree 40 Fig 4. 3: Vietnamese and FER2013 dataset .-- 5 G1 gi re, 40 Fig 4. 4: Accuracy of model train on Vietnames and Fer2013 dataset.
42 TABLE LIST Table 2. 1: Emotional Heart Rate SfafISEICS. 1: Datasets OV€TVICW. HH re 25 Table 3.
2: Raspberry SD€CIÍTCAfIOH. s--s< + thưa Hà nhà Hà ngườt 36 Table 3. 3: Camera SD€CIÍICAtIOT.é*EEE+kSEEtErrtEkrEkikrrtkkrrkrrrrrrrrrrrrrrrkee 37 Table 3. 4: Led strip SD€CIẨICAtIOH1.
1: Confusion matrix of model train on FER2013. 2: Confusion matrix of model train on Vietnamese and EFER2013. 41 VI LIST OF ABBREVIATIONS Abbreviations Original CNN Convolutional Neural Networks CPU Central Processing Unit HPA Hypothalamic-Pituitary-Adrenal axis RAM Random Access Memory SAM Sharpness-Aware Minimization ipRGC Intrinsically photosensitive retinal ganglion cells SCN Suprachiasmatic nucleus Vil ABSTRACT The proportion of people with depressive disorder, stress at work, life, people who use stimulants, negative thinking, and feelings are on an increasing trend over time, which will leave unpredictable and bad consequences for people's physical and mental health, and negative impacts on social life. Some studies have shown that light can make us feel happy, calm, and put people undergoing treatment for emotional and behavioral disorders in a better mood.
Light also affects sleep to help us sleep better, reduce stress, anxiety, and regulate breathing. Research into a light-based emotional support system can help users improve their physical and mental health and become more active. In terms of architecture, the system has 3 layers, respectively, the basic emotion detection layer to detect and classify emotions, the decision-making layer to script the music, and the actuator control layer to control the speakers to play. music from the script.
After the research, it is possible to evaluate the system's effectiveness, therefore developing and deploying applications in the smart home as well as hospital. Abstract Last decade, the pressure from society on individuals increases, and we often have to face a series of pressures and instability. Unexpected, undesirable events can lead to a long-lasting, serious both physical and mental impairment that reduces the quality of life. In high school, we have to face the pressure of scores, taking exams to get into college, competing for scholarships.
After graduating, we want to get a good job and affirm our position in society. At the office, we rush into work tasks, salaries, and job positions. In the family, which is the most peaceful place, we are also affected by the expectations of our loved ones. Even when there is no one, we still have to face the urge to be more beautiful, better, etc.
Pressure, depression, bad mood is not only created socially but it is also in our internal emotion. Facebook, Twitter, Zalo. these platforms not only make us happier, but also makes us sad, lonely, and lost in the virtual world, which get our bad expressions and moods appear. A researcher by the University of Michigan in 2015 showed that boredom, and frustration in young people is directly proportional to the time they spend wandering on social networks.
Another study in Korea, which is known as "Internet paradise", shows that most of the online game addicts here are hurt by various types of pressure, and games as a way to escape reality. The examples above are all causes of many problems in the psychological or emotional state of people. If these emotions or psychology are not treated or relieved but left to accumulate over time, it will lead to bad behaviors, manifestations, or moods such as sleep disturbances, thoughts of suicide when we can't find a solution or control emotions such as irritability, anger and negative illnesses such as bipolar disorder, neurological diseases such as personality disorder, schizophrenia mental illness such as memory loss and dementia. According to the World Health Organization (WHO), the number of people suffering from mental disorders has now exceeded 300 million, about 4.4% of the world population.
And depression is the leading disease, causing more than 800,000 suicides each year. In 2020, this is the second most dangerous group of diseases, after Doan Van Nghia 1 cardiovascular diseases, at the level of affecting human life. According to WHO, more than 3.5 million Vietnamese have to deal with mental disorders and bipolar disorder. 40,000 of them choose to end their lives every year.000 người ty tử ° do trầm cảm mỗi năm 30% ⁄ mac trầm cảm Việt Nam Nguốn Viện sức khổ Bệnh viện Boch Mal Fig 1.
1: Rate of depression in Vietnam[20] To be able to solve the problems of human emotions, negative psychology, and treat those diseases, many studies in the field of psychology have shown that music and light is a necessary part, which is the link between people and people. Besides that, light also reflects different aspects of reality firstly through exploiting the inner world, thinking, and expressing human emotions and feelings. Several studies have supported the benefits of light therapy for people with depression, anxiety, and severe work stress. Among the results noted in music therapy include: reduce muscle tension, increase self-esteem, reduce anxiety, strengthening interpersonal relationships, increased momentum, successful and safe emotional expression.
Because of the importance of emotional regulation, the harmonious interplay of rhythm and melody stimulates all of the patient's senses, helping them feel relaxed, calm and steady in their breathing and heart rate along with bodily functions. To partly reduce the pressure from many sides. especially smart solutions, and music therapy to care for patients with psychological and emotional diseases, the research Doan Van Nghia 2 and implementation need to be done. A system that plays the role of assisting users in regulating their emotions is necessary.
Objectives of the topic 1.1 Purpose Research and implement architecture support for emotion regulation based on light. Research and learning technique emotion regulation based on light. Build an architecture for an emotion regulation system. Implement an architecture to test the applicability of the emotion regulation solution.2 The meaning of the topic This thesis can be applied to smart home or hospital, support for treating patients who are facing with negative emotions, helping therapists exploit all the features of music (emotional, physical, social, spiritual, spiritual, aesthetic), and support patients to improve their physical as well as mental health.
With a variety of positive musical experiences as well as the light that are repetitive, improvised, and easily receptive, patients can develop perceptual, cognitive, motor, perceptual, social, and optimistic abilities and be more open-minded. In the psychology field, this system has the effect of calming, reducing pain, relieving stress, preventing aging, supporting cancer treatment, stabilizing heart rate, regulating blood pressure, balancing breathing rate, etc. relieving pressure, psychology, emotions gradually return to more stable of the patient. Doan Van Nghia 3 Chapter 2.
Human emotions through facial expressions The human face is the front of the head, including the eyes, nose, skin, and mouth. All human face is unique and plays important role in individual identity and vision, hearing, olfaction, eating, and breathing are primarily functions of the face. Moreover, the face takes an essential role in communication which express human emotions, one of the most important non-verbal ways we communicate. With 43 different muscles, our faces are capable of making more than 10,000 expressions, many of them tracing back to our primitive roots.