VIETNAM NATIONAL UNIVERSITY HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY -------------------- ĐẶNG THIÊN TÂN REALIZATION OF NEURON-SYNAPSE NETWORK-LIKE COMPUTING DEVICE BY NANOMATERIAL NETWORK Major: Engineering Physics Major code: 8520401 MASTER’S THESIS HO CHI MINH CITY, January 2024 THIS THESIS IS COMPLETED AT HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY – VNU-HCM Supervisor 1: Dr. Pham Tan Thi Supervisor 2: Prof. Hirofumi Tanaka Examiner 1: Prof. Phan Bach Thang Examiner 2: PhD.
Nguyen Trung Hau This master’s thesis is defended at HCM City University of Technology, VNU- HCM City on January 28th, 2024. Master’s Thesis Committee: 1. Huynh Quang Linh 2. Nguyen Xuan Thanh Tram 3.
Phan Bach Thang 4. Nguyen Trung Hau 5. Cao Thanh Tinh Approval of the Chairman of Master’s Thesis Committee and Dean of Faculty of Applied Science after the thesis being corrected (If any). CHAIRMAN OF DEAN OF FACULTY OF THESIS COMMITTEE APPLIED SCIENCE NATIONAL UNIVERSITY HCMC SOCIALIST REPUBLIC OF VIETNAM UNIVERSITY OF TECHNOLOGY Independence – Liberty – Happiness TASK SHEET OF THE MASTER’S THESIS Name: Đặng Thiên Tân ID: 1970500 Day of Birth: February 22, 1997 Place of birth: Binh Thuan Major: Engineering Physics Major code: 8520401 I.
THESIS’S TITLES: Realization of Neuron-Synapse Network-Like Computing Device by Nanomaterial Network. TASKS: Synthesizing and fabricating highly functional Ag/Ag2S nanoparticles (NPs) network with atomic switch behaviors. Demonstrating a preliminary study of Reservoir Computing hardware using Ag/Ag2S NPs aggregation for RC supervised learning tasks. DATE OF ASSIGNMENT: January 2023.
DATE OF COMPLETION: December 2024. Pham Tan Thi, Prof. Hirofumi Tanaka Ho Chi Minh city, 2024. SUPERVISOR HEAD OF DEPARTMENT (Name & Sign) (Name & Sign) Hirofumi Tanaka DEAN OF FACULTY OF APPLIED SCIENCE (Name & Sign) ACKNOWLEDGEMENT First and foremost, I would like to express my gratitude to my supervisor, Professor Hirofumi Tanaka, for his enthusiastic support in all of my Master's studies and research, as well as his perseverance, motivation, and extensive knowledge.
His advice was important to me throughout the research and completion of this thesis. In addition, I appreciate the help received from Asst. Second, I am grateful to the Faculty of Applied Science at Ho Chi Minh City University of Technology (HCMUT), particularly Dr. Pham Tan Thi, my supervisor, for providing me with the opportunity to pursue a Master's degree in Japan through the Double Degree Program.
Additionally, I would like to thank everybody at Tanaka Laboratory who helps and supports me in my daily research. Giving my special acknowledgement to everyone who assisted me with my research and experiments: PostDoc. Saman Azhari, PostDoc. Deep Banerjee, Dr.
Oradee Srikimkaew, Dr. Takumi Kotooka and MEng. Also, many thanks to secretary Ryoko Ikeno and all of the lecturers, staff at Kyushu Institute of Technology for their kind support. And thank you to everyone who has been a part of my life in Japan.
Finally, I'd like to express my heartfelt gratitude to my parents (Dang Duc Thinh and Dinh Thi Tuyen), as well as all family members, for their love, patience, and unending support. I am eternally grateful to them and hope that I have made them proud. i ABSTRACT In recent years, neuromorphic devices have been attracted because of their potential to replicate brain function. Along with that, reservoir computing (RC), a computational framework derived from the recurrent neural network, is suitable for temporal/sequential data processing.
RC were created using several nanomaterials with nonlinear physical properties, which have been studied and reported for low- power computing operation to replace the AI software system. Among them, functional nanoparticles (NPs) are promising because of large-scale and low-cost production. Atomic switch networks, especially Ag/Ag2S NPs networks, are expected to exhibit unique dynamics in such highly functional devices. In the present study, we demonstrated a preliminary study of RC hardware using Ag/Ag2S NPs aggregation, which is in-materio RC, for RC supervised learning tasks.
The RC device was fabricated by drop-casting Ag/Ag2S nanoparticles onto a multi electrodes device and characterized qualities for the physical reservoir to efficiently solve computational tasks. It was confirmed that the device exhibits characteristics required for RC such as high dimensionality, phase shift, nonlinearity characteristics. By utilizing those properties, the Ag/Ag2S device showed the ability to perform RC benchmark task: waveform generation, as well as some RC practical demonstrations such as: supervised binary one-hot classification task of tactile objects data, reconstruction of Boolean logic operations (AND, OR, XOR, NAND, NOR, XNOR) were successfully demonstrated. Furthermore, complex spoken digit classification task also successfully performed by Ag/Ag2S nanoparticles device with averagely 60 % accuracy archived.
The results suggest that the Ag/Ag2S nanoparticles device proved to be used as an in-materio reservoir device, which has an extraordinary potential for further complex supervised learning. ii TÓM TẮT Trong những năm gần đây, phần cứng cho trí tuệ nhân tạo (AI hardware) dựa trên khái niệm tính toán lưu trữ (Reservoir computing_RC) đã trở nên ngày càng phổ biến. Thiết bị RC cho AI đã được tạo ra bằng cách sử dụng các mạng ngẫu nhiên (random networks) có khả năng thực hiện các phép tính phức tạp. Các mạng ngẫu nhiên cấu thành từ các loại vật liệu nano với tính chất điện phi tuyến, đã được nghiên cứu và báo cáo về khả năng tính toán và tiết kiệm năng lượng với tiềm năng thay thế hệ thống phần mềm AI.
Trong số đó, các hạt nano (nanoparticles_NPs) là một hướng nghiên cứu thu hút do khả năng sản xuất quy mô lớn và giá thành thấp. Trong nghiên cứu này, chúng tôi đã thực hiện một nghiên cứu sơ bộ về xây dựng một phần cứng RC sử dụng mạng lưới của các hạt nano Ag/Ag2S. Thiết bị RC được thiết lập bằng cách tạo một mạng lưới hạt nano Ag/Ag2S trên một thiết bị đa điện cực và kiểm tra các đặc tính cần thiết cho RC gồm: tính phi tuyến, khả năng tạo tính hiệu sóng hài bậc cao, cũng như tín hiệu trễ pha. Bằng cách sử dụng những tính chất đó, thiết bị RC Ag/Ag2S đã cho thấy khả năng thực hiện tác vụ tính toán cơ sở của RC: tạo hình sóng (waveform generation task), cũng như thành công trong các tác vụ thực tiễn khác như nhận dạng vật thể dựa trên tín hiệu cảm biến xúc giác, tái tạo các phép toán Bôlean logics (AND, OR, XOR, NAND, NOR, XNOR).
Hơn nữa, tác vụ phức tạp như nhận dạng giọng nói cũng đã được thiết bị Ag/Ag2S NPs hoàn thành với độ chính xác trên 60 %. Những kết quả này cho thấy rằng thiết bị RC hạt nano Ag/Ag2S đã chứng minh được khả năng có thể sử dụng như một thiết bị tính toán RC, và có tiềm năng cao trong việc thực hiện hiệu quả các tính toán phức tạp hơn. iii THE COMMITMENT OF THE THESIS’ AUTHOR I, Dang Thien Tan, hereby declare that the master thesis titled "Realization of Neuron- Synapse Network-Like Computing Device by Nanomaterial Network" submitted to Ho Chi Minh University of Technology is a genuine and original work conducted by myself under the guidance of Dr. Pham Tan Thi and Prof.
Thesis author, Dang Thien Tan iv TABLE OF CONTENTS ACKNOWLEDGEMENT. iii THE COMMITMENT OF THE THESIS’ AUTHOR. iv TABLE OF CONTENTS. v TABLE OF FIGURES.
vii TABLE OF ABBREVIATIONS. Literature survey: in-materio unconventional computing. Outline of research. Ag/Ag2S nanoparticles synthesis.
16 CHAPTER 3 DEVICE CHARACTERIZATION. 21 CHAPTER 4 RESERVOIR COMPUTING DEMONSTRATION. Boolean logic configuration. Spoken digits classification.
46 CHAPTER 5 SUMMARY AND CONCLUSION. 50 LIST OF PUBLICATIONS. 57 vi TABLE OF FIGURES Figure 1.1 Structure comparison between feedforward neural network and recurrent neural network…………………………………………………………….2 Reservoir Computing framework……………………………………….3 Next-generation atomic switching network (ASN) device structure…… 6 Figure 1.4 Result of RC benchmark task: waveform generation for ASN device with Ag/Ag2S nanowire junction…………………………………………………… 7 Figure 1.5 Schematic of disordered Au nanoparticles network reconfigurable Boolean logic……………………………………………………………………….6 Schematic diagram of an AgI-based ASN device, from nanowire junction to chip…………………………………………………………………….7 Flow chart for spoken digit recognition RC task using ASN-based devices……………………………………………………………………………… 10 Figure 2.1 Ag/Ag2S NPs was synthesized following Brust-Schiffrin method…….2 The Ag/Ag2S NPs in-materio RC device realization………………….3 Preparation of Ag/Ag2S powder prepared for X-ray diffraction (XRD) analysis process…………………………………………………………………….4 Transmission electron microscopy (TEM) sample preparation……….5 Current-voltage measurement set up by using a Source Meter……….6 Schematic of an electrical measurement setup used for constructing the RC characterization and carrying out RC task……………………………………… 19 vii Figure 3.1 Synthesized Ag/Ag2S NPs structure characteristics…………………….2 Transmission electron microscopy (TEM) image of Ag/Ag2S NPs…… 21 Figure 3.3 Electrical characteristics of Ag/Ag2S RC device……………………….4 Lissajous plot of output from 15 electrode pads versus input voltage… 25 Figure 3.5 Fast Fourier transform spectrum of output current after applied a bipolar sinusoidal wave…………………………………………………………… 27 Figure 3.6 Impedance spectroscopy of one electrode from Ag/Ag2S NPs device.1 Schematic of waveform generation task……………………………….2 Waveform generation of sinusoidal wave input (11 Hz frequency, peak amplitude 1.0 V AC bias voltage)………………………………………………….3 Waveform generation of sinusoidal wave input (11 Hz frequency, peak amplitude 4.0 V AC bias voltage)………………………………………………….4 Nonlinear-memory heat-map table summarizing the results of comparison test accuracy…………………………………………………………… 36 Figure 4.5 Objects classification task schematic…………………………………… 38 Figure 4.6 One-hot vector binary classification of tactile grasped objects results….7 The schematic of RC task of Boolean logic function optimization…….8 Boolean logic operations task results…………………………………… 45 Figure 4.9 A diagram of spoken-digit classification……………………………….10 Normalized confusion matrix of spoken-digit classification and the dependence of spoken-digit classification on sampling rate……………………… 47 Figure 4.11 Comparison of the accuracy of spoken-digit classification between the software simulation and Ag/Ag2S device…………………………………………. 48 ix TABLE OF ABBREVIATIONS Abbreviations Explanation ANN Artificial neural network ASN Atomic switching network DAQ Data acquisition DI water Deionized water FFN Feedforward neural networks FFT Fast Fourier transform FSDD Free spoken-digits dataset IPA Isopropyl alcohol I-V Current-voltage measurement NMSE Normalized mean square error NPs Nanoparticles RC Reservoir computing RNN Recurrent neural network SEM Scanning electron microscopy SET Single electron tunneling TEM Transmission electron microscopy XRD X-ray diffraction V-t Voltage-time measurement x CHAPTER 1 INTRODUCTION 1.
Research motivation The von Neumann architecture performance was remarkably improved by downsizing and increasing the number of transistors pursuant to Moore’s law [1]. However, recently, the number of transistors on a chip were reached the limit and got more difficult for further improving computation performance. Simultaneously, advances in the development of artificial neural networks necessitate the creation of new chip platforms capable of supporting them with high power efficiency [2]. To address such issues, neuromorphic computing was proposed.
Neuromorphic device is an integrated circuit that replicate the structure of live neuron cells, particularly for brain simulation, which initially coined in 1990 by Carver Mead [3]. The human brain outperforms supercomputers in terms of computing power due to its fast signal processing speed and low power consumption [4]. Human brain is composed by billion neurons are connected and transmit information via synapses. Synapses, which construct a small gap between neurons that plays an analog logic and learning role in a neural network [5].
Synapses in a brain network transmit electrical signal from presynaptic to postsynaptic neurons to create a decision. This process could be reproduced in an electronic device, allowing us to potentially imitate the operation of the brain by replicating the essential functions of biological synapses [6]. Unconventional computing refers to computational frameworks that are inspired by the dynamics of natural systems, such as biological brains, to accomplish transformational advancements in cognitive technology [7]. In order to overcome the restrictions of traditional materials and procedures, researchers investigating "non- von Neumann" solutions, that is expected to have a low power consumption and the ability to handle increasingly complex problems.
Reservoir computing Artificial neural networks (ANNs) are computational models that simulate neurons and their networks. An ANN is made up of three layers: input layer, hidden layer, and output layer. They are represented a neuron-like network interconnected via binding weights between layers, known as synapse-like weighted links, which indicates the strength of the connections between neurons.