MINISTRY OF EDUCATION AND TRAINING HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION GRADUATION THESIS MAJOR: COMPUTER ENGINEERING TECHNOLOGY OPTIMIZE HIGH-LEVEL SYNTHESIS PROCESSING WITH DARK CHANNEL PRIOR ALGORITHM ON FPGA USING PYNQ-Z2 INSTRUCTOR: PHAM VAN KHOA PHD. STUDENT: TRAN TUAN KIET Ho Chi Minh city, July 2024 HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION FACULTY FOR HIGH QUALITY TRAINING GRADUATION PROJECT OPTIMIZE HIGH-LEVEL SYNTHESIS PROCESSING WITH DARK CHANNEL PRIOR ALGORITHM ON FPGA USING PYNQ-Z2 TRẦN TUẤN KIỆT Student ID: 20119009 Major: COMPUTER ENGINEERING TECHNOLOGY Advisor: PHẠM VĂN KHOA, PhD. Ho Chi Minh City, July 2024 HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION FACULTY FOR HIGH QUALITY TRAINING GRADUATION PROJECT OPTIMIZE HIGH-LEVEL SYNTHESIS PROCESSING WITH DARK CHANNEL PRIOR ALGORITHM ON FPGA USING PYNQ-Z2 TRẦN TUẤN KIỆT Student ID: 20119009 Major: COMPUTER ENGINEERING TECHNOLOGY Advisor: PHẠM VĂN KHOA, PhD. Ho Chi Minh City, July 2024 ii THE SOCIALIST REPUBLIC OF VIETNAM Independence – Freedom– Happiness -------- Ho Chi Minh City, July 05, 2024 ADVISOR’S EVALUATION SHEET Student name: Tran Tuan Kiet Student ID: 20119009 Major: Computer Engineering Technology Project title: OPTIMIZE HIGH-LEVEL SYNTHESIS PROCESSING WITH DARK CHANNEL PRIOR ALGORITHM ON FPGA USING PYNQ-Z2 Advisor: PhD.
Pham Van Khoa EVALUATION 1. Content of the project:. Approval for oral defense? (Approved or denied) 5. Mark: - in words: Ho Chi Minh City, month day year ADVISOR (Sign with full name) iii TRƯỜNG ĐẠI HỌC SƯ PHẠM KỸ THUẬT TP HCM CỘNG HÒA XÃ HỘI CHỦ NGHĨA VIỆT NAM KHOA ĐÀO TẠO QUỐC TẾ Độc Lập – Tự Do – Hạnh Phúc ********** PHIẾU BỔ SUNG VÀ CHỈNH SỬA ĐỒ ÁN TỐT NGHIỆP Tên đề tài: OPTIMIZE HIGH-LEVEL SYNTHESIS PROCESSING WITH DARK CHANNEL PRIOR ALGORITHM ON FPGA USING PYNQ-Z2 Tên sinh viên: Trần Tuấn Kiệt MSSV: 20119009 Chuyên ngành: Công nghệ kĩ thuật máy tính Tên GVHD: Phạm Văn Khoa I.
NỘI DUNG BỔ SUNG (Ghi rõ rõ nội dung bổ sung những mục trong ĐATN) STT Nội dung bổ sung Trang 1. Bổ sung giải thích hình 4.2-2 về sự khác nhau giữa kết quả của ARM 42 và FPGA 2. NỘI DUNG CHỈNH SỬA (Ghi rõ nội dung chỉnh sửa những mục trong ĐATN) STT Nội dung sửa Trang 1. Chỉnh sửa đánh số công thức hình 2.
Đánh số các trích dẫn 2, 3, 4, 5 còn sai sót trong bài 7, 8, 9, 20 3. Điều chỉnh trích dẫn theo thứ tự tăng dần 5, 6, 7, 8, 9, 16, 18, 19, 20 GVHD Tp. HCM Ngày…Tháng…năm (Ký và ghi rõ họ tên) SINH VIÊN (Ký tên, ghi rõ Họ và tên) iv TRƯỜNG ĐẠI HỌC SƯ PHẠM KỸ THUẬT TP HCM CỘNG HÒA XÃ HỘI CHỦ NGHĨA VIỆT NAM KHOA ĐIỆN- ĐIỆN TỬ Độc Lập – Tự Do – Hạnh Phúc ********** Ngaøy …… thaùng……. THỐNG KÊ CÁC NỘI DUNG BỔ SUNG VÀ CHỈNH SỬA ĐỒ ÁN TỐT NGHIỆP Tên đề tài: OPTIMIZE HIGH-LEVEL SYNTHESIS PROCESSING WITH DARK CHANNEL PRIOR ALGORITHM ON FPGA USING PYNQ-Z2 Tên sinh viên: Trần Tuấn Kiệt MSSV: 20119009 Chuyên ngành: Công nghệ kĩ thuật máy tính Tên GVHD: Phạm Văn Khoa STT THÔNG TIN BỔ SUNG /CHỈNH SỬA TRANG (Ghi rõ mục tên bổ sung/chỉnh sửa những mục gì trong ĐATN) 1.
Bổ sung giải thích hình 4.2-2 về sự khác nhau giữa kết quả của ARM 42 và FPGA 2. Chỉnh sửa đánh số công thức hình 2. Đánh số các trích dẫn 2, 3, 4, 5 còn thiếu trong bài 7, 8, 9, 20 4. Điều chỉnh trích dẫn theo thứ tự tăng dần 5, 6, 7, 8, 9, 16, 18, 19, 20 5.
GVHD ký tên SV ký tên v DISCLAIMER The views and opinions expressed in this thesis are those of the author and do not necessarily related the official policy or position of Ho Chi Minh city University of Technology and Education. The information provided in this document is for educational and informational purposes only and should not be construed as professional advice. The author makes no representations or warranties, express or implied, about the completeness, accuracy, reliability, suitability, or availability of the information, products, services, or related graphics contained in this thesis for any purpose. Any reliance you place on such information is therefore strictly at your own risk.
In no event will the author be liable for any loss or damage, including without limitation, indirect or consequential loss or damage, or any loss or damage whatsoever arising from loss of data or profits arising out of, or in connection with, the use of this thesis. The author reserves the right to make corrections, modifications, enhancements, or changes to the content at any time without notice. The author does not assume any responsibility for errors or omissions in the content of this thesis. vi ACKNOWLEDGEMENT First and foremost, I would like to express my sincere gratitude to my supervisor, Dr.
Pham Van Khoa, for their unwavering guidance, support, and mentorship throughout the course of this research project. Their expertise, invaluable insights, and continuous encouragement have been instrumental in shaping the development and successful completion of this thesis. I am also deeply grateful to Dr. Pham Van Khoa for providing me with the necessary resources and facilities to conduct this research.
I would also like to extend my heartfelt appreciation to my senior, Pham Nguyen Hoang Hai, for their valuable advice, assistance, and mentorship throughout this endeavor. Additionally, I would like to acknowledge the collaborative efforts of my friends, Phan Minh Nhat and Nguyen Huy Hoang, who worked with me on a separate thesis project. Their contributions and insights have indirectly influenced and strengthened the work presented here. I am deeply indebted to the Dr.
Pham Van Khoa for providing me with a stimulating intellectual environment that has been crucial to the success of this endeavor. A special thanks goes to my fellow graduate students and colleagues, whose camaraderie, thought-provoking discussions, and willingness to lend a helping hand have made this journey both enriching and enjoyable. Last but not least, I would like to express my heartfelt gratitude to my family and friends, whose unconditional love, encouragement, and unwavering support have been a constant source of strength and motivation throughout this endeavor. Without them, the completion of this thesis would not have been possible.
vii TABLE OF CONTENT CHAPTER 1: INTRODUCTION. 2 CHAPTER 2: BASIC KNOWLEDGE. Dark channel algorithms. Single image defogs using dark channel prior method.
Proposed dark channel algorithms. High-level synthesis. High-level synthesis definition. High-level synthesis technique.
Software, FPGA and ASIC compare. AMD Xilinx Vitis HLS supported library. Vitis Accelerated Library. Vitis Vision library.
AMD Xilinx Zynq platform. Metric assessing output of defog system. Full reference scoring. No reference scoring.
Human – subjective scoring. 21 CHAPTER 3: SYSTEM ARCHITECTURE AND IMPLEMENTATION. PYNQ-Z2 system architechture. ARM and FPGA compare.
PS-PL interface. Implementation on x86 and ARM systems. Development flow to FPGA. Using Xilinx Vitis HLS flow to create IP.
Implementation using Xilinx Vivado Flow. Implementation on FPGA. Vitis HLS IP kernel. Data flow inside each accelerators function.
Data flow in system. System integration on Xilinx Vivado. Upload bitstream file and system usage on PYNQ-Z2. 46 viii CHAPTER 4: RESULT.
Hardware resources used. Defogged images and error percentage. Runtimes and power. Comparison with previous studies.
51 CHAPTER 5: CONCLUSION AND FUTURE WORK. 55 ix LIST OF FIGURES Figure 2.1-1: Fog creation phenomenon to camera or human visual .1-2: Traditional method to find dark channel.1-3: Proposed algorithms to find dark channel .1-4: 3x3 kernel used in filter2D to find dark channel .1-5: Proposed algorithms to find diffusion matrix .1-6: Proposed algorithms to find restore out and output image .2-1: HLS Dataflow structure .2-2: HLS Pipeline technique .2-3: HLS unroll loop technique .2-4: Compare between multi-purpose CPU, FPGA and ASIC .3-1: Vitis Accelerated Library support variety purposes .4-1: PYNQ-Z2 using Zynq 7000 Z-7020 SoC .1-1: Development flow diagram .2-1: PS block interface .2-2: PS-PL interface in Zynq-7000 system .3-1: Evaluation dark channel prior to PS block via Jupiter Notebook on PYNQ- Z2 .3-2: SSIM, Brisque, PSNR metric in software evaluation .4-1: AMD Xilinx Vitis HLS development flow.4-2: AMD Xilinx Vivado implementation flow .5-1: IP_minMat dataflow .5-2: IP_darkChannel dataflow .5-3: IP_diffIm dataflow .5-4: IP_restoreOut dataflow .5-5: AMBA AXI4 is used to communicate between PS-PL .5-6: Dataflow between all 5 IP .5-7: Diagram to implement in Xilinx Vivado .5-8: PYNQ-Z2 implementation after placement and routing .6-1: Allocated memory on PS block .6-2: Control IP usage using Jupyter Notebook on PYNQ-Z2 .1-1: Detailed report hardware resources from Xilinx Vivado .2-1: Defogging with sample 1920x1080 image in ARM/x86 and FPGA .2-2: Difference image between ARM and FPGA .3-1: Power consumption on-chip report .3-2: FPS and power chart with changes frequency. 51 x LIST OF TABLES Table 1: SoC Zynq-7000 Z7020 hardware resources. 26 Table 2: Compare PS and PL block inside Zynq-7000.
26 Table 3: Function transform between ARM and FPGA. 34 Table 4: Paramenter define in Vitis HLS. 34 Table 5: Interface of minMat_accel(). 36 Table 6: Interface of darkChannel_accel().
36 Table 7: Interface of diffIm_accel(). 37 Table 8: Interface of restoreOut_accel(). 37 Table 9: Interface of LUT_accel(). 38 Table 10: Hardware resources utilization.
48 Table 11: FPS and total power. 50 Table 12: Survey all frequency from 100MHz to 200MHz. 51 Table 13: Compare with previous research. 52 xi LIST OF ABBREVIATIONS FPGA Field Programmable Gate Array ASIC Application Specific Integrated Circuit PS Processing system PL Programmable logic HLS High-level synthesis IP Intellectual Property DCP Dark channel prior LUT Look up table HDL Hardware describtion language PSNR Peak Signal-to-Noise Ratio SSIM Structural Similarity Index Measure.
MSE Mean root square TCL Tool command language DRC Design rule check SoC System on chip xii ABSTRACT This graduation project focuses on optimizing high-level synthesis processing with the dark channel prior algorithm on an FPGA using the PYNQ-Z2 platform. The research aims to develop an efficient hardware implementation of the dark channel prior algorithm, which is a widely used technique in image dehazing. The project explores the use of Xilinx Vitis HLS to create IP kernels and integrates them into a system-level design on the Xilinx Vivado platform. The performance of the FPGA-based implementation is evaluated in terms of hardware resource utilization, dehazing quality, runtime, and power consumption, and is compared with previous studies.
The thesis provides a comprehensive overview of the system architecture, implementation details, and the results obtained, demonstrating the effectiveness of the proposed approach in optimizing the dark channel prior algorithm on the FPGA. Keywords: High-level synthesis, Dark channel prior, FPGA, PYNQ-Z2, Image dehazing, Hardware optimization, Vitis HLS, Zynq-7000, single image defog xiii CHAPTER 1: INTRODUCTION 1. Research motivation The rapid advancement of edge computing has enabled a wide range of applications to be deployed directly on edge devices, bringing computation closer to data sources. However, the limited processing power and energy constraints of edge devices pose significant challenges for complex tasks such as image defogging.
Defogging algorithms typically require substantial computational resources, causing significant delays and compromising real-time performance on edge devices. The ability to efficiently defog images in real time is crucial for various critical applications, including autonomous vehicles, surveillance systems, and remote sensor. These applications heavily rely on accurate and prompt image analysis, even in adverse weather conditions, where fog can significantly degrade visibility. Current defogging methods, although effective, often suffer from prohibitively long runtimes when executed on embedded systems, impeding their practical deployment in real-world scenarios.
Hence, there exists a pressing need to develop novel approaches that can accelerate the runtime of defogging algorithms specifically tailored for edge devices. By addressing this challenge, we can unlock the full potential of edge computing in foggy environments, facilitating enhanced situational awareness, improved safety, and reliable decision-making of AI.