VIETNAM NATIONAL UNIVERSITY HO CHI MINH CITY HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY FACULTY OF COMPUTER SCIENCE AND ENGINEERING REPORT CAPSTONE PROJECT A VISUAL ART GENERATOR FOR MUSIC ARTISTS MAJOR: COMPUTER SCIENCE COUNCIL : OISP - COMPUTER SCIENCE SUPERVISOR : TRUONG TUAN ANH, Ph.D REVIEWER : MAI DUC TRUNG, MSc SECRETARY : TRUONG THI THAI MINH, MSc —o0o— STUDENT : HUYNH PHUOC THIEN ID : 1952463 HO CHI MINH CITY, 08/2023 VIETNAM NATIONAL UNIVERSITY HO CHI MINH CITY HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY FACULTY OF COMPUTER SCIENCE AND ENGINEERING REPORT CAPSTONE PROJECT A VISUAL ART GENERATOR FOR MUSIC ARTISTS MAJOR: COMPUTER SCIENCE COUNCIL : OISP - COMPUTER SCIENCE SUPERVISOR : TRUONG TUAN ANH, Ph.D REVIEWER : MAI DUC TRUNG, MSc SECRETARY : TRUONG THI THAI MINH, MSc —o0o— STUDENT : HUYNH PHUOC THIEN ID : 1952463 HO CHI MINH CITY, 08/2023 Supervisor Signature Supervisor Signature: Date:. 1 ĐẠI HỌC QUỐC GIA TP.HCM CỘNG HÒA XÃ HỘI CHỦ NGHĨA VIỆT NAM ---------- Độc lập - Tự do - Hạnh phúc TRƯỜNG ĐẠI HỌC BÁCH KHOA KHOA: KH & KT MÁY TÍNH __ NHIỆM VỤ LUẬN VĂN/ ĐỒ ÁN TỐT NGHIỆP BỘ MÔN: CÔNG NGHỆ PHẦN MỀM Chú ý: Sinh viên phải dán tờ này vào trang nhất của bản thuyết trình HỌ VÀ TÊN: Huỳnh Phước Thiện MSSV: 1952463 NGÀNH: Khoa học Máy tính LỚP: __________________ 1. Đầu đề luận văn/đồ án tốt nghiệp: Bộ sinh sản phẩm nghệ thuật tự động cho nghệ sĩ nhạc 2. Nhiệm vụ (yêu cầu về nội dung và số liệu ban đầu): This project requires creating a system to help artists in creating their own visualizer for their performance.
Some processes of the system are as follows: - The waveform will be analyzed to extract the data of the frequency. - The data will put together with some random art installation framework such as p5js (a JavaScript framework), OpenFrameworks (C++). - Based on these two steps, some suitable systems of processing visual art will be built. - The artists can set up their theme for their songs as well as control it over time with the suitable system that they choose.
Phase 1: ✔ Learning knowledges and technologies related to the topic ✔ Listing main features of the application (functional requirements) ✔ Design architecture of the system. Phase 2: ✔ Implement the algorithm and functionalities of the application ✔ Evaluate the algorithm and application. Ngày giao nhiệm vụ: 30/05/2023 4. Ngày hoàn thành nhiệm vụ: 15/09/2023 5.
Họ tên giảng viên hướng dẫn: Phần hướng dẫn: TS. Trương Tuấn Anh Nội dung và yêu cầu LVTN/ĐATN đã được thông qua Bộ môn. Ngày 30 tháng 05 năm 2023 CHỦ NHIỆM BỘ MÔN GIẢNG VIÊN HƯỚNG DẪN CHÍNH (Ký và ghi rõ họ tên) (Ký và ghi rõ họ tên) TS. TRƯƠNG TUẤN ANH TS.
TRƯƠNG TUẤN ANH PHẦN DÀNH CHO KHOA, BỘ MÔN: Người duyệt (chấm sơ bộ):________________________ Đơn vị: _______________________________________ Ngày bảo vệ: __________________________________ Điểm tổng kết: _________________________________ Nơi lưu trữ luận án: _____________________________ Commitment The author warrants that the specialized project "A Visual Art Generator For Music Artists" is truthful. The content described in the report, the software source code is done by the author himself, and the reference knowledge is fully cited. The author, Huynh Phuoc Thien 2 Acknowledgement The project "A Visual Art Generator For Artists" is the result of a year of research by the author, under the enthusiastic guidance of Truong Tuan Anh, Ph. at the Faculty of Computer Science and Engineering, Ho Chi Minh City University of Technology.
First of all, the author would like to express sincere and deep gratitude to Truong Tuan Anh, Ph. for always accompanying and supporting the author with valuable experi- ences and knowledge. The advice and instructions of the teachers have led to today’s sweet fruit. The author would like to thank his special advisor and friend, Nguyen Quoc Viet, for helping the author to conduct and bring his insane ideas into reality.
This is a valu- able foundation as well as an inspiration for the author to not give up on developing the project. The author would like to thank his family, relatives, and friends for helping him dur- ing the most difficult times. After 4 years of university, the author has had many good memories and valuable lessons to help him be more resilient on the path of personal development. The author would like to thank himself for being brave enough to face this insanely diffi- cult challenge, and for stepping out of his comfort zones and finally creating something meaningful.
Finally, the author is really grateful for having his beloved special one, best friend and soulmate, Nguyen Anh Thao Ngan, in his life and on his creative journey since day one. She has never doubted the author’s creative journey and always encourages the author to move forward. In the process of research and reporting, it is inevitable that mistakes can be made, and the author would love to ask for your comments and suggestions so that the project can be more and more complete, expanded, and can be put into practice. Once again sincerely thank you! The author, Huynh Phuoc Thien 3 Abstraction While unconventional, and perhaps met with skepticism, the approach the author pro- poses diverges from the common practices observed in the field.
Typically, digital artists tend to working with tools like TouchDesigner, Blender, Processing, or Python. How- ever, the author wants to show an alternative perspective. Over the past decade, the web browser has undergone a remarkable transformation. The advent of code splitting, ES6+ conventions, and HTML canvas has contributed signifi- cantly to this progression.
Moreover, the availability of powerful libraries like Three.js has revolutionized the creation of performance-friendly animations on the website’s very own. It is essential to recognize that the browser environment is now capable of supporting intricate visual art projects and complex animations that were previously reserved for traditional desktop applications. Therefore, the system from this project helps embrace the shift towards web-based technologies and enable artists to leverage the full potential of the modern browser and create captivating both audio and visual experiences. In this project, the author has researched and proposed a web-based generator system which can help music artists to play along with 3D objects and textures and create unique visualizers for their songs and performances.1 The urgency of the subject .5 Non-functional requirements .1 Main Features of TouchDesigner .2 What makes TouchDesigner different? .1 The features of Synesthesia .2 What makes Synesthesia different? .1 Next JS/React JS .2 Three JS/React Three Fiber .2 React Three Fiber .4 Web MIDI API.
25 4 System analysis and design 27 4.1 Current issues of VAG and a creative method to manipulate the render on a web-based application .2 Audio Analysis and Control .1 Definition of Audio .2 Audio File Formats .4 How to analyze and extract the Audio data ? .5 Audio Control with MIDI Devices .7 How the data flows .3 Get component together for rendering .6 Load Audio and Analyze .7 Catch audio signal using MIDI .8 Playing the audio .9 MIDI Script - Controlling by MIDI .10 MIDI Messages Control to modify the texture .1 User add 3D objects .2 Manual Controlling The Parameters .3 Orbit Control, Axis helper and Stats .1 MIDI Connection Testing .2 Testing Objects Rendering .3 User Acceptance Testing .1 Testing functional requirement .2 Testing non-functional requirement. 70 8 References 72 6 List of Tables 4.1 The description for "Audio File Upload and Analyze" use-case diagram 37 4.2 The description for "Choose 3D Objects" use-case diagram .3 The description for "Modify Texture" use-case diagram .4 The description for "Set Lighting" use case .5 The description for "Set Camera" use case .6 The description for "Render Scene" use case. 42 7 List of Figures 2.1 An example of a TouchDesigner project .2 Interface of TouchDesigner .3 OP Create Dialog - many methods for the creative process .1 Structure of Redux Toolkit .1 We can extract data from audio wave files.2 And put it in control for our scene renderer .3 The WAV format .4 The MP3 format .5 AnalyserNode extract the Frequency data .6 GainNode modify the Amplitude data .7 Novation Launchkey MK2 .8 The Coordinate System .9 The Basic Geometries .10 Two Types of Camera .11 Two Types of Lighting .12 The use case diagram for the Visual Art Generator .13 The general architecture diagram for VAG system .14 The deployment diagram for VAG system .15 The VAG application architecture diagram .16 Data flow of VAG’s 3D Components .1 Structure of Redux Store .3 The default rendering .4 The code of Texture Loader .5 The Brick Wall texture .6 The code to load 3D models to VAG .7 The skeleton model loaded to VAG .8 The code to load and analyze audio .9 The code to update the state of the audio node .10 The example code of playing the audio file .11 The code to request the MIDI access .12 The code to handle the success and failure of MIDI Connection .13 The code to retrieve the data - message of the MIDI data .14 The code to modify texture by MIDI’s note amplitude (velocity) .15 The code to modify texture by audio signal triggered by MIDI Controller 58 8 5.16 The Operator List .17 The Code of Operator List .18 The Function to render by clicking Add .19 The Geometry After Rendering .20 The GUI Manual Control .21 Orbit Control, Axis helper and Stats .22 The Coordinate System with Orbit Control, Axis Helper and Stats .1 Testing MIDI Connection .2 Testing Objects rendering. 65 9 1 Introduction In this chapter, the author will present an overview of the urgency of the topic, objectives, and scope of the project’s research.1 The urgency of the subject .5 Non-functional requirements .1 The urgency of the subject Audio and visual arts have shared a mutual and inspiring relationship, with each form enhancing and elevating the other’s impact.
This special identity of audio and visual elements is rooted in their innate ability to convey emotions, tell stories, and engage audiences on a deeper, multi-sensory level. Together, they create a more immersive and captivating experience, transcending the boundaries of traditional artistic expression. In the realm of live performances nowadays, audio and visual are equally crucial. Music concerts, for instance, often incorporate synchronized light shows, projections, or LED displays that complement the music’s mood and tempo.
These visual elements intensify the overall experience, creating a mesmerizing ambiance that immerses the audience in the artist’s world. Many applications have been used in practice. Examples include VJ- ing softwares which are used in a world-class concert of top tier artists such as Beyonce, Kanye West, Taylor Swift, etc. However, this approach requires a lot of resources and everything has to be on a scope of strict management in many levels.
The author wants to find a solution to, first, the complexity of the available visual tools, and second, enhancing the possibilities for music artists to work on and experiment their own creation. Thus, a suitable method is required to implement the visualizer on a web- based generator, and at the same time, the artists can manage the audio and trigger it to control the textures of the visualizer. The development of the Visual Art Generator helps as an alternative to purchases of other features on the market. It is also more simple to conduct and get access to.
In live music performance In typical live performances, the artists mainly work on their audio side and let a VJ team do the visual tasks. The synchronization between the audio and visual elements is critical to achieving a cohesive and immersive experience. Thus, VJs often rehearse alongside musicians, gaining an intimate understanding of the music’s structure, beats, and dynamics. However, this approach has consumed a lot of time and resources.
