Texts in Computer Science Ze-Nian Li Mark S. Drew Jiangchuan Liu Fundamentals of Multimedia Second Edition Texts in Computer Science Editors David Gries Fred B. Schneider For further volumes: http://www.com/series/3191 Ze-Nian Li Mark S. Drew • Jiangchuan Liu Fundamentals of Multimedia Second Edition 123 Ze-Nian Li Jiangchuan Liu Simon Fraser University Simon Fraser University Vancouver, BC Vancouver, BC Canada Canada Mark S.
Drew Simon Fraser University Vancouver, BC Canada Series editors David Gries Fred B. Schneider Department of Computer Science Department of Computer Science Cornell University Cornell University Ithaca, NY Ithaca, NY USA USA ISSN 1868-0941 ISSN 1868-095X (electronic) Texts in Computer Science ISBN 978-3-319-05289-2 ISBN 978-3-319-05290-8 (eBook) DOI 10.1007/978-3-319-05290-8 Springer Cham Heidelberg New York Dordrecht London Library of Congress Control Number: 2014933390 1st Edition: ! Prentice-Hall, Inc. 2004 ! Springer International Publishing Switzerland 2014 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed.
Exempted from this legal reservation are brief excerpts in connection with reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publisher’s location, in its current version, and permission for use must always be obtained from Springer. Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violations are liable to prosecution under the respective Copyright Law.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein.
Printed on acid-free paper Springer is part of Springer Science+Business Media (www.com) To my mom, and my wife Yansin Ze-Nian To Noah, Ira, Eva and, especially, to Jenna Mark To my wife Jill, and my children Jiangchuan Preface A course in Multimedia is rapidly becoming a necessity in Computer Science and Engineering curricula, especially now that multimedia touches most aspects of these fields. Multimedia was originally seen as a vertical application area, i., a niche application with methods that belong only to itself. However, like pervasive computing, with many people’s day regularly involving the Internet, multimedia is now essentially a horizontal application area and forms an important component of the study of algorithms, computer graphics, computer networks, image processing, computer vision, databases, real-time systems, operating systems, information retrieval, and so on. Multimedia is a ubiquitous part of the technological envi- ronment in which we work and think.
This book fills the need for a university-level text that examines a good deal of the core agenda that Computer Science sees as belonging to this subject area. This edition constitutes a significant revision, and we include an introduction to such current topics as 3D TV, social networks, high efficiency video compression and conferencing, wireless and mobile networks, and their attendant technologies. The textbook has been updated throughout to include recent developments in the field, including considerable added depth to the net- working aspect of the book. To this end, Dr.
Jiangchuan Liu has been added to the team of authors. While the first edition was published by Prentice-Hall, for this update we have chosen Springer, a prestigious publisher that has a superb and rapidly expanding array of Computer Science textbooks, particularly the excellent, dedicated, and long-running/established textbook series: Texts in Computer Science, of which this textbook now forms a part. Multimedia has become associated with a certain set of issues in Computer Science and Engineering, and we address those here. The book is not an intro- duction to simple design considerations and tools—it serves a more advanced audience than that.
On the other hand, the book is not a reference work—it is more a traditional textbook. While we perforce may discuss multimedia tools, we would like to give a sense of the underlying issues at play in the tasks those tools carry out. Students who undertake and succeed in a course based on this text can be said to really understand fundamental matters in regard to this material, hence the title of the text. In conjunction with this text, a full-fledged course should also allow students to make use of this knowledge to carry out interesting or even wonderful practical vii viii Preface projects in multimedia, interactive projects that engage and sometimes amuse and, perhaps, even teach these same concepts.
Who Should Read this Book? This text aims at introducing the basic ideas used in multimedia, for an audience that is comfortable with technical applications, e., Computer Science students and Engineering students. The book aims to cover an upper-level undergraduate multimedia course, but could also be used in more advanced courses. Indeed, a (quite long) list of courses making use of the first edition of this text includes many undergraduate courses as well as use as a pertinent point of departure for graduate students who may not have encountered these ideas before in a practical way. As well, the book would be a good reference for anyone, including those in industry, who are interested in current multimedia technologies.
The text mainly presents concepts, not applications. A multimedia course, on the other hand, teaches these concepts, and tests them, but also allows students to utilize skills they already know, in coding and presentation, to address problems in multimedia. The accompanying website materials for the text include some code for multimedia applications along with some projects students have developed in such a course, plus other useful materials best presented in electronic form. The ideas in the text drive the results shown in student projects.
We assume that the reader knows how to program, and is also completely comfortable learning yet another tool. Instead of concentrating on tools, however, the text emphasizes what students do not already know. Using the methods and ideas collected here, students are also enabled to learn more themselves, sometimes in a job setting: it is not unusual for students who take the type of multimedia course this text aims at to go on to jobs in multimedia-related industry immediately after their senior year, and sometimes before. The selection of material in the text addresses real issues that these learners will be facing as soon as they show up in the workplace.
Some topics are simple, but new to the students; some are somewhat complex, but unavoidable in this emerging area. Have the Authors Used this Material in a Real Class? Since 1996, we have taught a third-year undergraduate course in Multimedia Systems based on the introductory materials set out in this book. A one-semester course very likely could not include all the material covered in this text, but we have usually managed to consider a good many of the topics addressed, with mention made of a selected number of issues in Parts 3 and 4, within that time frame. Preface ix As well, over the same time period and again as a one-semester course, we have also taught a graduate-level course using notes covering topics similar to the ground covered by this text, as an introduction to more advanced materials.
A fourth-year or graduate-level course would do well to discuss material from the first three Parts of the book and then consider some material from the last Part, perhaps in conjunction with some of the original research references included here along with results presented at topical conferences. We have attempted to fill both needs, concentrating on an undergraduate audience but including more advanced material as well. Sections that can safely be omitted on a first reading are marked with an asterisk in the Table of Contents. What is Covered in this Text? In Part 1, Introduction and Multimedia Data Representations, we introduce some of the notions included in the term Multimedia, and look at its present as well as its history.
Practically speaking, we carry out multimedia projects using software tools, so in addition to an overview of multimedia software tools we get down to some of the nuts and bolts of multimedia authoring. The representation of data is critical in the study of multimedia, and we look at the most important data rep- resentations for use in multimedia applications. Specifically, graphics and image data, video data, and audio data are examined in detail. Since color is vitally important in multimedia programs, we see how this important area impacts mul- timedia issues.
In Part 2, Multimedia Data Compression, we consider how we can make all this data fly onto the screen and speakers. Multimedia data compression turns out to be a very important enabling technology that makes modern multimedia systems possible. Therefore we look at lossless and lossy compression methods, supplying the fundamental concepts necessary to fully understand these methods. For the latter category, lossy compression, arguably JPEG still-image compression stan- dards, including JPEG2000, are the most important, so we consider these in detail.
But since a picture is worth 1,000 words, and so video is worth more than a million words per minute, we examine the ideas behind the MPEG standards MPEG-1, MPEG-2, MPEG-4, MPEG-7, and beyond into new video coding standards H. Audio compression is treated separately and we consider some basic audio and speech compression techniques and take a look at MPEG Audio, including MP3 and AAC. In Part 3, Multimedia Communications and Networking, we consider the great demands multimedia communication and content sharing places on networks and systems. We go on to consider wired Internet and wireless mobile network tech- nologies and protocols that make interactive multimedia possible.
We consider current multimedia content distribution mechanisms, an introduction to the basics of wireless mobile networks, and problems and solutions for multimedia communication over such networks. x Preface In Part 4, Multimedia Information Sharing and Retrieval, we examine a number of Web technologies that form the heart of enabling the new Web 2.0 paradigm, with user interaction with Webpages including users providing content, rather than simply consuming content. Cloud computing has changed how services are pro- vided, with many computation-intensive multimedia processing tasks, including those on game consoles, offloaded to remote servers. This Part examines new- generation multimedia sharing and retrieval services in the Web 2.0 era, and discusses social media sharing and its impact, including cloud-assisted multimedia computing and content sharing.
The huge amount of multimedia content militates for multimedia aware search mechanisms, and we therefore also consider the challenges and mechanisms for multimedia content search and retrieval. Textbook Website The book website is http://www. There, the reader will find copies of figures from the book, an errata sheet updated regularly, programs that help demonstrate concepts in the text, and a dynamic set of links for the ‘‘Further Exploration’’ section in some of the chapters. Since these links are regularly updated, and of course URLs change quite often, the links are online rather than within the printed text.
Instructors’ Resources The main text website has no ID and password, but access to sample student projects is at the instructor’s discretion and is password-protected. For instructors, with a different password, the website also contains course instructor resources for adopters of the text. These include an extensive collection of online slides, solu- tions for the exercises in the text, sample assignments and solutions, sample exams, and extra exam questions.