ICS 613 Introduction to Big Data Computing Systems
The field of computer science is experiencing a transition from computation-intensive to data-intensive problems, wherein data is produced in massive amounts by large sensor networks, simulations, and social networks. Efficiently extracting, interpreting, and learning from very large datasets requires a new generation of data management technologies. This course gives an introduction to the Hadoop ecosystem as de facto big-data-management system and special consideration will be made to the Apache Spark data analysis framework. The fundamental concepts on which the emerging big data management systems are based are discussed first. Once a foundation is defined, technologies and algorithms that are used to work with big data sets are studied. Tentative topics covered include: distributed file system, map-reduce programming paradigm, Apache Spark basics, SparkSQL, Pig, Hive, Impala, and Scoop. The course is programming intensive and includes several programming assignment projects using the Hadoop ecosystem.
First day attendance is mandatory.
Prerequisites: Graduate standing. Note: Students are responsible to both be aware of and abide by prerequisites for ICS courses for which they enroll, and will be administratively dropped from a course if they have not met prerequisites.
Prerequisites
Special information
Prerequisites: Graduate standing. Note: Students are responsible to both be aware of and abide by prerequisites for ICS courses for which they enroll, and will be administratively dropped from a course if they have not met prerequisites.
4 Graduate credits
Effective January 10, 2016 to present
Learning outcomes
General
- Explain the sources and challenges of big data.
- Assess the similarities and differences between the emerging big data computing platforms and the traditional computing systems.
- Explain the key issues in big data management including distributed and parallel processing, data modeling, and query languages.
- Describe the principles of distributed file system.
- Develop applications using the map-reduce programming paradigm.
- Develop applications using components of the Hadoop ecosystem (e.g., Hive, Pig, Sqoop, Spark, and HBase).
- Demonstrate expertise in reading, presenting, and criticizing peer-reviewed papers in big data management.