SDS-2.2, Scalable Data Science

Archived YouTube video of this live unedited lab-lecture:

This is an elaboration of the Apache Spark mllib-progamming-guide on mllib-data-types.

Overview

Data Types - MLlib Programming Guide

• Local vector
• Labeled point
• Local matrix
• Distributed matrix
  • RowMatrix
  • IndexedRowMatrix
  • CoordinateMatrix
  • BlockMatrix

MLlib supports local vectors and matrices stored on a single machine, as well as distributed matrices backed by one or more RDDs. Local vectors and local matrices are simple data models that serve as public interfaces. The underlying linear algebra operations are provided by Breeze and jblas. A training example used in supervised learning is called a “labeled point” in MLlib.