Machine Learning Basics

TL;DR: The idea of classification and what it means for a classificator to classify data is explored, and the performance of a general classifier is evaluated, including naive Bayes and the simplest neural network, logistic regression.

Abstract: This chapter explores the fundamentals of machine learning, since deep learning is above everything else, a technique for machine learning. We explore the idea of classification and what it means for a classificator to classify data, and proceed to evaluating the performance of a general classifier. The first actual classifier we present is naive Bayes (which includes a general discussion on data encoding and normalization), and we also present the simplest neural network, logistic regression, which is the bread and butter of deep learning. We introduce the classic MNIST dataset of handwritten digits, the so-called ‘fruit fly of machine learning’. We present also two showcase techniques of unsupervised learning, K-means to explain clustering and the general principle of learning without labels and the principal component analysis (PCA) to explain how to learn representations. PCA is also explored in more detail later on. We conclude with a brief exposition on how to represent language for learning with bag of words.