Learning From Data: A Short Course

by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, Hsuan-Tien Lin

Machine learning allows computational systems to adaptively improve their performance with experience accumulated from the observed data. Its techniques are widely applied in engineering, science, finance, and commerce. This book is designed for a short course on machine learning. It is a short course, not a hurried course. From over a decade of teaching this material, we have distilled what we believe to be the core topics that every student of the subject should know. We chose the title `learning from data' that faithfully describes what the subject is about, and made it a point to cover the topics in a story-like fashion. Our hope is that the reader can learn all the fundamentals of the subject by reading the book cover to cover. ---- Learning from data has distinct theoretical and practical tracks. In this book, we balance the theoretical and the practical, the mathematical and the heuristic. Our criterion for inclusion is relevance. Theory that establishes the conceptual framework for learning is included, and so are heuristics that impact the performance of real learning systems. ---- Learning from data is a very dynamic field. Some of the hot techniques and theories at times become just fads, and others gain traction and become part of the field. What we have emphasized in this book are the necessary fundamentals that give any student of learning from data a solid foundation, and enable him or her to venture out and explore further techniques and theories, or perhaps to contribute their own. ---- The authors are professors at California Institute of Technology (Caltech), Rensselaer Polytechnic Institute (RPI), and National Taiwan University (NTU), where this book is the main text for their popular courses on machine learning. The authors also consult extensively with financial and commercial companies on machine learning applications, and have led winning teams in machine learning competitions.

Genres: Computer Science, Artificial Intelligence, Mathematics, Technology, Science, Nonfiction, Reference

213 pages, Hardcover

First published January 1, 2012

Reviews

[Vicki · Rating 3 out of 5]

This book is really dense in that it encodes a TON of information in a small package. If you are already comfortable with mathematical notation and have taken at least 2-3 linear algebra, math proof classes, and have some familiarity with ML-stats concepts like OLS, it will definitely boost your understanding and serves as a good outline for the basics like errors, regularization, bias, and others.

However what I think is misleading is that it’s marketed as an intro book. It’s only an intro book if you already have experience in the field or have taken courses on this. Complete beginners would be better served by OReilly books or Elements of Statistical Learning or both those things in complement to this book.

[Zarathustra Goertzel · Rating 5 out of 5]

Learning From Data does exactly what it sets out to do, and quite well at that.

The book focuses on the mathematical theory of learning, why it's feasible, how well one can learn in theory, etc. Why must one learn probabilistically? Why is overfitting a very real part of life? Why can't we obsessively try every single possible hypothesis until we find a perfect match? (Oh, yes, one could formalize problems with various logical fallacies after reading this :p)

As for learning algorithms, only a few linear, supervised ones were actually discussed. This is okay as the focus is on learning itself more than specific methods (and 3-4 more are covered in e-chapters).

The excercises throughout prompt the right questions, and the problems lead you into more depth (just reading over them should teach one a lot more :x).

Definitely recommended to anyone interested in learning (who can read basic linear maths) :3

[Romann Weber · Rating 5 out of 5]

This is an essentially perfect little prelude to machine learning. Despite the book's short length, there is great depth in the presentation. The authors have produced a remarkably well-written and carefully presented book, with some great color illustrations as well. This is a book clearly written with the reader in mind, and I hope it soon becomes a standard primer for those embarking on deeper ML research and study.

[Debasish Ghosh · Rating 4 out of 5]

Very clear explanation, a good mix of theory and practical items. Meant for a short course, doesn't deal w/ a lot of topics. But teaches fundamentals like VC dimension, regularization, overfitting, bias and variance in great details.

[Howard Bandy · Rating 5 out of 5]

An excellent introduction to machine learning, accessible with a small amount of university mathematics. Dr. Yaser Abu-Mostafa, one of the three authors, presents an excellent series of video lectures that follow the book very closely. The series is available from the host institution, Cal Tech: Learning from Data Video Lectures, and also on YouTube.

[Emil Petersen · Rating 5 out of 5]

This is a very good and short introduction on the problem of learning from data. I also watched the Caltech lectures done by Yaser while I read the book. They are some of the best lectures I've had. There is a couple of online chapters as well that effectively doubles the size of the book, but I have only had a good look at the online chapter on SVM's.

[Jethro Kuan · Rating 5 out of 5]

Excellent introduction to the theory of Machine Learning, I think they put it well themselves: it is a short course, but not a hurried course. Worth picking up a second time.

[JD · Rating 5 out of 5]

This book was fire. Easy to read and really informative.

[Odo · Rating 4 out of 5]

4.5/5.0

[Nick Greenquist · Rating 5 out of 5]

there's some pretty hardcore math in this book so I didn't fully understand it all, but it's one of the best machine learning books I've ever read.

[Emil · Rating 5 out of 5]

If you are looking for a practical handbook that contains algorithms and code that you can plug into a data set, this is not the book for you. The focus of the book is real understanding of machine learning concepts. You will know why and how things are done in a particular way. You will learn to derive algorithms and equations on your own. You would also be capable of tweaking parts of the algorithms. Make sure you understand the math really well. And also make sure you do the problem sets. This book gives a solid base on the theory of ML.

[Zhaodan Kong · Rating 5 out of 5]

A must-read for any machine learning practitioner. The authors elegantly blends theoretical underpinnings with easy-to-follow examples. However, as indicated on the book's cover, this is a book on fundamentals. You need to consult other books to see how the principles presented in this book play out in specific techniques. FYI, Dr. Abu-Mostafa has a class based on this book, which is available on Youtube.

[Fazlan · Rating 5 out of 5]

This is one of the greatest machine learning books available in the market. Prof Yaser and the co-authers have done a very good job in conveying the fundamentals of the subject so that you can easily catch up the complex topics from there on. The video lecture series available on his site can add value to the reading, and his way of explaining complex topics is second to none.

[Wudongnoodles · Rating 5 out of 5]

It is basically statistical learning. One of the co-authors used ESL before this book as the textbook for his class. So no Bayesian at all. A gentle dose of basic neural network.

The book is very rigorous.

[Andy]

Difficult to read unfortunately.

[Tianqi Zhao · Rating 5 out of 5]

This textbook includes all the essential knowledge of Machine Learning such as support vector machine, neural networks and clustering. The first part is based on theory, like Learning Theory, the second part is based on applications, but only focus on a few really important Machine Learning algorithms. Problem set is very challenging but you can get the most out of it after taking time to finish these questions. Written by Prof. Yaser Abu-Mostafa from Caltech and his two Ph.D. students: Prof. Malik Magdon-Ismail — a very popular Professor in Computer Science Department of Rensselaer Polytechnic Institute, and Prof. Hsuan-Tien Lin from National Taiwan University, a highly-respected and even more popular Professor in the Chinese-Speaking world. Prof. Yaser Abu-Mostafa and Prof. Malik Magdon-Ismail uploaded different courses for this book on YouTube and Prof. Hsuan-Tien Lin also uploaded his Chinese Machine Learning course, on Coursera and YouTube separately. For those who have solid mathematical background I would highly recommend you to read the book and practice the questions in this book over and over again.

[Srinidhi · Rating 5 out of 5]

I recommend this book if you wish to clearly understand why learning from data works. It provides theoretical as well as practical foundation of machine learning. I found this book to be indispensable while I took the author's MOOC on edx. I spent about 25 to 30 hours per week to understand the concepts and solve homework problems. The book covers only linear models. However, the dynamic e-chapters provided on the author's website cover neural networks, SVM, and similarity-based methods. The most important part for a machine learning practitioner is to understand the problem of overfitting. This book clearly explains the concept of overfitting and how to combat it. I use this book as a reference to design experiments to understand overfitting and model validation.

[J. Nic Fisk · Rating 5 out of 5]

Besides Andrew Ng's machine learning course on coursera, probably the best guide to machine learning I've used.

[Robert · Rating 4 out of 5]

I think this is the best book to teach an introduction to machine learning class.

[Michel Henrique · Rating 5 out of 5]

Really good resource on machine learning (as a first course). It doesn't rely on any particular tool it's pure math (statistics) that's what makes the book so good compared to others.

[DANIEL SCHNAIDER · Rating 3 out of 5]

This book does not have all the content from his online course but has the exact same name. Very disappointing

[Camille Birbes · Rating 5 out of 5]

Challenging, but rewarding. Best used alongside the freely available lectures on Youtube.

[!Minh · Rating 5 out of 5]

A nice book covering the foundation of learning theory from the frequentists perspective.

[Azeem · Rating 2 out of 5]

The book spends most of the start trying to answer the question "can one learn from the data". It is definitely an interesting question but past that the book doesn't have much more to offer.

For such a diverse field, this is definitely not an introduction I would recommend since it fails to give an overview of anything more complex than a linear regression.

[Joshua]

A superb little book. Very insightful and well written, and a great value. Definitely start with reading this one!

[Mark · Rating 3 out of 5]

ok

[Farhad Da · Rating 5 out of 5]

As the writer said, It's a short course, not a hurried one! I learned a lot from this book, it tries to teach foundations of Machine learning unlike a lot of books that their focus is just talking about a bunch of algorithms!

Now I know why and when learning from data is plausible.Also, how to avoid simple mistakes that a lot of people do while they practicing Machine Learning!

[Lav Kumar]

easy book to read

[Alex Egg · Rating 5 out of 5]

Good Frequentist approach to Linear Parametric Modeling. Good discussion on Regularization and Validation/Testing also. I like the practical rules-of-thumb the authors provide also -- they could have just left the theory, but they went out on a limb and made concrete statements. Maybe many of you could lament with me about Chapter 2 however. I could not get through it, too dense :)