Learning From Data

Learning From Data by Yaser S. Abu-Mostafa et al is good intro to both a theoretical and practical approach to understanding modeling. Let’s make things clear, this is a textbook – not a passive read. At about 200 pages, it on the slim side for a textbook, but as the authors note in the preface, the book is “a short course, not a hurried course”.Complexity:Despite not having much modeling experience, the book was relatively easy for me to understand, although some of it did go over my head. It is a very good choice as an introduction to the field. The authors do an incredible job of weaving narrative into the knowledge early on, although this becomes less common later in the book. By that I mean most sections contain examples relating the topic to real life applications which prevents the math from becoming too abstract. And there is a good amount of math. Before reading this book, I would recommend having taken multivariable calculus since they use gradients and other things fairly frequently. If you have taken enough math but think you are a bit rusty the book takes care of that. The authors have been kind enough to include a “Table of Notation” at the back of the book to let you refresh yourself if you come across an unfamiliar symbol. So if you forget what a downward pointing triangle means, there is still hope for you!Chapter 1: The Learning Problem.In this chapter the authors provide the basic background to learning from data. As stated earlier, this is where they do some of their best work connecting the theory to real life examples. The writing style in some of these examples is almost like prose which makes it a much more enjoyable and memorable read. They summarize some of the main types of learning and define some of the key terms and principles like error and noise.Chapter 2: Training versus Testing.This chapter begins to explain what the theory of generalization is, the associated error, and numerical approximations of generalizations. I would still categorize this chapter as background knowledge that will be used more in later parts of the book.Chapter 3: The Linear Model.This is really the meat of the book. If you want to quickly learn how to do a regression then jump straight to this chapter. It covers both linear and logistic regression and also touches on nonlinear transformations.Chapter 4: Overfitting.This chapter deals with the more advanced aspects of modeling. As the authors put it “the ability to deal with overfitting is what separates professionals from amateurs”. While I can safely say that I am still an amateur, it was nice to be exposed to some of the more advanced concerns of the field. Overfitting, for those who don’t know, is trying to fit to the data more than is needed. This is often done by using more degrees of freedom than necessary to make a model i.e. making a 10th order approximation of data whose original function is actually only 2nd order. To someone new to modeling it can be very tempting to increase the order of an approximation because it might seem that higher order = higher accuracy. This section of the book does a good job of explaining how that isn’t the case by introducing the idea of an overfit penalty, the increase in error from overfitting a curve.Chapter 5: Three Learning Principles.This chapter is different. Very different. Instead of continuing to introduce other types of models, the authors decide to use the fundamentals taught in earlier chapters to talk about three key principles that are useful in modeling: Occam’s razor, sampling bias, and data snooping. Occam’s razor is something that is well-known. The book uses Albert Einstein’s explanation of the term, “An explanation of the data should be made as simple as possible, but no simpler.” This relates to the previous chapter’s problem of overfitting. Sampling bias talks about errors in modeling that come from having data that is not representative of the overall population. Data snooping talks about deciding to make a prediction after looking at data rather than before. This section was probably the most interesting to read. Unlike the previous few chapters the authors return to relying on real world examples. They even use the historical example of the false prediction of the 1948 US presidential election between Truman and Dewey. Again this makes the information in this chapter much more memorable and was by far my favorite chapter of the book.Problems/Exercises:I only briefly skimmed over a few of the exercises and problems included in this book but they did help improve my understanding of the topics. According to the authors the provided practice problems are “easy” so if you want more of a challenge you will need to look elsewhere.Criticisms:In the middle chapters of the book, the authors use fewer real world examples and prose style writing. This is extremely unfortunate because, to someone unfamiliar with the field, it provided a hook to draw you into the more math heavy sections. Also, while the authors are very detailed and thorough in explaining different theories and types of models, they do not do a great job of listing the strengths and weaknesses of each. If I was required to make one of the models explained in this book, I probably could, but if I was asked to choose which model would be best for a given situation I would probably be unable to do so. They should have been clearer about what models are used in different situations and provided guidelines for selecting which model to use (beyond Occam’s razor). This would better connect the material to real world use and be more beneficial to the readers.Conclusion:Learning From Data provided a quick but thorough overview of modeling and machine learning. If you would like to learn more about the subject and have the required math background, it is a very good place to start. It will give you the background, main models, errors, and principles necessary for you to not only learn the language of the field but also critique and even create your own models. I highly recommend it.Score: 4.5/5

I can't pretend to have spent nearly as much time with this book as I've signed up to. It arrived this afternoon, and it got at least 20 minutes of my time looking ahead at what Professor Abu-Mostafa is going to say to the world tomorrow. But from page 15 to 27, here is what I can observe.A minor point: I love the color examples within the book. When I was in school, we didn't get textbooks like this.The tone of the text is very sympathetic in the direction of one who is interested in the subject. That is, if you need to learn from data, the text considers what you will need to know to succeed. The reader's need to understand is prioritized much higher than the reader's need to "be educated".Although I've been out of college for two decades, I have no problem following what the text has to say, or the direction it's headed in. I don't abstract things magnificently, my calculus isn't that hot, and I slid through school without a probability class as such. I love math, but I'm not a powerhouse at it compared to the peers who once sat next to me. But I can follow this text, understand the exercises, and I conjecture that I can work the problems which are meant to be worked.For a non-academic, I have considerable on-the-ground machine learning background. I've done a lot of backpropagation network training in my day in character recognition, acoustics, finance, and similar disciplines. This is where I find the text valuable: it can build on what I already know, and it starts in a place I'm already familiar with. So if this is your field or one of your fields, the book will make not just a good textbook but a useful reference.So let's talk about what this book is not, and perhaps AMLbook will publish something else for us. Don't expect a course in neural networks here. They're not in the table of contents ("hmmm..."), but you'll find them in the index pointing you to the epilogue. Support vector machines get the same treatment. Now support vector machines are a little new for me; they came into their own after I finished college. Neural networks, however, got an earlier start.What I can say is that this text is intentionally tool-neutral. I just read 13 pages that weren't about neural networks per se, but they covered vital groundwork that I learned via hands-on experience with real neural networks. My grasp of some of the "why" components of what I know is already improved.I am not in love with whatever chemical processes went into the manufacture of this book. Perhaps it came off the press so recently that I'm noticing that, and the effects will fade. For right now, it's a definite don't-cuddle-with volume which drives me to the bathroom sink when I put it down. I can live with this; it's a small price to pay for the knowledge.The amount I paid for this text was astonishingly low. No doubt the authors have thought this through carefully, but this text will hold its own against others that cost three to six times what I paid.I apologize if this is not the best-informed review. I'm hitting send a little early, because at the present time, you don't have many alternatives to read.

Great book. I would say it's generally well written and easy to understand, and it covers interesting subject matter. If it had more worked examples it would get five stars.

Shipping took awhile, but overall I am happy. You also need access to read the remaining 3 chapters online as they are not present in the physical copy.

I like the book very much. As a said, is very good for beginners

A must-have for every machine learning student.

It takes a dedicated teacher to write a book like this. No wonder the author is so passionate on screen which is also reflected in the book. We would love a second edition which incorporates all the extra chapters which is now available only as an online edition (that is ehybibratebit as 4/5 otherwise this surely deserves a 5/5)

Bought the book second-hand from the US. It assumes a lot of prior knowledge of Maths and programming. Despite high production quality, I wouldn't recommend it at all to machine learning beginners unless you are very comfortable with engineering mathematics (functions, statistics, calculus) and at least one programming language. I have already done Prof Andrew Ng's Coursera ML course, and still couldn't follow its first chapter after the first few pages. This book is intended to be the classroom text for the courses the authors take in their universities, so maybe it is not supposed to be used for self-education. It might become more comprehensible with teachers explaining things in parallel. Might be good for reference after you already know ML.