Data Science from Scratch: First Principles with Python

by Joel Grus

Data science libraries, frameworks, modules, and toolkits are great for doing data science, but they’re also a good way to dive into the discipline without actually understanding data science. In this book, you’ll learn how many of the most fundamental data science tools and algorithms work by implementing them from scratch.

If you have an aptitude for mathematics and some programming skills, author Joel Grus will help you get comfortable with the math and statistics at the core of data science, and with hacking skills you need to get started as a data scientist. Today’s messy glut of data holds answers to questions no one’s even thought to ask. This book provides you with the know-how to dig those answers out.

Get a crash course in Python Learn the basics of linear algebra, statistics, and probability—and understand how and when they're used in data science Collect, explore, clean, munge, and manipulate data Dive into the fundamentals of machine learning Implement models such as k-nearest Neighbors, Naive Bayes, linear and logistic regression, decision trees, neural networks, and clustering Explore recommender systems, natural language processing, network analysis, MapReduce, and databases

Genres: Programming, Computer Science, Technology, Nonfiction, Technical, Reference, Artificial Intelligence

330 pages, ebook

First published April 14, 2015

Reviews

[Dan · Rating 5 out of 5]

I worked thru all of the examples in this book. Rather than have you import numpy and pandas and scikit-learn, he walks you through how to build up these tools yourself. What you build will be terribly inefficient and you should never use them in real life, but you will get a great feel for how they work under the hood.

(I also learned that my linear algebra is very rusty and I need a brush up ...)

I disagree with some of the reviews that they he doesn't do a good job explaining the computation -- he does that in the comments of the code, where he walks you step by step thru what's he's doing.

A great intro for a beginner like me.

[Tim · Rating 2 out of 5]

Not terribly impressed with this one. The way I see it, readers of this book either will already know how to do data science, or they won't. If they do (and here I'm ignoring the fact that why would they, since the title of the book is "data science from scratch"), then they will find the explanations of concepts too basic, and the Python code implementation examples mostly useless (they, after all, are not using the libraries specifically designed to do data science, but rather implementing a naïve direct approach). And if they don't, then they will find the explanations of concepts too skimpy to be useful, with the Python code implementation examples still useless, because a) they are hard to follow without first understanding the concepts, and b) they are "from scratch", so you wouldn't even be able to re-use them in production for the concepts you did understand. In all, my biggest gripe is that the book covers not the "data science from scratch", as advertised, but rather "Python code implementation from scratch, as it applies to poorly explained data science concepts". The book isn't totally useless, but definitely not a good starting point to learn data science from scratch.

[Sefa]

Decent book on introduction to data science using Python.

BTW, we should seriously stop writing books on elementary data science using R or Python. We have too many and they already started to look alike.

[Vlad Gheorghe · Rating 3 out of 5]

This is not a stand-alone book. It won't show you how to solve real problems with code, as it doesn't use any actual library. And it won't teach you the theory and mathematics of data science (the explanations of the models are very synthetic, often frustratingly so).

Rather, the book lives in a limbo between theory and practice. The old saying goes, you haven't understood something until you can explain it to your grandmother. Then again, whoever came up with that saying probably never tried explaining backpropagation to her. If you lack a mathematically oriented grandma, the Python interpreter is a good substitute.

Working through this book, I feel that I have learned and consolidated many new concepts. However, I skipped several chapters which were quite compressed and hurried. For example, the chapter on Neural Networks is only 17 Kindle pages long. It introduces perceptrons, feed-forward, backpropagation, and a neural network coded from scratch, which the author applies to learning the fizzbuzz challenge. The average reader will experience a lot of frustration here; which is not necessarily an evil thing.

As promised, the author builds almost everything from scratch, so his code style gets to shine. I found his code to be elegant and pythonic, generally a joy to read, and maybe that's my greatest takeaway (though the emphasis on typing can get annoying, especially when it involves importing types or creating new objects).

Skimming through other reviews, I notice a general misunderstanding about the target of this book - of which I partly was a victim myself. The phrase "from scratch" implies a basic and easy approach, which would be suitable for beginners. Of course, in data science the basic approach is precisely the opposite, i.e. playing with the libraries without fully understanding how they work. To start building algorithms from scratch, you need to either be quite decent at mathematics, or to have played enough with software that you have built some mental models (or notional machines) about "how this stuff works".

In conclusion, I would only recommend this book to intermediate practitioners who want to make sure they have a solid grasp of the fundamental methods. If you are still a beginner, there are better choices.

[Mohammed Ashour · Rating 3 out of 5]

Good at:
- Practicing entry projects (exercises)
- simple language

Bad at:
- lack of some required details in some sections
- outdated code
- the apps -codes- are not that useful in some sections

Overall the book is a good refreshing read. but not that good for studying

[John · Rating 4 out of 5]

This was a fun survey of popular topics in contemporary data science. It was well written for a text book, and easy to read. I suppose it was light on formal proofs, but it made up for that by having you build toy models of all the major ideas. Well worth the read for me, as I am very new to data science but well versed in Python and math. I would like to see a follow-up book that covers the same topics, but using the real libraries people use in industry to solve these same problems.

[Nicholas Teague · Rating 4 out of 5]

to be read for purposes of demonstrating fundamentals. most of work here can be accomplished much simpler with advanced libraries, but this type of text helps one to understand the why and the building blocks of more elaborate practice.

[Sungjoo Ha · Rating 4 out of 5]

Good introductory book on data science. I would recommend this to people who wish to learn basic things in a hands-on fashion.

[Kevin Moore · Rating 4 out of 5]

Really good overview, but needed a little more information about which software packages implement the functionality discussed.

[Mehmet Çetin · Rating 4 out of 5]

A brief introduction to many concepts and step-by-step construction of a working code. I would expect a little more math and theory that's why I gave four stars instead of five.

[Mason Lower · Rating 4 out of 5]

Hmmm this book touches on lots of topics and is a good introduction (I think) for many of them. Includes lots of examples of how you could actually tackle problems but since it uses no libraries much of the code in this book is really usable in the real world but still good to understand the fundamentals and what’s going on behind the scenes. Also a decent amount of jokes in the book which makes it more entertaining to read.

[Lazysmokier · Rating 5 out of 5]

Отличная книга, чтобы погрузиться в мир машинного обучения. Не сказать, что после прочтения вы будете знать много, но зато про многое из области ML. Подход автора состоит в том, чтобы вместо детального описания алгоритмов словами, привести реализацию в виде кода на питоне. Хорошего, понятного, компактного кода (пользуясь питоном время от времени не первый год, не думал что этот язык может быть таким элегантным). Во-первых, это позволяет сэкономить место – хорошо написанный код лучше описания алгоритмов человечим языком, во-вторых, такая реализация с нуля дает более глубокое понимание того, что творится внутри библиотек, реализующих тот же функционал. В конце каждой главы есть раздел "куда смотреть дальше", где даются дельные советы, какие библиотеки, инструменты и материалы помогут вам развиться в заданном направлении. Собственно в книге рассмотрены разнообразные темы – от байесовского классификатора до обработки естественных языков

Книга написана живым и интересным языком. Читать её так же интересно, как и код автора

Книга рекомендуется всем начинающим знакомство с машинным обучением. Автор определенно энтузиаст и вас, скорее всего, энтузиазмом заразит. Кроме того, книга позволит определиться с конкретным направлением ML, которое будет полезно и интересно для вас. Знание питона не обязательно, но приветствуется

[Tony Poerio · Rating 3 out of 5]

Great book for a general overview of the concepts, and understanding what 'data science' actually means. Lots of code to drive to the points home, and it taught me quite a few Python tricks.

I can foresee using this as a reference for the main concepts, or when looking for a straightforward implementation of the algorithms discussed. The information is very solid.

If you want to power straight through, it's a tough read at times--but Joel's a very good writer, and I enjoyed the dry humor interspersed throughout. I'm new to data science myself, and happy I put the time into this one.

[Roger Mitchell · Rating 5 out of 5]

Fundamental concepts revealed, libraries for the win

Joel does a great job walking through the tasks a data scientist would take to solve hypothetical problems, and explaining the models most popularly implemented in machine learning. An overwhelming majority of the code examples are useless, which is intentional as Joel notes how to build things from scratch. Libraries (like pandas, scikit-learn, etc) provide APIs to accomplish many of these tasks without writing from scratch, but without the underlying knowledge and appreciation for how these are implemented, it may be easy to take for granted or not grasp the concepts.

[Felipe Scuciatto · Rating 4 out of 5]

De nada adianta conhecer ciência de dados sem fazer ciência de dados. Partindo deste pressuposto, este livro traz o essencial para "colocar a mão na massa" e torturar alguns dados. O mais interessante deste livro é que ele parte do absoluto zero nos algoritmos. Por não confiar em nenhuma biblioteca de análise, ele demonstra toda construção técnica por traz de regressões, redes neurais, árvores de decisão, classificadores bayesianos, etc.

Leitura recomendada para um sólido entendimento da prática de Data Science e Machine Leraning.

[Mark · Rating 4 out of 5]

more entertaining that an entry level programming language text would usually be, and not at the expense of content. well, maybe somewhat at the expense of content because some of the examples are a little too simple to give a real feel for what the methods are useful for. but overall lots of fun and very good information. i did find it a little frustrating, especially early on, that no equations were included and reading python was necessary to understand the fundamentals.

[Daeus · Rating 3 out of 5]

I really like the overview/coverage of data science as a whole. Granted, it would help a lot to have experience coding in python to read along, which makes it feel more like a data science overview for a software engineer. Lots of meh-quality humor and lots of coding and hard-coded links in the book that seem odd (maybe better an an ebook), but overall a good refresher. 

Quotes
- "variance measures how a single variable deviates from its mean, covariance measures how two variables vary in tandem from their means. ...a 'large' positive covariance means that x tends to be large when you is large and small when you Is small. ...this number can be hard to interpret, for a couple of reasons: (1) its units are the product of the inputs units.... (2) if each user had twice as many friends (but the same number of minutes), the covariance would be twice as large. But in a sense the variables would be just as interrelated. Said differently, its hard to say what counts as a 'large' covariance. For this reason, it's more common to look at correlation, which divides out the standard deviation of both variables. .... the correlation is unites and always lies between -1 (perfect anti-correlation), and 1 (perfect correlation)."
- "The key issue [Simpsons paradox] is that correlation is measuring the relationship between your two variables all else being equal.... the only real way to avoid this is by knowing your data and by doing what you can to make sure you've checked for possible confounding factors."
- "x=[-2,-1,0,1,2], y=[2,1,0,1,2] have zero correlation. But they certainly have a relationship-each element of y equals to absolute value of the corresponding element of x. What they don't have is a relationship in which knowing how x_i compares to mean (x) gives us information about how y_i compares to mean(y). That is the sort of relationship that correlation looks for. .... ×=[-2,-1,0,1,2], y=[99.98,99 99,100,100.01,100.02] are perfectly correlated, but (depending on how you're measuring it) it's quite possible that this relationship isn't all that interesting."
- F Scott Fitzgerald quote: "to write it, it took three months; to conceive it, three minutes; to collect the data in it, all my life."
- "...data science is mostly turning business problems into data problems and collecting data and understanding data and cleaning data and formatting data, after which machine learning is almost an afterthought. Even so, it's an interesting and essential afterthought..."
- "What is a model? It's simply a specification of a mathematical (or probabilistic) relationship that exists between different variables." .... " well use machine learning to refer to creating and using models that are learned from data. In other contexts this might be called predictive modeling or data mining...."
- "supervised models (in which there is a set of data labeled with the correct answers to learn from), and unsupervised models (in which there are no such labels).... semisupervised (in which only some of the data are labeled), and online (in which the model needs to continuously adjust to newly arriving data)."
- "A bigger problem is if you use the test/train split not just to judge a model but also to choose from among many models.... in such a situation, you should split the data into three parts: a training set for building models, a validation set for choosing among trained models, and a test set for judging the final model."
- "A model that predicted 'yes' when its even a little bit confident will probably have a high recall but a low precision; a model that predicts 'yes' only when its extremely confident is likely to have a low recall and a high precision. Alternatively, you can think of this as a trade-off between false positives and false negatives. Saying 'yes' too often will give you lots of false positives; saying 'no' too often will give you lots of false negatives."
- "High bias and low variance typically corresponding to underfitting." Ie a flat line model is pretty stable but consistently wrong.
"If your model has high bias (which means it performs poorly even on your training data) then one thing to try is to add more features..... if your model has high variance, then you can similarly remove features. But another solution is to obtain more data (if you can)."
- "Holding model complexity constant, the more data you have, the harder it is to overfit. On the other hand, more data won't help with the bias. If your model doesn't use enough features to capture regularities in the data, throwing more data at it won't help."... "Features are whatever inputs we provide to our model."
- "Naive Bayes classifier... is suited to yes-or-no features... regression models.... require numeric features (which could include dummy variables that are 0s and 1s). And decision trees... can deal with numeric or categorical data."
- "The key to Naive Bayes is making the (big) assumption that the presences (or absenses) of each word are independent of one another  condition on a message being spame or not. Intuitively, this assumption means that knowing whether a certain spam message contains the word 'viagra' gives you no information about whether that same message contains the word 'rolex.'.... this is an extreme assumption.... despite the unrealisticness of this assumption, this model often performs well and is used in actual spam filters."
- "In practice, you usually want to avoid multiplying lots of probabilities together  to avoid a problem called underfloor, in which computers font deal well with floating point numbers that are really close to zero." [Logs help here].
- [multiple regression] "You should think of the coefficients of the model as representing all-else-being-equal estimates of the impacts of each factor."
- [multiple regression] "Keep in mind, however  that adding new variables to a regression will necessarily increase the R-squared.... the regression as a whole may fit our data very well, but if some of the independent variables are correlated (or irrelevant), their coefficients might not mean much." ... "if the goal is to explain some phenomenon, a sparse model with three factors might be more useful than a slightly better model with hundreds....regularization is an approach in which we add to the error term a penalty that gets larger as beta gets larger. We then minimize the combined error and penalty. The more importance we place on the penalty term, the more we discourage large coefficients. [Eg ridge regression].... usually you'd want to rescale your data before using this approach. After all, if you changed years of experience to centuries of experience, its least Square coefficients would increase by a factor of 100 and suddenly get penalized much more, even though it's the same model."
- "Whereas the ridge penalty shrank the coefficients overall, the lasso penalty tends to force coefficients to be zero, which makes it good for learning sparse models."
- "Partitioning on SSN will product one-person subsets, each of which necessarily has Ero entropy. But a model that relies on SSN is certain not to generalize beyond the training set. For this reason, you should probably to try avoid (or bucket, if appropriate) attributes with large numbers of possible values when creating decision trees."

[Charles Godfrey Kamukama · Rating 4 out of 5]

"Data Science from Scratch: First Principles with Python" by Joel Grus is an outstanding resource for anyone looking to dive into the world of data science. Grus's approach is refreshingly practical, offering readers a hands-on experience right from the start.

One of the book's standout features is its emphasis on first principles. Grus takes the reader through the fundamental concepts of data science, providing a solid foundation for more advanced topics. The use of Python as the primary programming language is a wise choice, given its popularity and versatility in the field.

The book covers a wide range of topics, from basic data manipulation and visualization to more complex machine learning algorithms. Grus's explanations are clear and concise, making even complex concepts accessible to beginners. Additionally, the inclusion of code snippets and examples further reinforces the learning process.

What sets this book apart is Grus's ability to strike a perfect balance between theory and application. Theoretical concepts are seamlessly integrated with practical exercises, allowing readers to immediately apply what they've learned. This hands-on approach not only reinforces understanding but also builds confidence in one's ability to tackle real-world data science problems.

Furthermore, Grus's writing style is engaging and approachable. He presents the material in a manner that is both informative and enjoyable to read. The book's structure is well-organized, with each chapter building upon the previous one, creating a logical progression of learning.

While the book is accessible to beginners, it also caters to more experienced data scientists. The latter chapters delve into advanced topics, ensuring that the book remains relevant even as the reader's skills progress.

In conclusion, "Data Science from Scratch: First Principles with Python" is a must-read for anyone interested in data science. Whether you're a beginner looking to start your journey or an experienced practitioner seeking to deepen your knowledge, this book has something valuable to offer. Grus's practical approach, combined with his clear explanations and engaging writing style, make this book a standout in the field of data science literature.

[Jakob · Rating 5 out of 5]

Joel Grus’ “Einführung in Data Science” ist ein beeindruckendes Werk, das sich auf die Vermittlung von essenziellen Data-Science-Techniken und -Konzepten konzentriert. Es ist ein Buch, das besonders an Anfänger gerichtet ist. Wichtig zu erwähnen ist, dass sich die folgende Rezension auf die neuere Ausgabe bezieht.

Das Buch ist gut organisiert und die Inhalte sind leicht verständlich. Grus hat einen ausgezeichneten Job gemacht, indem er die oft komplexen Themen der Datenanalyse auf eine zugängliche und praktische Weise präsentiert. Er bietet viele nützliche Beispiele und Anleitungen, die den Lesern helfen, die Konzepte in der Praxis anzuwenden.
Hierbei hat mir besonders das Kapitel zur linearen Algebra gefallen, da es erfahrungsgemäß ziemlich genau die reale Arbeit wiederspiegelt. Linear Algebra ist essentiell für die Arbeit im Operations Research und somit ein fundamentales Werkzeug für angehende Data Scientists.

Ein weiterer bemerkenswerter Aspekt des Buches ist seine Aktualität. Grus bezieht sich auf die neuesten Trends und Technologien im Bereich der Datenanalyse und bietet den Lesern wertvolle Einblicke in die aktuellen Entwicklungen in der Branche.

“Einführung in Data Science” ist mehr als nur ein einführendes Buch zur Datenanalyse. Es ist ein Leitfaden, der den Lesern hilft, eine datenorientierte Denkweise zu entwickeln und die Kommunikationslücke zwischen Data Scientists, Führungskräften und anderen, die täglich mit Daten umgehen müssen, zu schließen.

Insgesamt ist “Einführung in Data Science” ein ausgezeichnetes Buch für jeden, der mehr über Datenanalyse lernen möchte. Mit der zunehmenden Menge an Daten, die in der heutigen Welt generiert werden, wird das Verständnis von Data Science immer wichtiger. Dieses Buch bietet einen soliden Einstieg in das Thema und ist ein wertvolles Werkzeug für jeden, der in der datenintensiven Welt von heute erfolgreich sein möchte.

Besonders hervorzuheben ist, dass das Buch Data Science sehr gut konzeptionell behandelt. Dies macht es zu einem praktischen Leitfaden für alle, die neu in Data Science einsteigen und sich mit den Herausforderungen der Datenanalyse auseinandersetzen möchten.

[Oskar · Rating 3 out of 5]

Quick read. And a great intro that brushes over the area of Data Science. Even though it does not convey much knowledge that could be used by a practicing Data Scientist. The "from Scratch" part of the title refers to the book's focus on the implementation of the popular algorithms using Python. From scratch. Which is... rather pointless. Anyone serious about Data Science would use pre-packaged, efficient libraries to train their models instead. The author does send the reader to external sources with the actual Data Science tools. I just wished the author spent more time on them.

It is interesting to see that many seemingly complicated Machine Learning algorithms could be implemented in Python. However, I wouldn't say that seeing such implementation contributed to a better understanding of them.

Read the book if you want to have a quick overview of some of the tools available in the industry. If you like what you are reading, move onto more hands-on, practical guides to get a better understanding of the standard practices in the industry.

[Rajesh · Rating 4 out of 5]

Practical book which covers what's essential for data analysts getting into statistical analysis, machine learning and related topics. Good book for those starting out, but didn't have much to offer on the statistical learning side, principles and concepts wise. You're better off looking at books such as IPSUR (Jay G Kearns) and ISLR (Hastie & Tibshirani) for such content. However, this is a practical book because it introduces many relevant ideas. Some qualms: MapReduce treatment is probably outdated already, since nobody uses that much anymore; not much on NLP, and not enough discussion of non-ML statistics that are part of data science - such as frequentist statistics, distribution modeling, time series and so on.

[Sabbir · Rating 4 out of 5]

As most Python packages are already compatible with Python3, this book may seem backdated as it uses Python2. However, this difference won't hold back anyone that understands a few differences between the two versions.

This book is a great start for anyone that want to start with any kind of Artificial Intelligence or Data Science related field. As it doesn't go deep into the theory and is rather the beginner friendly hands on kind of book it is very easy and enjoyable to follow.

It is to be noted that the book will not add much value to someone that has already learned the topics the hard way. The implementations used in this book are not usable in real life scenario. However, beginners will find it useful as the book stays true to its name "Data Science from Scratch".

[Rohit Sharma · Rating 5 out of 5]

Disciplines have reliably included data as a fundamental part. If we view any business data direction has perpetually been an errand. In every practical sense, all adventures are digitized, Nowadays. Besides, data aggregation in sensors, weblogs, cell devices, and contraptions has helped in the new times. Believe it or not, there are flourishing new advances arising to deal with this heavy slide of information. With the assistance of Data Science, the specialists can see the models and frequencies in many pieces of information which permit the corporate collect to regard. It will not be misleading to communicate that information scientists are the fate of this period that is drawing closer.

https://www.sevenmentor.com/data-scie...

[PJ · Rating 4 out of 5]

I've been meaning to get through this book for 1-2 years now, and I finally got around to it. The book is highly unique and useful in its approach, but I agree with the other reviewers that you need to have a preliminary understanding of Python, ML, and Data Science to get through this one. If you do, it's a great resource to get close to the basics of the craft. I found that I needed to consult online resources regularly to throughly understand the concepts, but pairing that with the code provided, it solidified my understanding of ML in a way that just reading about statistics and linear algebra wouldn't have done. Overall I would highly recommend if you're interested in this sort of stuff and are driven by a desire to understand how stuff works.

[Leonel Esteban · Rating 3 out of 5]

Very well written and balanced between deepness and readability. However, there is a gap when it jumps to natural language processing...it seems like it escalates extremely fast.
I understand the concept of learning the basics by learning from scratch, but if anyone is trying to learn how to DO data science, this is not the book (even the author states that and mentions "data science with python" from the same editorial). And if someone is trying to learn the fundamentals of data science...well, this isn't the book either, as it only covers fundamentals in a very, very, lightened, soft matter.

[Andrei · Rating 3 out of 5]

Ambivalent about this one. On one hand the idea of implementing major ml and data science algorithms bottoms up, only using the base library in Python, is great as you can get a deeper understanding. From this point of view the book is worth reading. However the theory is quite rushed, the mathematics could have been described separately in formulas, not only code, and it lacks any graphical illustrations that would help you to visually understand. Lazy from this point of view. Overall worth reading but do not expect miracles.

[Nicky · Rating 3 out of 5]

I suppose this book is a try at combining theory with code. Although it succeeds in this to some degree, I think a lot of the concepts in the book are poorly explained and the code is often messy and confusing. That is, I often find myself glancing over the code and only reading the comments and surrounding text.

The author sometimes decides to write tests to test how things work. But it is not consistent and it is unclear what should or should not be tested. The tests only add confusion and are off-topic it seems.

[Thomas Pinder · Rating 4 out of 5]

I read this prior to beginning an MSc in Data Science and found it to be a great introduction to data science, starting out with the very basics before moving into more general ML techniques and finishing up with some of the more complex topics such as MapReduce. Not an in-depth textbook by any means, but I do not think that is the purpose of this book, moreover to give the reader a well-rounded idea of the field.

[Ulises Jimenez · Rating 4 out of 5]

It is a wonderful book to understand the detail of some machine learning methods implementation. It is also a good practice to use Python basic. As it is suggested, everything function is constructed from scratch. I really enjoyed the book, however I would not recommend it to learn ML and go directly to developing ML applications.
I rate it 4 because , some examples shown in the book do not provide data to test them

[Samuel · Rating 2 out of 5]

I have started this the second time now.

I really like the basic idea of doing things "from scratch", to get a better understanding, but I realize that it really requires you to run through pretty much every code example to follow intelligibly. Add to this that it starts fairly basic, I feel it is taking just a bit too much of my time to seem worth it. Considering dropping it. We'll see. Probably great for someone very new to python though.