Feature Engineering for Machine Learning: Principles and Techniques for Data Scientists

by Alice Zheng, Amanda Casari

Feature engineering is a crucial step in the machine-learning pipeline, yet this topic is rarely examined on its own. With this practical book, you’ll learn techniques for extracting and transforming features—the numeric representations of raw data—into formats for machine-learning models. Each chapter guides you through a single data problem, such as how to represent text or image data. Together, these examples illustrate the main principles of feature engineering.

Rather than simply teach these principles, authors Alice Zheng and Amanda Casari focus on practical application with exercises throughout the book. The closing chapter brings everything together by tackling a real-world, structured dataset with several feature-engineering techniques. Python packages including numpy, Pandas, Scikit-learn, and Matplotlib are used in code examples.

You’ll examine:

Feature engineering for numeric data: filtering, binning, scaling, log transforms, and power transforms Natural text techniques: bag-of-words, n-grams, and phrase detection Frequency-based filtering and feature scaling for eliminating uninformative features Encoding techniques of categorical variables, including feature hashing and bin-counting Model-based feature engineering with principal component analysis The concept of model stacking, using k-means as a featurization technique Image feature extraction with manual and deep-learning techniques

Genres: Programming, Computer Science, Science, Technology, Artificial Intelligence, Nonfiction, Technical

360 pages, Kindle Edition

Published March 23, 2018

About the author

Alice is a technical leader in the field of machine learning. Her experience spans algorithm and platform development and applications. Currently, she is a Senior Manager in Amazon's Ad Platform. Previous roles include Director of Data Science at GraphLab/Dato/Turi, machine learning researcher at Microsoft Research, Redmond, and postdoctoral fellow at Carnegie Mellon University. She received a Ph.D. in Electrical Engineering and Computer science, and B.A. degrees in Computer Science in Mathematics, all from U.C. Berkeley.

Reviews

[Ahmed · Rating 3 out of 5]

الكتاب بيتكلم عن فكرة الـ Feature Engineering، يعني إزاي تاخد بيانات زي النصوص أو الأرقام أو الصور، وتظبطها عشان تبقى جاهزة للموديل.

الفصول اللي عجبتني في الكتاب:

* الفصل التالت: بيتكلم عن النصوص، إزاي تحول كلام عادي لـ Bag-of-Words أو n-Grams، وتنضف الكلام من الزيادة زي Stopwords.

* الفصل الرابع: هنا بيخش على tf-idf، ودي طريقة تخلّيك تعرف الكلمات المهمة في النص بدل ما تسيب كل حاجة زي بعضها.

* الفصل التامن: بيدخل في الصور، وإزاي تستخرج منها ميزات زي SIFT و HOG، أو تستخدم الشبكات العصبية زي AlexNet.

سلبيات:

* الفصل الأخير: ده اللي الكتاب مركز عليه، بيتكلم عن إزاي أبني نظام توصية للأوراق الأبحاث حجمه 2.5 جيجا. المفروض إن الفصل ده هو المثال العملي بتاع الكتاب، لكن مكنش مكتمل بشكل نهائي. يعني نتيجة الموديل في النهاية مكنتش كويسة، وكان ممكن يعملوا تعديلات عليه عشان يحسنوا نتيجة التوصيات.

باختصار: الفصل كان مركز جدًا على التجربة خطوة بخطوة (من استيراد البيانات لتعديل الميزات)، لكن ما اداش مساحة كبيرة لتحليل النتايج أو اقتراح حلول مختلفة، لأن نتيجة النموذج مكنتش أفضل حاجة، وتحس إن أهم فصل في الكتاب مكنش مكتمل.

* أحيانًا بيخش في تفاصيل تقنية زيادة (زي PCA في الفصل السادس)، ممكن تدوخ لو مش متظبط في الرياضيات، وفي نفس الوقت كان ممكن يختصر.<

[Mahmoud Rabie · Rating 4 out of 5]

I liked the book a lot, yes the book contains a lot of math but the author tried to always explain in simple words

The book don't focus on giving a tips and tricks or a how to guide for feature engineering, instead it focus on the effect of feature engineering on the data and the models

i.e. the effect of using log transformation on linear regression

As the book size is small - around 200 pages - the book don't cover a lot of topics, but at the end of the book your understanding for feature engineering will be changed and you will get a good deep idea of the impact of feature engineering phase of your model training and performance

[Amir Sarabadani · Rating 4 out of 5]

Most of the graphs are hand-written, that was weird.

[Rick Sam · Rating 3 out of 5]

A Quick read, teaches you basics in Feature Engineering, this might give you raw knowledge about Feature Engineering, meanwhile in Production or Industry, one has to practice or apply based on the the raw-knowledge.

If someone could come up with a best way to gain know-how or procedural knowledge faster, do let me know.

I have a summary of this, if you want do PM me, it might save your time.

Here's an Outline of the Book:

0. Introduction and my thoughts
1. Machine Learning Pipeline
2. Fancy Tricks with Simple Numbers
3. Text Data: Flattening, Filtering and Chunking
4. Effects of Feature Scaling: From Bag of Words to TF-IDF
5. Categorical Variables: Counting Eggs in Age of Robotic Chickens
6. Dimensionality Reduction: Squashing Data Pancake with PCA
7. Non-linear Featurization with K-Means Model Stacking
8. Automating Featurizer: Image Feature Extraction and Deep Learning
9. Back to the Feature: Building an Academic Paper Recommender
10. Linear Modeling and Linear Algebra Basics

I would recommend this to Statisticians, Computer Scientists, PhD researchers, Software Engineers

Deus Vult,
Gottfried

[Joe · Rating 2 out of 5]

Fine start, but there's not a lot here. There are some good tricks for people who don't have a lot of experience building predictive models.

The book also focuses on some specific types of data that lots of people won't need to work with: images and text (as in trying to get a computer to understand text). More general discussion about categorical and numeric variables would have been better, since those ideas can be translated to other contexts more easily.

This book will probably be obsolete within five years or so. I got the sense that the authors and publishers would agree with me. The book looks rushed. Most of the data visualizations are drawn by hand and scanned into the book. That's pretty lazy, since many of them could have been mocked up to look better with any visualization software (even Excel).

[Auggie Heschmeyer · Rating 2 out of 5]

This book is great if you want to know the exact mathematical expressions for the "feature engineering" part of the title, but a total drag if you're looking for anything practical. This book is clearly after a depth not breadth approach, but they go deep down the wrong paths. For instance, the final chapter is billed as a case study. However, rather than engineering the right features to do the task they lay out, they engineer some bad features then slap on some okay ones and say, "There you go. Just do that and that's feature engineering." They don't even give you a way to gauge how accurate their method was.

[Chaouki · Rating 5 out of 5]

I believe, in general ,women write better science books than Men. Male writing is most often rigged with pomp and ego and unrequired complexity. I guess it's because women have a thicker stomach and more patience to handle the irksome task to write a science book. With that said, this book is just my cup of tea , quick loose summary of everything that's necessary, oils up the cogs and softens up the road for further readings. Great book.
One more thing, O'Reilly books rock Hard 🤘

[Mehdi · Rating 3 out of 5]

Good book. I liked Chapter 2 5 and 9.

I wish more time was spent on:
* Feature transformation of numerical values
* Feature selection
* Determining which transformation is adequate with which model type (NN, Random Forest, ..)
* The incremental impact of additional features over other selected features

[Gene Ishchuk · Rating 5 out of 5]

it is quite good, maybe not 100% noob-friendly, it is for me to reread again but there is a list of concepts I haven't heard of before

[Virajdatt Kohir · Rating 3 out of 5]

Ummmm, good tricks thought it would help me (intermediate) but most of the things mentioned are for beginners. It did help me with revising things.

[Pawin · Rating 3 out of 5]

Concepts are not fully explained. The book is recommended for the readers who already grasp all basic concepts and just need further examples.

[Maria Pantsiou · Rating 5 out of 5]

Nice introduction to feature engineering, with examples and common scenarios that data scientists encounter daily.

[THN · Rating 3 out of 5]

Good for a general review, but too basic (e.g., not explaining the data distribution suitable for an engineering technique) and having some errors (the PCA chapter, although its explanation is good and useful).

[Mike Fowler · Rating 4 out of 5]

Good clear explanations of concepts with plenty of examples. Appendix with useful refresher on linear algebra.