Feature engineering, data transformation and normalization

Feature engineering creates or transforms input variables so a model can learn better, for example extracting the month from a date or encoding categories as numbers. Categorical variables are encoded with one-hot encoding for nominal categories or ordinal encoding when an order exists.

Scaling puts numeric features on comparable ranges:

Worked example: for values 10, 20, 30, 40, 50 min-max scaling maps 20 to (20 - 10) / 40 = 0.25.

Scaling matters for distance-based methods such as k-NN and K-Means and for gradient descent, because features with large ranges would dominate distances and slow the updates. Tree-based models are mostly insensitive to scaling. Scaling does not remove outliers and does not guarantee higher accuracy.

from sklearn.preprocessing import StandardScaler
sc = StandardScaler().fit(X_train)
X_train_s = sc.transform(X_train)
X_test_s = sc.transform(X_test)

Exam trap: fit the scaler on the training set only. Fitting it on the full dataset before splitting leaks test information into preprocessing and makes evaluation look better than it should.