Regression Model Evaluation in TensorFlow: MAE and MSE
Model evaluation is the process that measures how well a trained model predicts on data it has never seen. In my previous post, I built several simple regression models with TensorFlow’s Sequential API. Here I go in-depth on evaluating those models using a held-out testing dataset and the Mean Absolute Error (MAE) and Mean Squared Error (MSE) metrics.
Data Preparation: Train/Test Split with tf.range()
First, to keep results reproducible, I set a random seed (check my previous post on TensorFlow seeds if you’re curious how that works). As in the post on regression in TensorFlow, I use the tf.range() function to generate a set of X input values,
and y outputs, as follows:
# Creating a random seed
tf.random.set_seed(57)
# Generating data
X = tf.range(-100, 300, 4)
y = X + 7
X, y
(<tf.Tensor: shape=(100,), dtype=int32, numpy=
array([-100, -96, -92, -88, -84, -80, -76, -72, -68, -64, -60,
-56, -52, -48, -44, -40, -36, -32, -28, -24, -20, -16,
-12, -8, -4, 0, 4, 8, 12, 16, 20, 24, 28,
32, 36, 40, 44, 48, 52, 56, 60, 64, 68, 72,
76, 80, 84, 88, 92, 96, 100, 104, 108, 112, 116,
120, 124, 128, 132, 136, 140, 144, 148, 152, 156, 160,
164, 168, 172, 176, 180, 184, 188, 192, 196, 200, 204,
208, 212, 216, 220, 224, 228, 232, 236, 240, 244, 248,
252, 256, 260, 264, 268, 272, 276, 280, 284, 288, 292,
296], dtype=int32)>, <tf.Tensor: shape=(100,), dtype=int32, numpy=
array([-93, -89, -85, -81, -77, -73, -69, -65, -61, -57, -53, -49, -45,
-41, -37, -33, -29, -25, -21, -17, -13, -9, -5, -1, 3, 7,
11, 15, 19, 23, 27, 31, 35, 39, 43, 47, 51, 55, 59,
63, 67, 71, 75, 79, 83, 87, 91, 95, 99, 103, 107, 111,
115, 119, 123, 127, 131, 135, 139, 143, 147, 151, 155, 159, 163,
167, 171, 175, 179, 183, 187, 191, 195, 199, 203, 207, 211, 215,
219, 223, 227, 231, 235, 239, 243, 247, 251, 255, 259, 263, 267,
271, 275, 279, 283, 287, 291, 295, 299, 303], dtype=int32)>)
I separate the training and testing datasets for the respective model training and evaluation steps
with a split_data() function:
# Split data into train and test sets
def split_data(X, y):
X_train = X[:80] # First 80% of the data
y_train = y[:80]
X_test = X[80:] # last 20% percent of the data
y_test = y[80:]
return(X_train, X_test, y_train, y_test)
(X_train, X_test, y_train, y_test) = split_data(X, y)
X_train = tf.expand_dims(X_train, axis = -1)
X_test = tf.expand_dims(X_test, axis = -1)
Calculating MAE and MSE Error Metrics
Mean Absolute Error (MAE) is a regression metric that averages the absolute differences between predicted and actual values, treating every error proportionally. Mean Squared Error (MSE) is a regression metric that squares each error before averaging, which amplifies the impact of significant outliers. Given the predicted y_pred and the testing data y_test, I compute both with [tf.keras.losses][1]. MAE and MSE are the usual metrics for regression problems; you can also use the [Huber loss][2], which behaves like MSE for small errors and like MAE for large ones, making it less sensitive to outliers.
Creating Sequential Models with Tunable Hyperparameters
🔒 Subscribe to keep reading.
Evaluating Models with model.predict() and Error Metrics
🔒 Subscribe to keep reading.
Final Thoughts
🔒 Subscribe to keep reading.
References
🔒 Subscribe to keep reading.
