Elena' s AI Blog

TensorFlow: Regression Model

21 Jan 2022 (updated: 06 Aug 2026) / 32 minutes to read

Elena Daehnhardt


 Flux: A smooth curved trend line fitting scattered data points on a soft grid, representing a neural network learning a num...


TL;DR:
  • Build TensorFlow regression models: use Sequential API, compile with loss='mae', fit with train/test split. Adjust epochs, learning rate, and layers to improve predictions for numerical values.

📚 This post is part of the "Machine Learning" series

Series: Machine Learning (Part 7 of 27)

Previous: Part 6 — Tensors in TensorFlow

Next: Part 8 — TensorFlow: Global and Operation-level Seeds

What Is Regression in Machine Learning?

Regression is a supervised machine learning task that predicts a continuous numerical value from one or more input features. Regression is defined in Wikipedia as:

In statistical modeling, regression analysis is a set of statistical processes for estimating the relationships between a dependent variable (often called the ‘outcome’ or ‘response’ variable) and one or more independent variables (often called ‘predictors,’ ‘covariates,’ ‘explanatory variables’ or ‘features’). The most common form of regression analysis is linear regression. One finds the line (or a more complex linear combination) that most closely fits the data according to a specific mathematical criterion.

In plain terms, you want to predict a numerical value from other numerical values, as described in the TensorFlow Developer Certificate course. In Machine Learning, regression analysis is widely used for prediction and forecasting. For instance, you can use regression models to predict house sale prices, modelled against the number of bedrooms, bathrooms, or garages. Other applications of regression include predicting how many people will buy an app, forecasting seasonal sales, and even predicting bounding-box coordinates in an object detection task. With regression, you are answering the questions “How many?” and “How much?”

Building a Regression Model in TensorFlow: Three Steps

When you build neural networks in TensorFlow with the Keras Sequential API, you generally follow three steps:

  1. create a model and define the input, hidden and output layers, and the number of neurons in each layer;
  2. compile the model with the required loss function, optimiser, and evaluation metrics;
  3. fit the model to find patterns between features and labels.

In the code below, I create a simple regression model with input features stored in X and output stored in y. For demonstration purposes, I kept the data small — just enough to show how you build a regression model and then improve it by adjusting hyperparameters at the compile and fit steps.

Linear regression suits a linear trend in the data, and mine fits one: each y-value here can be predicted as X+7.

Dataset: y=f(X)

The neural network below consists of a single hidden layer with one neuron, which is all the regression functionality this example needs.

# Regression example with TensorFlow
import tensorflow as tf
# print(tf.__version__) # 2.7.0

import numpy as np
import matplotlib.pyplot as plt

# Set a random seed for reproducibility
tf.random.set_seed(57)

# Create features
X = np.array([-5., -3., -1., 1., 3., 4., 5., 8.])

# Create labels
y = np.array([2., 4., 6., 8., 10., 11., 12., 15.])

# Turn input and output Numpy arrays into tensors with float32 data type
X = tf.cast(tf.constant(X), dtype=tf.float32)
y = tf.cast(tf.constant(y), dtype=tf.float32)

# Visualise it
plt.scatter(X, y)
plt.xlabel("X")
plt.ylabel("y")

# Expand the X shape for the new version of TensorFlow
X = tf.expand_dims(X, axis=-1)

# Create a Sequential model
model = tf.keras.Sequential([
    tf.keras.layers.Dense(1)
])

# Compile the model with Mean Absolute Error (MAE) loss function and SGD optimiser
model.compile(loss=tf.keras.losses.mae,
              optimizer=tf.keras.optimizers.SGD(), 
              metrics=["mae"])

# Fit the model
model.fit(X, y, epochs=5)

# Make predictions
model.predict(np.array([[10]]))

Epoch 1/5
1/1 [==============================] - 0s 252ms/step - loss: 6.7141 - mae: 6.7141
Epoch 2/5
1/1 [==============================] - 0s 6ms/step - loss: 6.6816 - mae: 6.6816
Epoch 3/5
1/1 [==============================] - 0s 6ms/step - loss: 6.6491 - mae: 6.6491
Epoch 4/5
1/1 [==============================] - 0s 6ms/step - loss: 6.6166 - mae: 6.6166
Epoch 5/5
1/1 [==============================] - 0s 5ms/step - loss: 6.5841 - mae: 6.5841

Note that I wrap the input to predict() in np.array() — current Keras versions reject a plain Python list here and expect a NumPy array, tensor, or tf.data.Dataset instead (see the Model.predict documentation).

I tried to predict the unseen value 10 and got a fairly high loss and MAE. The model predicted 12 instead of the 17 I expected — five epochs with the default SGD optimiser just is not enough for this to converge.

Improving Regression Model Performance: Hyperparameter Tuning

To improve a regression model, you can apply the following hyperparameter adjustments in any combination — change one thing at a time so you can actually tell what helped, as recommended in the TensorFlow Developer Certificate course:

  1. adding or removing layers;
  2. increasing the number of hidden neurons;
  3. changing the activation function;
  4. changing the optimisation function;
  5. adjusting the learning rate (potentially, the most critical hyperparameter);
  6. adding more data;
  7. increasing the number of epochs.
# Rebuild the model
# Create a Sequential model
model = tf.keras.Sequential([
    tf.keras.layers.Dense(1)
])

# Compile the model with MAE loss function and SGD optimiser
model.compile(loss=tf.keras.losses.mae,
              optimizer=tf.keras.optimizers.Adam(learning_rate=0.1), 
              metrics=["mae"])

# Fit the model
model.fit(X, y, epochs=50)

# Make predictions
model.predict(np.array([[10]]))

/usr/local/lib/python3.7/dist-packages/keras/optimizer_v2/adam.py:105: UserWarning: The `lr` argument is deprecated, use `learning_rate` instead.
  super(Adam, self).__init__(name, **kwargs)
Epoch 1/50
1/1 [==============================] - 0s 261ms/step - loss: 10.2923 - mae: 10.2923
Epoch 2/50
1/1 [==============================] - 0s 9ms/step - loss: 9.9423 - mae: 9.9423
Epoch 3/50
1/1 [==============================] - 0s 7ms/step - loss: 9.5923 - mae: 9.5923
Epoch 4/50
1/1 [==============================] - 0s 5ms/step - loss: 9.2423 - mae: 9.2423
Epoch 5/50
1/1 [==============================] - 0s 5ms/step - loss: 8.8923 - mae: 8.8923
Epoch 6/50
1/1 [==============================] - 0s 5ms/step - loss: 8.5423 - mae: 8.5423
Epoch 7/50
1/1 [==============================] - 0s 8ms/step - loss: 8.2503 - mae: 8.2503
Epoch 8/50
1/1 [==============================] - 0s 5ms/step - loss: 8.0050 - mae: 8.0050
Epoch 9/50
1/1 [==============================] - 0s 6ms/step - loss: 7.7632 - mae: 7.7632
Epoch 10/50
1/1 [==============================] - 0s 5ms/step - loss: 7.5239 - mae: 7.5239
Epoch 11/50
1/1 [==============================] - 0s 5ms/step - loss: 7.2868 - mae: 7.2868
Epoch 12/50
1/1 [==============================] - 0s 6ms/step - loss: 7.0512 - mae: 7.0512
Epoch 13/50
1/1 [==============================] - 0s 5ms/step - loss: 6.8168 - mae: 6.8168
Epoch 14/50
1/1 [==============================] - 0s 6ms/step - loss: 6.5835 - mae: 6.5835
Epoch 15/50
1/1 [==============================] - 0s 4ms/step - loss: 6.3509 - mae: 6.3509
Epoch 16/50
1/1 [==============================] - 0s 7ms/step - loss: 6.1189 - mae: 6.1189
Epoch 17/50
1/1 [==============================] - 0s 8ms/step - loss: 5.8875 - mae: 5.8875
Epoch 18/50
1/1 [==============================] - 0s 6ms/step - loss: 5.6563 - mae: 5.6563
Epoch 19/50
1/1 [==============================] - 0s 8ms/step - loss: 5.4255 - mae: 5.4255
Epoch 20/50
1/1 [==============================] - 0s 7ms/step - loss: 5.1948 - mae: 5.1948
Epoch 21/50
1/1 [==============================] - 0s 5ms/step - loss: 4.9642 - mae: 4.9642
Epoch 22/50
1/1 [==============================] - 0s 11ms/step - loss: 4.7337 - mae: 4.7337
Epoch 23/50
1/1 [==============================] - 0s 7ms/step - loss: 4.5031 - mae: 4.5031
Epoch 24/50
1/1 [==============================] - 0s 11ms/step - loss: 4.2726 - mae: 4.2726
Epoch 25/50
1/1 [==============================] - 0s 7ms/step - loss: 4.0420 - mae: 4.0420
Epoch 26/50
1/1 [==============================] - 0s 6ms/step - loss: 3.8113 - mae: 3.8113
Epoch 27/50
1/1 [==============================] - 0s 10ms/step - loss: 3.5804 - mae: 3.5804
Epoch 28/50
1/1 [==============================] - 0s 9ms/step - loss: 3.4646 - mae: 3.4646
Epoch 29/50
1/1 [==============================] - 0s 5ms/step - loss: 3.4254 - mae: 3.4254
Epoch 30/50
1/1 [==============================] - 0s 4ms/step - loss: 3.3826 - mae: 3.3826
Epoch 31/50
1/1 [==============================] - 0s 5ms/step - loss: 3.3365 - mae: 3.3365
Epoch 32/50
1/1 [==============================] - 0s 8ms/step - loss: 3.3657 - mae: 3.3657
Epoch 33/50
1/1 [==============================] - 0s 7ms/step - loss: 3.3754 - mae: 3.3754
Epoch 34/50
1/1 [==============================] - 0s 7ms/step - loss: 3.3626 - mae: 3.3626
Epoch 35/50
1/1 [==============================] - 0s 9ms/step - loss: 3.3297 - mae: 3.3297
Epoch 36/50
1/1 [==============================] - 0s 6ms/step - loss: 3.2920 - mae: 3.2920
Epoch 37/50
1/1 [==============================] - 0s 10ms/step - loss: 3.2169 - mae: 3.2169
Epoch 38/50
1/1 [==============================] - 0s 9ms/step - loss: 3.1071 - mae: 3.1071
Epoch 39/50
1/1 [==============================] - 0s 9ms/step - loss: 3.0001 - mae: 3.0001
Epoch 40/50
1/1 [==============================] - 0s 5ms/step - loss: 2.8834 - mae: 2.8834
Epoch 41/50
1/1 [==============================] - 0s 8ms/step - loss: 2.7579 - mae: 2.7579
Epoch 42/50
1/1 [==============================] - 0s 6ms/step - loss: 2.6245 - mae: 2.6245
Epoch 43/50
1/1 [==============================] - 0s 8ms/step - loss: 2.5000 - mae: 2.5000
Epoch 44/50
1/1 [==============================] - 0s 7ms/step - loss: 2.4162 - mae: 2.4162
Epoch 45/50
1/1 [==============================] - 0s 7ms/step - loss: 2.3321 - mae: 2.3321
Epoch 46/50
1/1 [==============================] - 0s 6ms/step - loss: 2.2477 - mae: 2.2477
Epoch 47/50
1/1 [==============================] - 0s 4ms/step - loss: 2.1632 - mae: 2.1632
Epoch 48/50
1/1 [==============================] - 0s 6ms/step - loss: 2.0949 - mae: 2.0949
Epoch 49/50
1/1 [==============================] - 0s 7ms/step - loss: 2.0640 - mae: 2.0640
Epoch 50/50
1/1 [==============================] - 0s 6ms/step - loss: 2.0114 - mae: 2.0114
array([[17.117014]], dtype=float32)

That UserWarning in the captured output above is from the TensorFlow 2.7 environment I originally wrote this post against: Keras renamed the lr keyword to learning_rate for every optimiser back in the Keras 2.3.0 release, and at that point lr still worked with just a warning. That backward compatibility is gone now — on today’s Keras 3 (the default since TensorFlow 2.16), passing lr= raises ValueError: Argument(s) not recognized: {'lr': ...} instead of a warning, so I have updated the code above to use learning_rate=0.1 directly. See the tf.keras.optimizers.Adam documentation for the current signature.

The loss and MAE are much lower now, and the prediction (17.1) is close to the expected 17. I got there by switching to the Adam optimiser with a learning rate of 0.1 and raising epochs from 5 to 50.

Evaluating Regression Models with Larger Datasets

In practice, you repeat building and fitting a model with different combinations of hyperparameters. During training, the model learns patterns from the data — finding parameters such as weights. You evaluate the model, tweak the hyperparameters, and repeat the build-fit-evaluate cycle until you get a reasonably good performance.

A well-performing model generally needs a larger dataset than the eight points I used above — the more complex the task, the more data you need. Let’s generate a larger dataset for the regression example.

# Generate a larger data
X = tf.range(-100, 300, 4)
X

<tf.Tensor: shape=(100,), dtype=int32, numpy=
array([-100,  -96,  -92,  -88,  -84,  -80,  -76,  -72,  -68,  -64,  -60,
        -56,  -52,  -48,  -44,  -40,  -36,  -32,  -28,  -24,  -20,  -16,
        -12,   -8,   -4,    0,    4,    8,   12,   16,   20,   24,   28,
         32,   36,   40,   44,   48,   52,   56,   60,   64,   68,   72,
         76,   80,   84,   88,   92,   96,  100,  104,  108,  112,  116,
        120,  124,  128,  132,  136,  140,  144,  148,  152,  156,  160,
        164,  168,  172,  176,  180,  184,  188,  192,  196,  200,  204,
        208,  212,  216,  220,  224,  228,  232,  236,  240,  244,  248,
        252,  256,  260,  264,  268,  272,  276,  280,  284,  288,  292,
        296], dtype=int32)>

# Make labels for the dataset
y = X + 7
y

<tf.Tensor: shape=(100,), dtype=int32, numpy=
array([-93, -89, -85, -81, -77, -73, -69, -65, -61, -57, -53, -49, -45,
       -41, -37, -33, -29, -25, -21, -17, -13,  -9,  -5,  -1,   3,   7,
        11,  15,  19,  23,  27,  31,  35,  39,  43,  47,  51,  55,  59,
        63,  67,  71,  75,  79,  83,  87,  91,  95,  99, 103, 107, 111,
       115, 119, 123, 127, 131, 135, 139, 143, 147, 151, 155, 159, 163,
       167, 171, 175, 179, 183, 187, 191, 195, 199, 203, 207, 211, 215,
       219, 223, 227, 231, 235, 239, 243, 247, 251, 255, 259, 263, 267,
       271, 275, 279, 283, 287, 291, 295, 299, 303], dtype=int32)>

Visualising Regression Data and Model Structure

Data visualisation helps you understand the dataset better. You can also plot training and test predictions against the ground truth (or labels), and draw the model’s structure itself with TensorFlow’s plot_model functionality.

Train, Validation, and Test Dataset Splits

When you train a model, you want to make sure it does not just learn the data “by heart” but also works well on unseen data. This property is called model generalisation, and you support it by splitting the dataset into three subsets, as recommended in Google’s Machine Learning Crash Course:

  1. the training dataset to train your model (roughly 70-80% of all data);
  2. the validation dataset to tune the model (roughly 10-15% of the data);
  3. the test dataset to evaluate the final model (roughly 10-15% of the data).

These are rules of thumb rather than fixed rules — the right split depends on how much data you have overall.

For simplicity, I split this dataset into just training and testing sets:

# Split data into train and test sets
X_train = X[:80] # First 80% of the data
y_train = y[:80] 

# Test data
X_test = X[80:] # last 20% percent of the data
y_test = y[80:]

# Visualising the data split into train an test sets
plt.figure(figsize=(10, 7))

# Plot training data in blue
plt.scatter(X_train, y_train, c="b", label="Training data")

# Plot test data in red
plt.scatter(X_test, y_test, c="r", label="Testing data")
plt.legend()

plt.xlabel("X")
plt.ylabel("y")

The figure shows the dataset with training data points in blue and testing data points in red.

Larger dataset: y=f(X)

I will also cover cross-validation in my next post.

Making Predictions with a Trained Regression Model

Let’s create a new model and fit it with the training dataset. The neural network has 3 neurons in the first hidden layer and one in the output layer. I track Mean Absolute Error (MAE) and Mean Squared Error (MSE) as metrics while using the Adam optimiser at the compile step.

Current Keras Dense layers need an explicit feature dimension, so X has to be 2-D before you fit — hence the expand_dims call below stays active this time (I left it commented out further up only because the 1-D tensors worked without it in the exact TensorFlow build this post was written against).

# Expanding X to fit the current TensorFlow version requirement
X = tf.expand_dims(X, axis=-1)

# Create a Sequential model
model = tf.keras.Sequential([
    tf.keras.layers.Dense(3, name="first_layer"),                         
    tf.keras.layers.Dense(1, name="output_layer")
])


# Compile the model with MAE loss function and SGD optimiser
model.compile(loss=tf.keras.losses.mae,
              optimizer=tf.keras.optimizers.Adam(learning_rate=0.1), 
              metrics=["mae", "mse"])

# Fit the model
model.fit(X, y, epochs=50)

/usr/local/lib/python3.7/dist-packages/keras/optimizer_v2/adam.py:105: UserWarning: The `lr` argument is deprecated, use `learning_rate` instead.
  super(Adam, self).__init__(name, **kwargs)
Epoch 1/50
4/4 [==============================] - 0s 3ms/step - loss: 120.9953 - mae: 120.9953 - mse: 23894.6367
Epoch 2/50
4/4 [==============================] - 0s 4ms/step - loss: 37.3695 - mae: 37.3695 - mse: 2743.7510
Epoch 3/50
4/4 [==============================] - 0s 4ms/step - loss: 30.2086 - mae: 30.2086 - mse: 1857.1360
Epoch 4/50
4/4 [==============================] - 0s 3ms/step - loss: 36.0968 - mae: 36.0968 - mse: 2010.0787
Epoch 5/50
4/4 [==============================] - 0s 4ms/step - loss: 22.9426 - mae: 22.9426 - mse: 899.0366
Epoch 6/50
4/4 [==============================] - 0s 5ms/step - loss: 19.2425 - mae: 19.2425 - mse: 539.2803
Epoch 7/50
4/4 [==============================] - 0s 4ms/step - loss: 11.3198 - mae: 11.3198 - mse: 230.8450
Epoch 8/50
4/4 [==============================] - 0s 4ms/step - loss: 11.6120 - mae: 11.6120 - mse: 245.3984
Epoch 9/50
4/4 [==============================] - 0s 4ms/step - loss: 16.8810 - mae: 16.8810 - mse: 422.8721
Epoch 10/50
4/4 [==============================] - 0s 5ms/step - loss: 15.4557 - mae: 15.4557 - mse: 409.4062
Epoch 11/50
4/4 [==============================] - 0s 4ms/step - loss: 9.3804 - mae: 9.3804 - mse: 163.5114
Epoch 12/50
4/4 [==============================] - 0s 4ms/step - loss: 6.3950 - mae: 6.3950 - mse: 63.2417
Epoch 13/50
4/4 [==============================] - 0s 3ms/step - loss: 7.1727 - mae: 7.1727 - mse: 97.3974
Epoch 14/50
4/4 [==============================] - 0s 3ms/step - loss: 4.3348 - mae: 4.3348 - mse: 30.1714
Epoch 15/50
4/4 [==============================] - 0s 3ms/step - loss: 7.2365 - mae: 7.2365 - mse: 97.9294
Epoch 16/50
4/4 [==============================] - 0s 4ms/step - loss: 7.9825 - mae: 7.9825 - mse: 122.1995
Epoch 17/50
4/4 [==============================] - 0s 4ms/step - loss: 7.0333 - mae: 7.0333 - mse: 95.2871
Epoch 18/50
4/4 [==============================] - 0s 3ms/step - loss: 6.7239 - mae: 6.7239 - mse: 74.4304
Epoch 19/50
4/4 [==============================] - 0s 4ms/step - loss: 4.4470 - mae: 4.4470 - mse: 31.7933
Epoch 20/50
4/4 [==============================] - 0s 4ms/step - loss: 1.7557 - mae: 1.7557 - mse: 4.8011
Epoch 21/50
4/4 [==============================] - 0s 5ms/step - loss: 2.2091 - mae: 2.2091 - mse: 7.5242
Epoch 22/50
4/4 [==============================] - 0s 4ms/step - loss: 3.2336 - mae: 3.2336 - mse: 20.6686
Epoch 23/50
4/4 [==============================] - 0s 3ms/step - loss: 6.2744 - mae: 6.2744 - mse: 77.9671
Epoch 24/50
4/4 [==============================] - 0s 4ms/step - loss: 9.0602 - mae: 9.0602 - mse: 135.6305
Epoch 25/50
4/4 [==============================] - 0s 3ms/step - loss: 3.6392 - mae: 3.6392 - mse: 21.3788
Epoch 26/50
4/4 [==============================] - 0s 4ms/step - loss: 1.2313 - mae: 1.2313 - mse: 2.7846
Epoch 27/50
4/4 [==============================] - 0s 3ms/step - loss: 4.6018 - mae: 4.6018 - mse: 35.7042
Epoch 28/50
4/4 [==============================] - 0s 3ms/step - loss: 5.4567 - mae: 5.4567 - mse: 53.5563
Epoch 29/50
4/4 [==============================] - 0s 4ms/step - loss: 3.9640 - mae: 3.9640 - mse: 33.2863
Epoch 30/50
4/4 [==============================] - 0s 3ms/step - loss: 2.6091 - mae: 2.6091 - mse: 13.5995
Epoch 31/50
4/4 [==============================] - 0s 5ms/step - loss: 7.2700 - mae: 7.2700 - mse: 86.4028
Epoch 32/50
4/4 [==============================] - 0s 3ms/step - loss: 10.4771 - mae: 10.4771 - mse: 171.6588
Epoch 33/50
4/4 [==============================] - 0s 3ms/step - loss: 8.3980 - mae: 8.3980 - mse: 161.0003
Epoch 34/50
4/4 [==============================] - 0s 4ms/step - loss: 4.2892 - mae: 4.2892 - mse: 43.4268
Epoch 35/50
4/4 [==============================] - 0s 3ms/step - loss: 1.9115 - mae: 1.9115 - mse: 8.9037
Epoch 36/50
4/4 [==============================] - 0s 3ms/step - loss: 5.1150 - mae: 5.1150 - mse: 46.9002
Epoch 37/50
4/4 [==============================] - 0s 4ms/step - loss: 6.2562 - mae: 6.2562 - mse: 74.0645
Epoch 38/50
4/4 [==============================] - 0s 3ms/step - loss: 3.2418 - mae: 3.2418 - mse: 17.0526
Epoch 39/50
4/4 [==============================] - 0s 5ms/step - loss: 3.0819 - mae: 3.0819 - mse: 18.1384
Epoch 40/50
4/4 [==============================] - 0s 4ms/step - loss: 2.2617 - mae: 2.2617 - mse: 9.4483
Epoch 41/50
4/4 [==============================] - 0s 3ms/step - loss: 4.8202 - mae: 4.8202 - mse: 47.1045
Epoch 42/50
4/4 [==============================] - 0s 4ms/step - loss: 7.5390 - mae: 7.5390 - mse: 125.5217
Epoch 43/50
4/4 [==============================] - 0s 3ms/step - loss: 4.8652 - mae: 4.8652 - mse: 39.2499
Epoch 44/50
4/4 [==============================] - 0s 4ms/step - loss: 4.8499 - mae: 4.8499 - mse: 42.2790
Epoch 45/50
4/4 [==============================] - 0s 4ms/step - loss: 1.0383 - mae: 1.0383 - mse: 3.7415
Epoch 46/50
4/4 [==============================] - 0s 3ms/step - loss: 0.8587 - mae: 0.8587 - mse: 1.4711
Epoch 47/50
4/4 [==============================] - 0s 4ms/step - loss: 1.6601 - mae: 1.6601 - mse: 6.1024
Epoch 48/50
4/4 [==============================] - 0s 3ms/step - loss: 3.7125 - mae: 3.7125 - mse: 30.4232
Epoch 49/50
4/4 [==============================] - 0s 6ms/step - loss: 5.7093 - mae: 5.7093 - mse: 60.4152
Epoch 50/50
4/4 [==============================] - 0s 4ms/step - loss: 5.1068 - mae: 5.1068 - mse: 39.9806

I use the plot_model() function to draw the network shown below.

Model Graph with plot_model()

# Import plot_model
from tensorflow.keras.utils import plot_model

plot_model(model, show_shapes=True)

It helps to see how far the model predictions are from the ground truth, so let’s visualise the predictions against the ground truth in a scatter plot.

# Make some predictions
y_pred = model.predict(X_test)

# Print the predicted and ground truth values
for predicted, truth in zip(y_pred, y_test):
  print("Ground truth:%d    predicted:%f"%(truth.numpy(), predicted[0]))

Ground truth:227    predicted:217.726746
Ground truth:231    predicted:221.557556
Ground truth:235    predicted:225.388351
Ground truth:239    predicted:229.219147
Ground truth:243    predicted:233.049957
Ground truth:247    predicted:236.880753
Ground truth:251    predicted:240.711548
Ground truth:255    predicted:244.542343
Ground truth:259    predicted:248.373154
Ground truth:263    predicted:252.203949
Ground truth:267    predicted:256.034760
Ground truth:271    predicted:259.865570
Ground truth:275    predicted:263.696350
Ground truth:279    predicted:267.527161
Ground truth:283    predicted:271.357971
Ground truth:287    predicted:275.188782
Ground truth:291    predicted:279.019562
Ground truth:295    predicted:282.850372
Ground truth:299    predicted:286.681152
Ground truth:303    predicted:290.511963

Finally, I write a plotting function to draw scatter plots of the training, testing, and predicted datasets.

# A plotting function
def plot_data(train_data=X_train, train_labels=y_train, 
                     test_data=X_test, test_labels=y_test, predictions=y_pred):
  plt.figure(figsize=(10, 5))
  # Drawing scatter plots
  plt.scatter(train_data, train_labels, c="g", label="Training data")
  plt.scatter(test_data, test_labels, c="b", label="Testing data")
  plt.scatter(test_data, predictions, c="r", label="Predictions data")
  plt.legend()

plot_data()

As the plot shows, the predicted data sits a little below the testing data values.

Larger dataset: y=f(X)

Evaluating Model Predictions with MAE and MSE

You can use the model’s evaluate() method with the testing dataset to check the model predictions. Since I defined the MAE and MSE metrics during compilation, I get both values back in the result.

# Evaluate the model on the test set
model.evaluate(X_test, y_test)

1/1 [==============================] - 0s 125ms/step - loss: 10.4243 - mae: 10.4243 - mse: 109.5560
[10.424253463745117, 10.424253463745117, 109.55604553222656]

You can also calculate the MAE and MSE directly, given the predictions (y_pred) and ground truth labels (y_test). The functional tf.metrics.mean_absolute_error/mean_squared_error aliases this post originally used were dropped when Keras moved to its version-3 backend, so I now call the equivalent tf.keras.losses.mae and tf.keras.losses.mse functions instead — same maths, current API.

# Calculate the Mean Absolute Error (MAE) with y_pred and y_test
# Please note that we remove an extra dimension with tf.squeeze
mae = tf.keras.losses.mae(y_true=tf.cast(y_test, tf.float32), y_pred=tf.squeeze(y_pred))

# Calculate the Mean Square Error (MSE)
mse = tf.keras.losses.mse(y_true=tf.cast(y_test, tf.float32), y_pred=tf.squeeze(y_pred))

print("MAE=%f, MSE=%f"%(mae, mse))

MAE=10.415435, MSE=109.370789

Did you like this post? Please let me know if you have any comments or suggestions.

Python posts that might be interesting for you



Final Thoughts

A TensorFlow regression model predicts a continuous numerical value through three steps: building a Keras Sequential model, compiling it with a loss function and optimiser, and fitting it to data. In this post, I generated a dataset, split it into training and testing sets, visualised the data and network structure, tuned hyperparameters (Adam optimiser with learning_rate=0.1, 50 epochs), and evaluated predictions with MAE and MSE. Next, I cover model evaluation and the widely used cross-validation technique.

TensorFlow Regression FAQ

How do you build a regression model in TensorFlow?

Follow three steps with the Keras Sequential API: (1) create the model and define its layers with tf.keras.Sequential([tf.keras.layers.Dense(1)]); (2) compile it with a loss function, optimiser, and metrics, e.g. model.compile(loss=tf.keras.losses.mae, optimizer=tf.keras.optimizers.Adam(), metrics=['mae']); (3) fit it to the data with model.fit(X, y, epochs=50).

Which loss function and metric should I use for a TensorFlow regression model?

Use Mean Absolute Error (MAE, tf.keras.losses.mae) or Mean Squared Error (MSE) as the loss for predicting numerical values. Track mae and mse as metrics during compilation so model.evaluate() returns them for the test set. MAE reports the average absolute prediction error in the target’s units; MSE penalises larger errors more heavily.

How can I improve a regression model’s predictions in TensorFlow?

Adjust hyperparameters one at a time: add or remove layers, change the number of hidden neurons, change the activation or optimisation function, tune the learning rate (often the most critical hyperparameter), add more data, or increase the number of epochs. In this post, switching to the Adam optimiser with learning_rate=0.1 and raising epochs from 5 to 50 moved the prediction for input 10 from 12 to 17.1, close to the expected 17.

What does the warning ‘The lr argument is deprecated, use learning_rate instead’ mean?

Keras renamed the optimiser argument lr to learning_rate in the 2.3.0 release. On older Keras versions this only produced a UserWarning; on current Keras 3 (bundled with TensorFlow 2.16+), passing lr= raises a hard ValueError: Argument(s) not recognized. Replace tf.keras.optimizers.Adam(lr=0.1) with tf.keras.optimizers.Adam(learning_rate=0.1) to fix it.

References

  1. TensorFlow Developer Certificate in 2022: Zero to Mastery
desktop bg dark

About Elena

Elena, a PhD in Computer Science, simplifies AI concepts and helps you use machine learning.

Citation
Elena Daehnhardt. (2022) 'TensorFlow: Regression Model', daehnhardt.com, 21 January 2022. Available at: https://daehnhardt.com/blog/2022/01/21/tf-regression/
All Posts
Copyright © Elena Daehnhardt
Privacy and Terms