Reproducible Randomness in TensorFlow: Why Seeds Matter

A random seed is an integer that initializes a pseudo-random number generator so it produces the same sequence of values on every run. When training Machine Learning models, we want to avoid ordering biases in the data while keeping that data order identical between runs or system restarts — for example, in Cross-Validation experiments. TensorFlow provides two seed types, global and operation-level, to achieve reproducibility of results ([tf.random.set_seed documentation][1]).

tf.random.set_seed Documentation: Global vs Operation-level Seeds in TensorFlow

TensorFlow defines two kinds of seeds, and their interaction determines reproducibility:

To begin, let’s create a mutable tensor with tf.Variable.

# Create a variable tensor
tensor = tf.Variable([[[1, 2, 3],
                       [4, 5, 6]],
                      [[7, 8, 9],
                       [10, 11, 12]],
                      [[13, 14, 15],
                       [16, 17, 18]]])

In the code below, I use the .assign() method to change the first element (a matrix) in the tensor, filling its values with zeros.

# Change elements of the first tensor element
tensor[0].assign([[0, 0, 0], [0, 0, 0]])

<tf.Variable 'UnreadVariable' shape=(3, 2, 3) dtype=int32, numpy=
array([[[ 0,  0,  0],
        [ 0,  0,  0]],

       [[ 7,  8,  9],
        [10, 11, 12]],

       [[13, 14, 15],
        [16, 17, 18]]], dtype=int32)>

Here’s the syntax for each. Set the global seed with tf.random.set_seed():

# Set a random seed with value of 57
tf.random.set_seed(57)

Set the operation-level seed directly on an operation, such as when shuffling the tensor with tf.random.shuffle():

tf.random.shuffle(tensor, seed=57)

The following sections cover all four combinations of these two seed types and whether each one guarantees reproducible results.

Scenario 1: No Seeds Defined (Not Reproducible)

When neither a global nor an operation-level seed is defined, TensorFlow chooses an arbitrary seed from its pool. The result varies each time the code re-runs, so this scenario does not guarantee reproducibility.

Scenario 2: Only Global Seed Defined

🔒 Subscribe to keep reading.

Scenario 3: Only Operation-level Seed Defined

🔒 Subscribe to keep reading.

Scenario 4: Both Seeds Defined (Fully Reproducible)

🔒 Subscribe to keep reading.

Summary: Choosing the Right Seed for Reproducibility

🔒 Subscribe to keep reading.