What Is Mixed Precision Training in TensorFlow?

Q: How do I enable mixed precision training in TensorFlow?

Set a global policy before building your model: tf.keras.mixed_precision.set_global_policy('mixed_float16') on NVIDIA GPUs, or 'mixed_bfloat16' on TPUs. After setting the policy, the compute dtype becomes float16 while the variable dtype stays float32 . Build the model as usual but force the final layer to float32 with layers.Activation('softmax', dtype=tf.float32) .

Q: What is the difference between float16 and float32 in TensorFlow?

float16 (half precision) uses 16 bits with a range of about [-65500, 65500], while float32 (single precision) uses 32 bits with a range up to ~3.4e+38. float16 halves memory use and runs faster on accelerators, but its narrow range and lower precision can cause overflow or underflow, which is why mixed precision keeps numerically sensitive operations such as the loss and output layer in float32 .

Q: Why does mixed_float16 warn about compute capability at least 7.0?

mixed_float16 only accelerates training on NVIDIA GPUs with compute capability 7.0 or higher (Volta, Turing, Ampere and newer), which have Tensor Cores. On older GPUs, Apple Metal, or CPUs, TensorFlow logs the warning and runs correctly but without a speed-up. On Google TPUs, use mixed_bfloat16 instead.

Mixed precision is a training technique that combines 16-bit floating-point operations (float16 or bfloat16) with 32-bit operations (float32) to speed up deep learning training while preserving numerical stability. According to TensorFlow’s own mixed precision guide, the technique improves performance by more than 3x on modern NVIDIA GPUs with compute capability 7.0 or higher, by around 60% on Cloud TPUs, and by more than 2x on recent Intel CPUs — by running most operations in float16 (or bfloat16 on TPUs) while keeping the loss and output layer in float32.

When you are training large Machine Learning models, you want to minimise the training time. In TensorFlow, you can enable mixed precision model training, which gives a real performance improvement because it runs most operations at 16 bits (float16) instead of full single precision (float32). Google TPUs and NVIDIA GPUs have dedicated hardware — Tensor Cores, on the NVIDIA side — built to run 16-bit operations faster than 32-bit ones, see Mixed precision. For the official API reference covering tf.keras.mixed_precision.set_global_policy and the full list of supported policies, see TensorFlow’s mixed_precision documentation. The performance gain comes from two things: float16 values take half the memory of float32, so more data fits through memory bandwidth per second, and Tensor Cores execute float16 matrix multiplications and convolutions directly in hardware. In this post, I will outline the relevant data types and show you the main steps for setting up mixed-precision training in TensorFlow.

Computer Data Types: Bits, Bytes, and Numeric Ranges

Computers store and process data as bits, each one set to 0 (no signal) or 1 (signal present) — that on/off state is what flows through the circuitry, and it is called binary data representation. You are used to thinking in decimal (powers of 10), so we group bits into bytes of 8, and from there into larger formats built for different kinds of numbers. The more bits a format uses, the more values it can store. There are other numerical bases too — octal, hexadecimal — and converting between them is straightforward, see Computer number format.

Decimal Values	Binary Values
0	0000
1	0001
2	0010
3	0011
4	0100
5	0101
6	0110
7	0111
8	1000
9	1001
10	1010

Table 1. Values in Decimal and Binary Bases

In one byte, we can store 2^8 possible values. In 4 bytes, we can store 2^32 values. You can also represent fractions using fixed-point numbers, where you set aside a fixed number of bits for the integer part and a fixed number for the fractional part. That works, but you cannot represent exact fractions such as 1/3 this way — the recurring tail of 3s gets truncated once you run out of fractional bits. The range of stored values is limited by the number of bits used, and that limit is exactly why precision loss matters whenever a computation needs a specific numeric range.

Floating-point Format: float16 vs float32 vs float64

The floating-point format is a numeric representation that stores a sign, an exponent, and a significand (mantissa) so the decimal point can “float” to any position, letting a fixed number of bits encode both very large and very small numbers. This works like scientific notation: a significand multiplied by an exponent, which lets you represent both very large and very tiny values — exactly what you need when computing neural networks. Table 2 shows the layout for the three precision levels defined by the IEEE 754 standard.

Precision Type	Total bits	Sign bits	Exponent bits	Significand bits	Range
Half	16	1	5	10	[-65504.0..65504.0]
Single	32	1	8	23	[-3.4028235e+38..3.4028235e+38]
Double	64	1	11	52	[-1.7976931348623157e+308..1.7976931348623157e+308]

Table 2. Floating Point, Precision Types

I got these values from tf.float16.max, tf.float32.min/.max, and Python’s own sys.float_info — note that TensorFlow’s mixed precision guide confirms the float16 cutoff directly: values above 65504 overflow to infinity, and values below 6.0 × 10⁻⁸ underflow to zero. You can check your own system’s floating-point information with sys.float_info:

import sys
sys.float_info

sys.float_info(max=1.7976931348623157e+308, max_exp=1024, max_10_exp=308, min=2.2250738585072014e-308, min_exp=-1021, min_10_exp=-307, dig=15, mant_dig=53, epsilon=2.220446049250313e-16, radix=2, rounds=1)

The figure below shows a float number stored in two bytes of computer memory. Can you decode its value? Write me about its decimal representation or share your thoughts at the contact page.

Half-precision Float-16 Representation Example

You can read about floating-point numbers and operations considering converting binary numbers to the floating-point representation at the Imperial College website.

Mixed Precision in TensorFlow: Setup and the mixed_float16 Policy

Higher precision floats occupy 32 or 64 bits of memory and need more computational resources to process. So it makes sense to use lower-precision numbers such as float16 where you can — modern accelerators have dedicated hardware for 16-bit computations, and TensorFlow’s mixed precision guide explains how to combine 16-bit and 32-bit float operations for more efficient model training. I will walk you through a working example, following that guide together with the TensorFlow Developer Certificate in 2022: Zero to Mastery course material and the “Mixed precision” Colab notebook from TensorFlow.

Following the TensorFlow Developer Certificate in 2022: Zero to Mastery course, you implement mixed-precision TensorFlow models in two steps:

Activate mixed precision with mixed_precision.set_global_policy("mixed_float16").
Set the output layer to use float32.

Next, I will show you in detail how to confirm mixed precision is active, and how to check your hardware.

Setting the mixed_float16 Policy with tf.keras.mixed_precision.set_global_policy()

I run the following code to check GPU and TPU availability and set the mixed-precision policy.

import tensorflow as tf

try:
    tpu = tf.distribute.cluster_resolver.TPUClusterResolver() # TPU detection
    policyConfig = 'mixed_bfloat16'
except ValueError:
    policyConfig = 'mixed_float16'

policy = tf.keras.mixed_precision.Policy(policyConfig)
tf.keras.mixed_precision.set_global_policy(policy)

WARNING:TensorFlow:Mixed precision compatibility check (mixed_float16): WARNING
Your GPU may run slowly with dtype policy mixed_float16 because it does not have compute capability of at least 7.0. Your GPU:
  METAL, no compute capability (probably not an Nvidia GPU)
See https://developer.nvidia.com/cuda-gpus for a list of GPUs and their compute capabilities.
If you will use compatible GPU(s) not attached to this host, e.g. by running a multi-worker model, you can ignore this warning. This message will only be logged once

The code gives me a warning because my GPU (MacBook Pro M1 Max) is not designed to exploit mixed-precision computations. I do not expect any speed-up on this machine, but I can still run the code to test the Keras API.

Fixing “Your GPU may run slowly with dtype policy mixed_float16 because it does not have compute capability of at least 7.0”

Verbatim warning: Your GPU may run slowly with dtype policy mixed_float16 because it does not have compute capability of at least 7.0.

Cause: mixed_float16 only accelerates training on NVIDIA GPUs with compute capability 7.0+ (Volta, Turing, Ampere and newer Tensor Core hardware). Apple Metal GPUs, pre-Volta NVIDIA cards, and CPUs trigger this warning.

Fix: The warning is informational — training still runs correctly. To get an actual speed-up, use compatible hardware. On Google TPUs switch the policy to mixed_bfloat16; on unsupported hardware revert to single precision:

tf.keras.mixed_precision.set_global_policy("float32")

Confirm the device’s compute capability against the NVIDIA CUDA GPUs list.

On Colab, you get a GPU allocated at random unless you pay for the Pro tier, so you cannot always guarantee compute capability 7.0+. Once NVIDIA drivers are installed, TensorFlow’s guide recommends checking the GPU type with:

!nvidia-smi -L

After setting up the policy, we check which data types are used in operations and while storing variables.

print('Compute dtype: %s' % policy.compute_dtype)
print('Variable dtype: %s' % policy.variable_dtype)

Compute dtype: float16
Variable dtype: float32

Building a Mixed Precision Keras Model with a float32 Output Layer

To build a model large enough to actually benefit from mixed precision, I use tf.keras.applications.EfficientNetB0. Small toy models rarely show a speed-up — the TensorFlow runtime overhead dominates — so the model needs enough compute to be worth accelerating. Note the dtype=tf.float32 on the output layer: that is what keeps softmax numerically stable under the mixed_float16 policy.

# Download the model
baseline_model = tf.keras.applications.EfficientNetB0(include_top=False)

# Freeze underlying layers
baseline_model.trainable = False

# Create a functional model
INPUT_SHAPE = (224, 224, 3)
inputs = layers.Input(shape=INPUT_SHAPE, name="input_layer")

# For models (unlike EfficientNetBx) not including rescaling 
# x = preprocessing.Rescaling(1./255)(x)

x = baseline_model(inputs, training=False)
x = layers.GlobalAveragePooling2D()(x)
x = layers.Dense(400)(x)

# Mixed precision requires output layer in float32 type, which is more numerically stabil than float16
outputs = layers.Activation("softmax", dtype=tf.float32, name="softmax_float32")(x)

# Create the model
model = tf.keras.Model(inputs, outputs)

# Compile the model
model.compile(loss="categorical_crossentropy", # sparse_categorical_crossentropy when our labels are integers 
             optimizer=tf.keras.optimizers.Adam(),
             metrics="accuracy")

To switch mixed precision back off, reset the global policy to plain float32:

tf.keras.mixed_precision.set_global_policy("float32")

Loss scaling is the other piece worth knowing about: with mixed_float16, gradients can underflow to zero during the backward pass because float16 has a much narrower dynamic range than float32. Model.fit handles this for you automatically. In a custom training loop, you have to wrap your optimizer explicitly with tf.keras.mixed_precision.LossScaleOptimizer — see the loss scaling section of the TensorFlow guide for the full picture.

Final Thoughts: When to Use Mixed Precision Training

Mixed precision training is a hardware-aware optimisation technique that trades a small amount of numerical precision for a real speed-up — more than 3x on Tensor Core GPUs, around 60% on TPUs — plus lower memory use. I have outlined computer number formats with a focus on floating-point precision, and shown you a working example of building a mixed-precision model with Keras and TensorFlow. Use it when your model is large enough to be compute-bound and your accelerator supports float16 (compute capability 7.0+) or bfloat16 (TPU); keep the loss and output layer in float32 to avoid overflow. I will focus on something more exciting than computer number formats in my next post!

TensorFlow Mixed Precision FAQ

How do I enable mixed precision training in TensorFlow?

Set a global policy before building your model: tf.keras.mixed_precision.set_global_policy('mixed_float16') on NVIDIA GPUs, or 'mixed_bfloat16' on TPUs. After setting the policy, the compute dtype becomes float16 while the variable dtype stays float32. Build the model as usual but force the final layer to float32 with layers.Activation('softmax', dtype=tf.float32).

What is the difference between float16 and float32 in TensorFlow?

float16 (half precision) uses 16 bits with a range of about [-65500, 65500], while float32 (single precision) uses 32 bits with a range up to ~3.4e+38. float16 halves memory use and runs faster on accelerators, but its narrow range and lower precision can cause overflow or underflow, which is why mixed precision keeps numerically sensitive operations such as the loss and output layer in float32.

Why must the output layer be float32 in mixed precision?

The softmax output and loss computation are numerically sensitive: running them in float16 can produce overflow or NaN values because of the narrow float16 range. Setting dtype=tf.float32 on the final activation layer keeps the model fast in the hidden layers while preserving numerical stability where it matters. See the TensorFlow mixed precision guide.

Why does mixed_float16 warn about compute capability at least 7.0?

mixed_float16 only accelerates training on NVIDIA GPUs with compute capability 7.0 or higher (Volta, Turing, Ampere and newer), which have Tensor Cores. On older GPUs, Apple Metal, or CPUs, TensorFlow logs the warning and runs correctly but without a speed-up. On Google TPUs, use mixed_bfloat16 instead.