Mixed Precision

AI Python Life chatGPT Apps Git TensorFlow ML Blogging DL MAC Preprocessing NLP SEO Conda PhD Regression AI Law CNN Classification Data Fine Tuning Kaggle Mixed Precision RS Robots Study Transfer Learning genAI

Floating-point format and Mixed Precision in TensorFlow

Elena Daehnhardt · 19 May, 2022

When creating large Machine Learning models, we want to minimise the training time. In TensorFlow, it is possible to do mixed precision model training, which helps in significant performance improvement because it uses lower-precision operations with 16 bits (such as float16) together with single-precision operations (f.i. using float32 data type). Google TPUs and NVIDIA GPUs devices can perform operations with 16-bit datatype much faster