Mixed Precision

AI Python Life chatGPT Apps Git TensorFlow Blogging ML DL MAC Preprocessing NLP SEO Conda PhD Regression genAI AI Law CNN Classification Data Fine Tuning Kaggle LLM Mixed Precision RS Robots Study Transfer Learning

Floating-point format and Mixed Precision in TensorFlow

Elena Daehnhardt · 19 May, 2022

When creating large Machine Learning models, we want to minimise the training time. In TensorFlow, it is possible to do mixed precision model training, which helps in significant performance improvement because it uses lower-precision operations with 16 bits (such as float16) together with single-precision operations (f.i. using float32 data type). Google TPUs and NVIDIA GPUs devices can perform operations with 16-bit datatype much faster