Mixed Precision

AI Python Life chatGPT Apps TensorFlow Git ML Blogging DL MAC Preprocessing NLP Conda PhD Regression SEO CNN Classification Data Fine Tuning Kaggle Mixed Precision RS Robots Study Transfer Learning

Floating-point format and Mixed Precision in TensorFlow

19 May, 2022

When creating large Machine Learning models, we want to minimise the training time. In TensorFlow, it is possible to do mixed precision model training, which helps in significant performance improvement because it uses lower-precision operations with 16 bits (such as float16) together with single-precision operations (f.i. using float32 data type). Google TPUs and NVIDIA GPUs devices can perform operations with 16-bit datatype much faster