v0.0.6
Pre-release
Pre-release
This release fixes several problems related to BatchNormalization, targeting numerical inconsistencies between QD and ID stage and substantially improving the stability in FQ training. In particular:
- BN parameters are no longer as big as possible by default, they are now calibrated according to the BN output range; BN additive parameter is no longer requantized.
- BatchNormalization in QD/ID stages can be calibrated using statistics collected from validation (or with a reasonable default)
- BatchNormalization freezing also disables gradients, unless explicitly requested not to do so.