Skip to content

v0.0.6

Pre-release
Pre-release
Compare
Choose a tag to compare
@FrancescoConti FrancescoConti released this 26 May 14:07
· 28 commits to master since this release

This release fixes several problems related to BatchNormalization, targeting numerical inconsistencies between QD and ID stage and substantially improving the stability in FQ training. In particular:

  • BN parameters are no longer as big as possible by default, they are now calibrated according to the BN output range; BN additive parameter is no longer requantized.
  • BatchNormalization in QD/ID stages can be calibrated using statistics collected from validation (or with a reasonable default)
  • BatchNormalization freezing also disables gradients, unless explicitly requested not to do so.