fix: properly gate things by features & test for that #5128

Job	Run time
bench test	7m 24s
Check generated proto/fbs files are up to date	58s
Build (default)	2m 50s
Build (all-features)	2m 30s
License Check and Audit Check	1m 38s
miri	10m 41s
Python (lint)	3m 26s
Python (test)	4m 3s
Rust (docs)	59s
Rust (tests)	6m 11s
	40m 40s

Provide feedback