fix: binary_numeric is now correct (+ tests!) #1721
Conversation
Lessons learned: do not merge without at least basic tests. We check for a constant array but do not create a new constant array with the appropriate length for the child on which we want to delegate the binary numeric operation. That is now fixed. I also added a very basic test suite for applying the (now six) numeric operations on pairs of arrays where one of the two are constant. In order to properly support constant arrays on the left- hand-side, I added FlippedSub and FlippedDiv which allow us to commute/flip the operator so that we need not teach ConstantArray to check if its right-hand-side supports an operation on constants.
| Sub, | ||
| /// Same as [BinaryNumericOperator::Sub] but with the parameters flipped: `right - left`. | ||
| FlippedSub, |
There was a problem hiding this comment.
I'm not sure about these operators... it sort of forces everyone to handle bi-directional operators?
There was a problem hiding this comment.
If an array has no clever binary numeric operation, it just uses the Arrow implementation, but, yes, if an array implements subtract and it wants to handle constant right-hand-sides, it needs to deal with FlippedSub.
I suppose, if we had unary negation, we could use Add to flip. It seems bad to introduce a second pass over the array though.
An array could implement just Sub and fallback to Arrow for FlippedSub.
I kinda think this is just the essential complexity of a non-commutative operation? Or maybe I misunderstand what you mean about bi-directional operators?
There was a problem hiding this comment.
This idea sounds cool but I think the clever binary numeric functions are easy to implemented for both LHS and RHS. Maybe other operators would have some benefits by doing this? I'm not sure.
| array: &DictArray, | ||
| rhs: &ArrayData, | ||
| op: BinaryNumericOperator, | ||
| ) -> VortexResult<Option<ArrayData>> { |
There was a problem hiding this comment.
We need a pre-check to verify that the values of the dictionary array are of a primitive type. If not, we can fast-fail to avoid creating a new ConstantArray.
Lessons learned: do not merge without at least basic tests.
Four major changes:
binary_numeric.rsWe check for a constant array but do not create a new constant array with the appropriate length for the child on which we want to delegate the binary numeric operation. That is now fixed.
I also added a very basic test suite for applying the (now six) numeric operations on pairs of arrays where one of the two are constant.
In order to properly support constant arrays on the left-hand-side, I added FlippedSub and FlippedDiv which allow us to commute/flip the operator so that we need not teach ConstantArray to check if its right-hand-side supports an operation on constants.