Replace Never & Eventually functions with synchronous versions

[brackendawson]

Summary

Eventually, EventuallyWithT, and Never have serious defects which require changes to their behaviour to rectify:

• They can leak goroutines.
• The can return a verdict without ever testing the condition!!!
• They always wait 1 tick before testing the condition.

Making them syncronous fixes every known problem with these functions.

Changes

• Create a synchronous version of EventuallyWithT called EventuallySync. Making the call to condition syncrhonous means we no longer have to decide what to do (or what not to do) when an asynchronous call to the user's condition function doesn't return before the end of the test. This solves the goroutine leak (assert: Eventually should not leak a goroutine #1611) but also forces the definition of a new function as there could be many badly written tests which would now deadlock if this was implimented in the existing functions. This also means evaluation of the test result happens between calls to the condition function, making the assertion easier to use for the user and solving If condition passed to Never does not return before waitFor, then the Never assertion passes. #1654.
• Create a synchronous version of Never for the same reasons. Implementing it as Consistently allows it to use CollectT too and delivers Feature: assert.Consistently #1087.
• Always run condition immediately. This solves the timing race (Eventually can fail having never run condition #1652) where the select statement could be reached for the first time with both tick and waitFor channels ready, where the function would randomly return a verdict without actually testing the condition. This also delivers a popular request: Succeed quickly in Eventually, EventuallyWithT #1424
• Mark Eventually and EventuallyWithT as deprecated with warnings about their faults and suggest the use of EventuallySync instead.
• Mark Never as deprecated with warnings about its faults and suggest the use of Consistently instead.
• Make the flaking TestEventuallyTimeout less likelt to fail, or fail more quickly.

Motivation

• Eventually/Never/etc.. can leak goroutines.
• Eventually/Never/etc.. can show either test result without testing anything.
• Eventually/Never/etc.. make our tests flake.
• People want Eventually/Never/etc.. to be faster.
• People want a Consistently assertion.

Related issues

• Closes Feature: assert.Consistently #1087
• Closes Succeed quickly in Eventually, EventuallyWithT #1424
• Closes assert: Eventually should not leak a goroutine #1611
• Closes Eventually can fail having never run condition #1652
• Closes If condition passed to Never does not return before waitFor, then the Never assertion passes. #1654

[cszczepaniak]

What does this goroutine gain us? We're waiting on it synchronously immediately after, so it's equivalent to just calling the condition inline here (without a goroutine)

[brackendawson]

CollectT.FailNow will call runtime.GoExit. If we didn't use a new goroutine for each callback then the user calling FailNow would stop the whole test. This is the only reason we can't do this without any additional goroutines at all.

[brackendawson]

It looks like @dolmen doesn't like CollectT, and I can't say I disagree with him.

So we actually could make this work with no additional goroutines. The choice is to either use func() bool or use an interface with an un-exported implementation. I find that every time I use Eventually I want to use assertions inside the condition function, so the latter is my preference. But we can panic an un-exported value rather than calling runtime.Goexit. All panics from the condition function should be handled regardless.

[cszczepaniak]

I'm not sure I totally follow. It sounds like an unexported implementation makes sense, but could you sketch out what that code would look like?

[cszczepaniak]

I wonder if we should still support failing the test immediately when deadline elapses, even if we're in the middle of calling the condition. We'd effectively just keep the goroutine above that I commented on and select over the done channel + the timeout channel. Maybe this is what was meant by leaking a goroutine though.

If we don't do that, I question the value of the API accepting a duration as a timeout because the duration isn't super closely tied with the maximum time this function might take. Maybe it would be better to just accept a maximum number of tries instead.

[brackendawson]

I wonder if we should still support failing the test immediately when deadline elapses, even if we're in the middle of calling the condition. We'd effectively just keep the goroutine above that I commented on and select over the done channel + the timeout channel. Maybe this is what was meant by leaking a goroutine though.

This makes the callback asynchronous (sometimes) and puts us back in the situation of having to allow it to potentially leak. It was always asking for trouble. I tried a similar fix in #1653 but I don't like it.

If we don't do that, I question the value of the API accepting a duration as a timeout because the duration isn't super closely tied with the maximum time this function might take. Maybe it would be better to just accept a maximum number of tries instead.

This is an excellent point. I hadn't considered that synchronous behaviour would suit a different function signature better. times int is far more natural than waitFor time.Duration. I've updated the PR.

Thanks for the review!

[brackendawson]

The issues described recently in #1396 lead me to believe the collect object and the condition function need more work to support the test being failed immediately from the condition function. Either of these options need to be used:

• The condition function be run on the same goroutine as the test. This would allow the use of testing.T.FailNow to be supported but means the collect object's FailNow implementation must revert back to a panic with a private value, which it was recently changed away from.
• The condition function should return a bool: func() bool, if false it means fail immediately.