This repository has been archived by the owner on Nov 11, 2022. It is now read-only.
Version 0.4.20150727
Pre-release
Pre-release
davorbonaci
released this
28 Jul 17:18
·
1497 commits
to master
since this release
- Removed the requirement to explicitly set
--project
if Google Cloud SDK has the default project configuration set. - Added support for creating BigQuery sources from a query.
- Added support for custom unbounded sources in the
DirectPipelineRunner
andDataflowPipelineRunner
. SeeUnboundedSource
for details. - Removed unnecessary
ExecutionContext
argument inBoundedSource.createReader
and related methods. - Changed
BoundedReader.splitAtFraction
to require thread-safety (i.e. safe to call asynchronously withadvance
orstart
). AddedRangeTracker
to help implement thread-safe readers. Users are heavily encouraged to use the class rather than implementing an ad-hoc solution. - Modified
Combine
transforms by lifting them into (and above) theGroupByKey
resulting in better performance. - Modified triggers such that after a
GroupByKey
, the system will switch to a "Continuation Trigger", which attempts to preserve the original intention regarding handling of speculative and late triggerings instead of returning to the default trigger. - Added
WindowFn.getOutputTimestamp
and changedGroupByKey
behavior to allow incomplete overlapping windows to not hold up progress of earlier, completed windows. - Changed triggering behavior so that empty panes are produced if they are the first pane after the watermark (
ON_TIME
) or the final pane. - Removed the
Window.Trigger
intermediate builder class. - Added validation that allowed lateness is specified on the
Window
PTransform
when a trigger is specified. - Re-enabled verification of
GroupByKey
usage. Specifically, the key must have a deterministic coder and usingGroupByKey
with an unboundedPCollection
requires windowing or triggers. - Changed
PTransform
names so that they may no longer contain the=
or;
characters.