New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Reshuffle implementation in Java SDK #28853

Merged

kennknowles merged 1 commit into apache:master from kennknowles:reshuffle

Jan 3, 2024

Merged

Fix Reshuffle implementation in Java SDK #28853

Fix Reshuffle implementation in Java SDK

GitHub Actions / Test Results succeeded Dec 19, 2023 in 0s

All 107 tests pass, 1 skipped in 2m 7s

108 tests - 1 847 107 ✔️ - 1 677 2m 7s ⏱️ - 1h 40m 4s
  15 suites -   192     1 💤 -   170
  15 files -   192     0 ❌ ±      0

Results for commit 7827d1d. ± Comparison against earlier commit ff6d4baa.

Annotations

Check notice on line 0 in .github

github-actions / Test Results

1 skipped test found

There is 1 skipped test, see "Raw output" for the name of the skipped test.

Raw output


            org.apache.beam.sdk.io.hdfs.HadoopFileSystemTest ‑ testReadPipeline

Check notice on line 0 in .github

github-actions / Test Results

108 tests found

There are 108 tests, see "Raw output" for the full list of tests.

Raw output


            org.apache.beam.sdk.io.hadoop.SerializableConfigurationTest ‑ testConstruction
org.apache.beam.sdk.io.hadoop.SerializableConfigurationTest ‑ testCreateNewConfiguration
org.apache.beam.sdk.io.hadoop.SerializableConfigurationTest ‑ testCreateNewJob
org.apache.beam.sdk.io.hadoop.SerializableConfigurationTest ‑ testFromMap
org.apache.beam.sdk.io.hadoop.SerializableConfigurationTest ‑ testSerializationDeserialization
org.apache.beam.sdk.io.hadoop.WritableCoderTest ‑ testAutomaticRegistrationOfCoderProvider
org.apache.beam.sdk.io.hadoop.WritableCoderTest ‑ testIntWritableEncoding
org.apache.beam.sdk.io.hadoop.WritableCoderTest ‑ testNullWritableEncoding
org.apache.beam.sdk.io.hadoop.format.HDFSSynchronizationTest ‑ testCatchingRemoteException
org.apache.beam.sdk.io.hadoop.format.HDFSSynchronizationTest ‑ testMissingJobId
org.apache.beam.sdk.io.hadoop.format.HDFSSynchronizationTest ‑ testMultipleTaskDeletion
org.apache.beam.sdk.io.hadoop.format.HDFSSynchronizationTest ‑ testTaskAttemptIdAcquire
org.apache.beam.sdk.io.hadoop.format.HDFSSynchronizationTest ‑ testTaskIdLockAcquire
org.apache.beam.sdk.io.hadoop.format.HDFSSynchronizationTest ‑ tryAcquireJobLockTest
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOCassandraTest ‑ testHIFReadForCassandra
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOCassandraTest ‑ testHIFReadForCassandraQuery
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOElasticTest ‑ testHifIOWithElastic
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOElasticTest ‑ testHifIOWithElasticQuery
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testComputeSplitsIfGetSplitsReturnsEmptyList
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testComputeSplitsIfGetSplitsReturnsListHavingNullValues
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testComputeSplitsIfGetSplitsReturnsNullValue
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testCreateReaderIfSplitNotCalled
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testGetCurrentSourceFunction
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testGetFractionConsumedForBadProgressValue
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testImmutablityOfOutputOfReadIfRecordReaderObjectsAreImmutable
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testImmutablityOfOutputOfReadIfRecordReaderObjectsAreMutable
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testReadBuildsCorrectly
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testReadBuildsCorrectlyIfWithConfigurationIsCalledMoreThanOneTime
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testReadBuildsCorrectlyInDifferentOrder
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testReadDisplayData
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testReadIfCreateRecordReaderFails
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testReadObjectCreationFailsIfConfigurationIsNull
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testReadObjectCreationFailsIfKeyCoderIsNull
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testReadObjectCreationFailsIfKeyTranslationFunctionIsNull
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testReadObjectCreationFailsIfValueCoderIsNull
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testReadObjectCreationFailsIfValueTranslationFunctionIsNull
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testReadObjectCreationWithConfiguration
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testReadObjectCreationWithConfigurationKeyTranslation
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testReadObjectCreationWithConfigurationKeyTranslationAndCoder
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testReadObjectCreationWithConfigurationKeyTranslationValueTranslation
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testReadObjectCreationWithConfigurationValueTranslation
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testReadObjectCreationWithConfigurationValueTranslationAndCoder
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testReadObjectCreationWithSkipKeyValueClone
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testReadValidationFailsMissingConfiguration
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testReadValidationFailsMissingInputFormatInConf
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testReadValidationFailsMissingKeyClassInConf
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testReadValidationFailsMissingValueClassInConf
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testReadValidationFailsWithWrongInputTypeKeyTranslationFunction
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testReadValidationFailsWithWrongInputTypeValueTranslationFunction
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testReadWithNullCreateRecordReader
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testReaderAndParentSourceReadsSameData
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testReadersGetFractionConsumed
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testReadersStartWhenZeroRecords
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testReadingData
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testReadingDataWithCoder
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testReadingDataWithNullKeys
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testReadingDataWithNullValues
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testReadingWithConfigurableInputFormat
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testSkipKeyValueClone
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOReadTest ‑ testValidateConfigurationWithDBInputFormat
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOSequenceFileTest ‑ batchTest
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOSequenceFileTest ‑ batchTestWithoutPartitioner
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOSequenceFileTest ‑ streamTest
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOWriteTest ‑ testWriteDisplayData
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOWriteTest ‑ testWriteObjectCreationFailsIfConfigurationIsNull
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOWriteTest ‑ testWriteValidationFailsMissingJobIDInConf
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOWriteTest ‑ testWriteValidationFailsMissingKeyClassInConf
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOWriteTest ‑ testWriteValidationFailsMissingOutputFormatInConf
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOWriteTest ‑ testWriteValidationFailsMissingValueClassInConf
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOWriteTest ‑ testWritingData
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOWriteTest ‑ testWritingDataFailInvalidKeyType
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOWriteTest ‑ testWritingDataFailInvalidValueType
org.apache.beam.sdk.io.hadoop.format.IterableCombinerTest ‑ testCombining
org.apache.beam.sdk.io.hadoop.format.IterableCombinerTest ‑ testSerializing
org.apache.beam.sdk.io.hdfs.HadoopFileSystemModuleTest ‑ testConfigurationSerializationDeserialization
org.apache.beam.sdk.io.hdfs.HadoopFileSystemModuleTest ‑ testObjectMapperIsAbleToFindModule
org.apache.beam.sdk.io.hdfs.HadoopFileSystemOptionsRegistrarTest ‑ testServiceLoader
org.apache.beam.sdk.io.hdfs.HadoopFileSystemOptionsTest ‑ testDefaultJustSetHadoopConfDirConfiguration
org.apache.beam.sdk.io.hdfs.HadoopFileSystemOptionsTest ‑ testDefaultJustSetHadoopConfDirMultiPathConfiguration
org.apache.beam.sdk.io.hdfs.HadoopFileSystemOptionsTest ‑ testDefaultJustSetYarnConfDirConfiguration
org.apache.beam.sdk.io.hdfs.HadoopFileSystemOptionsTest ‑ testDefaultJustSetYarnConfDirMultiPathConfiguration
org.apache.beam.sdk.io.hdfs.HadoopFileSystemOptionsTest ‑ testDefaultSetYarnConfDirAndHadoopConfDirAndSameConfiguration
org.apache.beam.sdk.io.hdfs.HadoopFileSystemOptionsTest ‑ testDefaultSetYarnConfDirAndHadoopConfDirAndSameDir
org.apache.beam.sdk.io.hdfs.HadoopFileSystemOptionsTest ‑ testDefaultSetYarnConfDirAndHadoopConfDirMultiPathAndSameConfiguration
org.apache.beam.sdk.io.hdfs.HadoopFileSystemOptionsTest ‑ testDefaultSetYarnConfDirAndHadoopConfDirNotSameConfiguration
org.apache.beam.sdk.io.hdfs.HadoopFileSystemOptionsTest ‑ testDefaultUnsetEnvHdfsConfiguration
org.apache.beam.sdk.io.hdfs.HadoopFileSystemOptionsTest ‑ testParsingHdfsConfiguration
org.apache.beam.sdk.io.hdfs.HadoopFileSystemRegistrarTest ‑ testServiceLoader
org.apache.beam.sdk.io.hdfs.HadoopFileSystemTest ‑ testCopy
org.apache.beam.sdk.io.hdfs.HadoopFileSystemTest ‑ testCopySourceMissing
org.apache.beam.sdk.io.hdfs.HadoopFileSystemTest ‑ testCreateAndReadFile
org.apache.beam.sdk.io.hdfs.HadoopFileSystemTest ‑ testCreateAndReadFileWithShift
org.apache.beam.sdk.io.hdfs.HadoopFileSystemTest ‑ testCreateAndReadFileWithShiftToEnd
org.apache.beam.sdk.io.hdfs.HadoopFileSystemTest ‑ testDelete
org.apache.beam.sdk.io.hdfs.HadoopFileSystemTest ‑ testDeleteNonExisting
org.apache.beam.sdk.io.hdfs.HadoopFileSystemTest ‑ testMatch
org.apache.beam.sdk.io.hdfs.HadoopFileSystemTest ‑ testMatchDirectory
org.apache.beam.sdk.io.hdfs.HadoopFileSystemTest ‑ testMatchForNonExistentFile
org.apache.beam.sdk.io.hdfs.HadoopFileSystemTest ‑ testMatchForRecursiveGlob
org.apache.beam.sdk.io.hdfs.HadoopFileSystemTest ‑ testMatchNewResource
org.apache.beam.sdk.io.hdfs.HadoopFileSystemTest ‑ testReadPipeline
org.apache.beam.sdk.io.hdfs.HadoopFileSystemTest ‑ testRename
org.apache.beam.sdk.io.hdfs.HadoopFileSystemTest ‑ testRenameExistingDestination
org.apache.beam.sdk.io.hdfs.HadoopFileSystemTest ‑ testRenameMissingSource
org.apache.beam.sdk.io.hdfs.HadoopFileSystemTest ‑ testRenameMissingTargetDir
org.apache.beam.sdk.io.hdfs.HadoopFileSystemTest ‑ testRenameRetryScenario
org.apache.beam.sdk.io.hdfs.HadoopResourceIdTest ‑ testGetFilename
org.apache.beam.sdk.io.hdfs.HadoopResourceIdTest ‑ testResourceIdTester

View more details on GitHub Actions