Zstandard codec for Avro

[damccorm]

Currently AvroSource supports only snappy, deflate, bzip2 and xz codecs.

Avro library allows to write files with Zstandard codec too. Also BEAM-6365 added support of the codec but only for Text files.

Would be helpful to have Zstandard for Avro as well.

Imported from Jira BEAM-13319. Original Jira may contain additional context.
Reported by: splix.

[yyy1000]

Interested in this!
I'm new to this and may have misunderstanding. :)
When I checked the original Jira, I found this PR(#16271) is still Work In Progress.
Could this task be done before that?

[damccorm]

@RustedBones @kennknowles @aromanenko-dev @mosche could one of you comment here? (FYI, I imagine responses may be delayed as we're in the midst of holiday season)

[aromanenko-dev]

I think it's blocked by current old Avro version that Beam support. We are working on extracting Avro into a separate extension and add a support of multiple Avro versions there (more details here #24292).

[yyy1000]

OK, I will find other tasks to do first. :)

[RustedBones]

I think this can be closed since there is new avro extension module is released