feat: return Bytes from readers #1330
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Move to draft to fix some issues |
lwwmanning
approved these changes
Nov 15, 2024
a10y
force-pushed
the
aduffy/return-bytes
branch
from
6f015a2 to
d750815
Compare
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Vortex bytes_at
| Benchmark suite | Current: d750815 | Previous: 19e52fd | Ratio |
|---|---|---|---|
bytes_at/array_data |
770.7877306863361 ns (1.1092233706131651) |
797.840546382778 ns (1.1929926618061586) |
0.97 |
bytes_at/array_view |
570.874715183593 ns (0.8123285374183524) |
567.6225728551844 ns (0.8558829264741803) |
1.01 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
DataFusion
| Benchmark suite | Current: d750815 | Previous: 19e52fd | Ratio |
|---|---|---|---|
arrow/planning |
816764.343592285 ns (1579.929687498603) |
816326.8311140983 ns (2487.2104055056698) |
1.00 |
arrow/exec |
1446824.6986553278 ns (6909.533939769375) |
1407995.4828392086 ns (6337.699184313882) |
1.03 |
vortex-pushdown-compressed/planning |
508381.80742534203 ns (1224.1903770061908) |
505338.3355290909 ns (1382.6791990936035) |
1.01 |
vortex-pushdown-compressed/exec |
2698880.1742105265 ns (9997.45197368483) |
2722383.790526314 ns (14182.977203947958) |
0.99 |
vortex-pushdown-uncompressed/planning |
512816.0934426691 ns (1428.2885082619032) |
504049.98535751126 ns (1103.7328608312237) |
1.02 |
vortex-pushdown-uncompressed/exec |
2575834.6785 ns (3322.626087500248) |
2556158.1544999997 ns (3012.828337499639) |
1.01 |
vortex-nopushdown-compressed/planning |
838165.1296490458 ns (2647.0850113038905) |
825749.2415427145 ns (1447.7347883851035) |
1.02 |
vortex-nopushdown-compressed/exec |
3177514.205625 ns (30819.787164062494) |
2828945.9622222222 ns (13089.708305555629) |
1.12 |
vortex-nopushdown-uncompressed/planning |
832957.742379662 ns (1478.391550726432) |
825309.0433382558 ns (1548.707405666355) |
1.01 |
vortex-nopushdown-uncompressed/exec |
4747821.266363635 ns (12560.047795453575) |
6239434.1811111085 ns (123259.91337500047) |
0.76 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Random Access
| Benchmark suite | Current: d750815 | Previous: 19e52fd | Ratio |
|---|---|---|---|
random-access/vortex-tokio-local-disk |
2717112.5615789485 ns (13972.555769737111) |
3104188.0005555535 ns (92559.46118055587) |
0.88 |
random-access/vortex-local-fs |
3440296.218666667 ns (14565.435458333464) |
4350089.167500001 ns (55831.21999999974) |
0.79 |
random-access/parquet-tokio-local-disk |
218952926.63333336 ns (3780064.929583341) |
216800698.2 ns (3975536.3666666597) |
1.01 |
This comment was automatically generated by workflow using github-action-benchmark.
a10y
force-pushed
the
aduffy/return-bytes
branch
from
d750815 to
6a32385
Compare
There was a problem hiding this comment.
TPC-H
| Benchmark suite | Current: d750815 | Previous: 19e52fd | Ratio |
|---|---|---|---|
tpch_q1/vortex-in-memory-pushdown |
470879844.85 ns (948783.2037500143) |
446489664.5 ns (1030269.1331250072) |
1.05 |
tpch_q1/arrow |
550184805.4 ns (1064765.449999988) |
532248593.8 ns (3057050) |
1.03 |
tpch_q1/parquet |
729348179.7 ns (1773499.0262500048) |
727384320.3 ns (1302073.9037500024) |
1.00 |
tpch_q1/vortex-file-compressed |
522542854.8 ns (2291801.188749999) |
488747462.4 ns (2794791.9643749893) |
1.07 |
tpch_q2/vortex-in-memory-pushdown |
118783730.70599206 ns (664198.0400669649) |
116226629.90789683 ns (591953.2799801603) |
1.02 |
tpch_q2/arrow |
117381749.33595237 ns (666603.8458988145) |
114955512.88222222 ns (562592.8558541685) |
1.02 |
tpch_q2/parquet |
145627661.02857143 ns (630402.6107142866) |
142351893.9726984 ns (1072228.9433809519) |
1.02 |
tpch_q2/vortex-file-compressed |
161556116.2148809 ns (1858021.9013645798) |
155380914.04845238 ns (1190326.9654151797) |
1.04 |
tpch_q3/vortex-in-memory-pushdown |
178622433.6159127 ns (3105266.316553086) |
174684202.3431746 ns (1325701.4289682359) |
1.02 |
tpch_q3/arrow |
147521577.46285716 ns (1199612.2228482068) |
142417610.3331746 ns (1156644.650680557) |
1.04 |
tpch_q3/parquet |
330622308.25 ns (841259.0143750012) |
330510398.65 ns (2295330.8793750107) |
1.00 |
tpch_q3/vortex-file-compressed |
243171964.3666667 ns (2548451.969999984) |
248733479.89999995 ns (1385441.549999997) |
0.98 |
tpch_q4/vortex-in-memory-pushdown |
133617970.99634919 ns (1962795.1820892915) |
131607546.16511905 ns (508951.46938542277) |
1.02 |
tpch_q4/arrow |
181815797.4334921 ns (1140896.1662896723) |
179065180.64896822 ns (733209.9264861196) |
1.02 |
tpch_q4/parquet |
193323840.83333334 ns (509768.16666667163) |
194064582.96666664 ns (461802.59999999404) |
1.00 |
tpch_q4/vortex-file-compressed |
195590226.36666667 ns (566195.7554166615) |
199212525.7 ns (620508.0333333164) |
0.98 |
tpch_q5/vortex-in-memory-pushdown |
282109875.2 ns (2254924.334374994) |
280723990.2 ns (2834956.924999982) |
1.00 |
tpch_q5/arrow |
262969138.5 ns (1017502.6806250066) |
258053177.45 ns (1749816.271874994) |
1.02 |
tpch_q5/parquet |
455880777.45 ns (4201138.625) |
424250585.9 ns (1292050.974999994) |
1.07 |
tpch_q5/vortex-file-compressed |
330111484.85 ns (2575474.075000018) |
322130015.1 ns (1197568.5818750262) |
1.02 |
tpch_q6/vortex-in-memory-pushdown |
78574238.49359126 ns (206796.68526488543) |
77486834.90833333 ns (147388.10208333284) |
1.01 |
tpch_q6/arrow |
26133297.584375 ns (297457.52542968653) |
25372803.549295638 ns (124630.0060285218) |
1.03 |
tpch_q6/parquet |
137887992.5138095 ns (425458.62940475345) |
136271402.44182542 ns (551749.5936438441) |
1.01 |
tpch_q6/vortex-file-compressed |
14497671.885379817 ns (554287.7761706365) |
14655806.18123583 ns (914564.1052957773) |
0.99 |
tpch_q7/vortex-in-memory-pushdown |
572286422.4 ns (3454026.25) |
564206784.8 ns (3174975.71875) |
1.01 |
tpch_q7/arrow |
518870204.9 ns (3315451.0225000083) |
519689131.9 ns (5455625.806250006) |
1.00 |
tpch_q7/parquet |
695111749.6 ns (13610204.350000024) |
644851638.5 ns (3676610.850000024) |
1.08 |
tpch_q7/vortex-file-compressed |
632995716.3 ns (4505294.246249974) |
612642966.1 ns (2756119.6587499976) |
1.03 |
tpch_q8/vortex-in-memory-pushdown |
230519464.96666664 ns (1653512.1200000197) |
226631225.53333336 ns (1018590.2933333367) |
1.02 |
tpch_q8/arrow |
210397405.73333332 ns (1446283.9945833534) |
205172424.43333334 ns (1523705.892083332) |
1.03 |
tpch_q8/parquet |
483736728.35 ns (3936611.461250007) |
475621681.85 ns (1548176.5493749976) |
1.02 |
tpch_q8/vortex-file-compressed |
289651621.4 ns (3170666.559374988) |
288223547.9 ns (1710140.8806249797) |
1.00 |
tpch_q9/vortex-in-memory-pushdown |
426595397.7 ns (3317471.7056249976) |
397508220.85 ns (3967083.375) |
1.07 |
tpch_q9/arrow |
408685156.1 ns (3586146.743750006) |
384176031.85 ns (2776685.4237499833) |
1.06 |
tpch_q9/parquet |
692962821 ns (3431214.9087499976) |
685855689.3 ns (2564584.706250012) |
1.01 |
tpch_q9/vortex-file-compressed |
469170499.75 ns (2242380.162499994) |
454157932 ns (3725459.620625019) |
1.03 |
tpch_q10/vortex-in-memory-pushdown |
286875582.6 ns (1911205.3756249845) |
278281824.75 ns (1338275.7125000358) |
1.03 |
tpch_q10/arrow |
255255465.7 ns (2101930.635833338) |
238857759 ns (1438073.2966666818) |
1.07 |
tpch_q10/parquet |
489187208.3 ns (3502595.2787499726) |
477261444.95 ns (2218421.912500024) |
1.02 |
tpch_q10/vortex-file-compressed |
378877902.45 ns (4569301.150000006) |
368885647.95 ns (1984393.6012499928) |
1.03 |
tpch_q11/vortex-in-memory-pushdown |
120976439.42464285 ns (807531.796059534) |
113782231.9038889 ns (861324.3571944311) |
1.06 |
tpch_q11/arrow |
115463519.83436508 ns (1983736.2601934448) |
111279800.04749998 ns (468120.2023333311) |
1.04 |
tpch_q11/parquet |
118853427.17190476 ns (1099906.6084523797) |
118662649.79170635 ns (335505.67077282816) |
1.00 |
tpch_q11/vortex-file-compressed |
213824189.3333333 ns (3990614.6820833236) |
193448665.4 ns (1315329.6116666645) |
1.11 |
tpch_q12/vortex-in-memory-pushdown |
258548121.1 ns (421989.6918749958) |
255809267.85 ns (471641.4818750024) |
1.01 |
tpch_q12/arrow |
185568870.1 ns (392255.87458333373) |
185031277.2 ns (475254.59583334625) |
1.00 |
tpch_q12/parquet |
311641867.7 ns (848924.6100000143) |
311220993 ns (966134.3093749881) |
1.00 |
tpch_q12/vortex-file-compressed |
347911798.55 ns (1601380.351875037) |
360587631 ns (1319453.8568750024) |
0.96 |
tpch_q13/vortex-in-memory-pushdown |
167762719.28630954 ns (988190.4759523869) |
165390728.36380953 ns (1060754.9073958397) |
1.01 |
tpch_q13/arrow |
187587155.41698414 ns (4031883.7219920605) |
160707103.46674603 ns (548370.1615128964) |
1.17 |
tpch_q13/parquet |
330223084.8 ns (1533720.5849999785) |
302869889.65 ns (2588147.5443750024) |
1.09 |
tpch_q13/vortex-file-compressed |
197887276.5 ns (1989060.9700000137) |
181906974.84103173 ns (1120548.8332539797) |
1.09 |
tpch_q14/vortex-in-memory-pushdown |
80307539.00807539 ns (141774.75028770417) |
76281343.08488095 ns (242368.9009866044) |
1.05 |
tpch_q14/arrow |
35702558.80055555 ns (367951.774166666) |
32847158.011428565 ns (417604.899880955) |
1.09 |
tpch_q14/parquet |
222682773.7 ns (756115.4445833266) |
221079709.9 ns (787499.673333332) |
1.01 |
tpch_q14/vortex-file-compressed |
101642727.07761905 ns (691843.495910719) |
108502082.98551588 ns (1589777.627494529) |
0.94 |
tpch_q15/vortex-in-memory-pushdown |
114399255.67051587 ns (607766.5454285741) |
110729814.35301587 ns (432550.0140476078) |
1.03 |
tpch_q15/arrow |
59317018.923888884 ns (390765.3163888939) |
56803083.15194444 ns (355592.62249999493) |
1.04 |
tpch_q15/parquet |
300695289.35 ns (1080246.300000012) |
295687623.3 ns (999425.1699999869) |
1.02 |
tpch_q15/vortex-file-compressed |
198369781.73333332 ns (2889490.1333333254) |
196361252.46666664 ns (1531155.7570833266) |
1.01 |
tpch_q16/vortex-in-memory-pushdown |
104384279.4152381 ns (415860.8294464201) |
97707243.08309524 ns (556197.1003363132) |
1.07 |
tpch_q16/arrow |
89629870.09646825 ns (610858.7931547537) |
85780562.816627 ns (480190.02240525186) |
1.04 |
tpch_q16/parquet |
105553443.14626983 ns (538025.3069503978) |
98715262.3197619 ns (193427.3925357163) |
1.07 |
tpch_q16/vortex-file-compressed |
112103358.69777778 ns (1086246.3722222224) |
109448269.30317461 ns (536601.1759345233) |
1.02 |
tpch_q17/vortex-in-memory-pushdown |
591456387.1 ns (3524059.0087500215) |
563627234.3 ns (6673801.199999988) |
1.05 |
tpch_q17/arrow |
500195155.4 ns (6158373.921249986) |
473721012 ns (6094922.114374995) |
1.06 |
tpch_q17/parquet |
634166254.2 ns (4142409.600000024) |
624235705.5 ns (4279606.5437499285) |
1.02 |
tpch_q17/vortex-file-compressed |
608473938.6 ns (7865160.699999988) |
585663340.8 ns (4958622.699999988) |
1.04 |
tpch_q18/vortex-in-memory-pushdown |
1142090901.1 ns (18714462.37624991) |
1019491552.8 ns (3360189.4874999523) |
1.12 |
tpch_q18/arrow |
1077296836.9 ns (20849836.149999976) |
1005804475.5 ns (4707561.972499967) |
1.07 |
tpch_q18/parquet |
1263891040.9 ns (21038549.642500043) |
1196887902.7 ns (6570226.091249943) |
1.06 |
tpch_q18/vortex-file-compressed |
1109801775.8 ns (9188435.433750033) |
1081303080.6 ns (6378922.8000000715) |
1.03 |
tpch_q19/vortex-in-memory-pushdown |
264905785.75 ns (846763.8387499899) |
261476361 ns (663315.1918749958) |
1.01 |
tpch_q19/arrow |
166755438.0627778 ns (238197.89938887954) |
165322643.3392857 ns (573600.5132976323) |
1.01 |
tpch_q19/parquet |
417868951.6 ns (842771.9356250167) |
413282383.9 ns (996637.1037499905) |
1.01 |
tpch_q19/vortex-file-compressed |
373926493.7 ns (4729473.639374971) |
327626335.85 ns (1931488.8056250215) |
1.14 |
tpch_q20/vortex-in-memory-pushdown |
259168765.7 ns (3139336.961250007) |
224188110.6 ns (2455543.4333333373) |
1.16 |
tpch_q20/arrow |
222952473.7666667 ns (2109791.0258333534) |
194556527.76666668 ns (2820099.2787499875) |
1.15 |
tpch_q20/parquet |
351595668.1 ns (6114706.289375007) |
316227125.45 ns (1598766.7568750083) |
1.11 |
tpch_q20/vortex-file-compressed |
361527870.95 ns (2555228.214999974) |
294323604.65 ns (2085045.7181250155) |
1.23 |
tpch_q21/vortex-in-memory-pushdown |
1045219208.2 ns (4051658.539999962) |
884471924.5 ns (6486686.608749986) |
1.18 |
tpch_q21/arrow |
998830103.6 ns (12792420.911249995) |
832794564.5 ns (3990756.8774999976) |
1.20 |
tpch_q21/parquet |
1113260192.2 ns (8543022.733749986) |
912994350.5 ns (5089504.934999943) |
1.22 |
tpch_q21/vortex-file-compressed |
1255871687.7 ns (5236981.372499943) |
1049407669.9 ns (4296564.668750048) |
1.20 |
tpch_q22/vortex-in-memory-pushdown |
80705238.5839484 ns (216098.3628372997) |
74650472.0747222 ns (300617.3986840248) |
1.08 |
tpch_q22/arrow |
77664452.54021825 ns (1155584.0873908699) |
73501145.42275794 ns (291074.101259917) |
1.06 |
tpch_q22/parquet |
105160457.55869047 ns (413767.34017857164) |
102608330.04694445 ns (354271.9569756985) |
1.02 |
tpch_q22/vortex-file-compressed |
88742017.62821427 ns (654231.129583329) |
84485975.24474205 ns (267546.6926946938) |
1.05 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Vortex Compression
| Benchmark suite | Current: d750815 | Previous: 19e52fd | Ratio |
|---|---|---|---|
compress time/taxi |
1258153358.4 ns (7485866.518749952) |
1226193215.8 ns (8267075.4499999285) |
1.03 |
compress time/taxi throughput |
470808924 bytes |
470808924 bytes |
1 |
parquet_rs-zstd compress time/taxi |
1844514799.9 ns (25302574.721249938) |
1790388627.1 ns (4328287.3000000715) |
1.03 |
parquet_rs-zstd compress time/taxi throughput |
470808924 bytes |
470808924 bytes |
1 |
decompress time/taxi |
427457180.7 ns (3328341.098749995) |
412281588.45 ns (2101830.385625005) |
1.04 |
decompress time/taxi throughput |
470808924 bytes |
470808924 bytes |
1 |
parquet_rs-zstd decompress time/taxi |
320478127.55 ns (1742820.885624975) |
322375254.35 ns (892937.9575000107) |
0.99 |
parquet_rs-zstd decompress time/taxi throughput |
470808924 bytes |
470808924 bytes |
1 |
vortex:parquet-zstd size/taxi |
0.9862053104663782 ratio |
0.9820915744429003 ratio |
1.00 |
vortex:raw size/taxi |
0.11722105760255301 ratio |
0.11673345427071811 ratio |
1.00 |
vortex size/taxi |
55188720 bytes |
54959152 bytes |
1.00 |
compress time/AirlineSentiment |
884951.7832628295 ns (1971.4306347666425) |
890317.7923202615 ns (1099.9829442402115) |
0.99 |
compress time/AirlineSentiment throughput |
2020 bytes |
2020 bytes |
1 |
parquet_rs-zstd compress time/AirlineSentiment |
55964.42244587183 ns (86.935671687992) |
56158.52456952911 ns (86.19691965942548) |
1.00 |
parquet_rs-zstd compress time/AirlineSentiment throughput |
2020 bytes |
2020 bytes |
1 |
decompress time/AirlineSentiment |
111549.16692180347 ns (1366.5541622358505) |
112069.18426171674 ns (588.9670071330911) |
1.00 |
decompress time/AirlineSentiment throughput |
2020 bytes |
2020 bytes |
1 |
parquet_rs-zstd decompress time/AirlineSentiment |
32644.60646869537 ns (43.0836543532605) |
32631.19506201017 ns (50.3679042704789) |
1.00 |
parquet_rs-zstd decompress time/AirlineSentiment throughput |
2020 bytes |
2020 bytes |
1 |
vortex:parquet-zstd size/AirlineSentiment |
8.897621509824198 ratio |
8.897621509824198 ratio |
1 |
vortex:raw size/AirlineSentiment |
4.25940594059406 ratio |
4.25940594059406 ratio |
1 |
vortex size/AirlineSentiment |
8604 bytes |
8604 bytes |
1 |
compress time/Arade |
3016403660.3 ns (17814029.299999952) |
2987317821.1 ns (5434235.650000095) |
1.01 |
compress time/Arade throughput |
787023760 bytes |
787023760 bytes |
1 |
parquet_rs-zstd compress time/Arade |
3132286432.2 ns (11286894.899999857) |
3107061960.3 ns (9499830.399999857) |
1.01 |
parquet_rs-zstd compress time/Arade throughput |
787023760 bytes |
787023760 bytes |
1 |
decompress time/Arade |
790334314.9 ns (2552172.949999988) |
786410933.9 ns (4221588.352499962) |
1.00 |
decompress time/Arade throughput |
787023760 bytes |
787023760 bytes |
1 |
parquet_rs-zstd decompress time/Arade |
701208279 ns (3819100.5) |
702191760.6 ns (2326362.75) |
1.00 |
parquet_rs-zstd decompress time/Arade throughput |
787023760 bytes |
787023760 bytes |
1 |
vortex:parquet-zstd size/Arade |
0.4888569437638918 ratio |
0.4888573628957606 ratio |
1.00 |
vortex:raw size/Arade |
0.18969384100932354 ratio |
0.1896940036473613 ratio |
1.00 |
vortex size/Arade |
149293560 bytes |
149293688 bytes |
1.00 |
compress time/Bimbo |
9980499782 ns (75937561.09375) |
10360322664.4 ns (23414287.388750076) |
0.96 |
compress time/Bimbo throughput |
7121333608 bytes |
7121333608 bytes |
1 |
parquet_rs-zstd compress time/Bimbo |
22587451007 ns (95785734.78375053) |
23016264959.2 ns (112052738.34000015) |
0.98 |
parquet_rs-zstd compress time/Bimbo throughput |
7121333608 bytes |
7121333608 bytes |
1 |
decompress time/Bimbo |
4895557146.2 ns (28482748.801250458) |
5106599405.1 ns (38505130.80999994) |
0.96 |
decompress time/Bimbo throughput |
7121333608 bytes |
7121333608 bytes |
1 |
parquet_rs-zstd decompress time/Bimbo |
4668927192.8 ns (21427719.81374979) |
4658771693.9 ns (29659200.26999998) |
1.00 |
parquet_rs-zstd decompress time/Bimbo throughput |
7121333608 bytes |
7121333608 bytes |
1 |
vortex:parquet-zstd size/Bimbo |
1.2796873649446245 ratio |
1.3360301431167363 ratio |
0.96 |
vortex:raw size/Bimbo |
0.06974984172093851 ratio |
0.07282082606233099 ratio |
0.96 |
vortex size/Bimbo |
496711892 bytes |
518581396 bytes |
0.96 |
compress time/CMSprovider |
16223402018.2 ns (74521644.05625057) |
16577688991.2 ns (96452159.9300003) |
0.98 |
compress time/CMSprovider throughput |
5149123964 bytes |
5149123964 bytes |
1 |
parquet_rs-zstd compress time/CMSprovider |
20397208595.2 ns (58639190.48500061) |
20806393948.9 ns (80877566.6712513) |
0.98 |
parquet_rs-zstd compress time/CMSprovider throughput |
5149123964 bytes |
5149123964 bytes |
1 |
decompress time/CMSprovider |
5860403802.4 ns (33813991.99749994) |
5882912307.8 ns (80178367.67750025) |
1.00 |
decompress time/CMSprovider throughput |
5149123964 bytes |
5149123964 bytes |
1 |
parquet_rs-zstd decompress time/CMSprovider |
5631768975.8 ns (40031236.27499962) |
5338505484.5 ns (95715994.73750019) |
1.05 |
parquet_rs-zstd decompress time/CMSprovider throughput |
5149123964 bytes |
5149123964 bytes |
1 |
vortex:parquet-zstd size/CMSprovider |
1.2251118693133474 ratio |
1.2354218723141748 ratio |
0.99 |
vortex:raw size/CMSprovider |
0.18308884124585043 ratio |
0.1846266562325086 ratio |
0.99 |
vortex size/CMSprovider |
942747140 bytes |
950665540 bytes |
0.99 |
compress time/Euro2016 |
3019708959.4 ns (5662368.807500124) |
3012346839.1 ns (16431724.488749743) |
1.00 |
compress time/Euro2016 throughput |
393253221 bytes |
393253221 bytes |
1 |
parquet_rs-zstd compress time/Euro2016 |
1582799604 ns (4249897.95875001) |
1594928625 ns (8136497.5500000715) |
0.99 |
parquet_rs-zstd compress time/Euro2016 throughput |
393253221 bytes |
393253221 bytes |
1 |
decompress time/Euro2016 |
344014350.8 ns (2614621.2462500036) |
346995714.6 ns (826915.650000006) |
0.99 |
decompress time/Euro2016 throughput |
393253221 bytes |
393253221 bytes |
1 |
parquet_rs-zstd decompress time/Euro2016 |
492496946.35 ns (1858619.3081249893) |
497515465.5 ns (3839581.4287500083) |
0.99 |
parquet_rs-zstd decompress time/Euro2016 throughput |
393253221 bytes |
393253221 bytes |
1 |
vortex:parquet-zstd size/Euro2016 |
1.4442903616349427 ratio |
1.4442424517596089 ratio |
1.00 |
vortex:raw size/Euro2016 |
0.4366436657870375 ratio |
0.43662918148100816 ratio |
1.00 |
vortex size/Euro2016 |
171711528 bytes |
171705832 bytes |
1.00 |
compress time/Food |
1369216715 ns (7358192.450000048) |
1402607705.2 ns (16670545.628750086) |
0.98 |
compress time/Food throughput |
332718229 bytes |
332718229 bytes |
1 |
parquet_rs-zstd compress time/Food |
1133631744 ns (9697188.65625) |
1125563478.1 ns (2130329.390000105) |
1.01 |
parquet_rs-zstd compress time/Food throughput |
332718229 bytes |
332718229 bytes |
1 |
decompress time/Food |
193621970.1666667 ns (2398854.8599999696) |
188282183.06666666 ns (1686960.7054166645) |
1.03 |
decompress time/Food throughput |
332718229 bytes |
332718229 bytes |
1 |
parquet_rs-zstd decompress time/Food |
224805514.7 ns (810891.6383333653) |
224757553.8666667 ns (1072945.4833333492) |
1.00 |
parquet_rs-zstd decompress time/Food throughput |
332718229 bytes |
332718229 bytes |
1 |
vortex:parquet-zstd size/Food |
1.2759192772025778 ratio |
1.2763167377279163 ratio |
1.00 |
vortex:raw size/Food |
0.13893571187528772 ratio |
0.13897937645009525 ratio |
1.00 |
vortex size/Food |
46226444 bytes |
46240972 bytes |
1.00 |
compress time/HashTags |
2949703151.2 ns (20650130.51999998) |
2915836662.7 ns (6506182.743750095) |
1.01 |
compress time/HashTags throughput |
804495592 bytes |
804495592 bytes |
1 |
parquet_rs-zstd compress time/HashTags |
2556466366.9 ns (13834313.349999905) |
2536571981.5 ns (4400674.0162501335) |
1.01 |
parquet_rs-zstd compress time/HashTags throughput |
804495592 bytes |
804495592 bytes |
1 |
decompress time/HashTags |
598123228.6 ns (3217115.8687500358) |
597945287.3 ns (3810210.496249974) |
1.00 |
decompress time/HashTags throughput |
804495592 bytes |
804495592 bytes |
1 |
parquet_rs-zstd decompress time/HashTags |
800509753.4 ns (8948255.379999995) |
790897598.5 ns (7307959.181249976) |
1.01 |
parquet_rs-zstd decompress time/HashTags throughput |
804495592 bytes |
804495592 bytes |
1 |
vortex:parquet-zstd size/HashTags |
1.7211443501669363 ratio |
1.7212943589400906 ratio |
1.00 |
vortex:raw size/HashTags |
0.2866068655849142 ratio |
0.2866318452121488 ratio |
1.00 |
vortex size/HashTags |
230573960 bytes |
230594056 bytes |
1.00 |
compress time/TPC-H l_comment chunked without fsst |
4316901382.5 ns (34626789.90374994) |
4405314870.8 ns (56665310.29624987) |
0.98 |
compress time/TPC-H l_comment chunked without fsst throughput |
249197090 bytes |
249197090 bytes |
1 |
parquet_rs-zstd compress time/TPC-H l_comment chunked without fsst |
909415239.2 ns (2087267.9512500167) |
917208202.5 ns (1889805.3500000238) |
0.99 |
parquet_rs-zstd compress time/TPC-H l_comment chunked without fsst throughput |
249197090 bytes |
249197090 bytes |
1 |
decompress time/TPC-H l_comment chunked without fsst |
220286816.26666665 ns (2188246.4066666514) |
218092679.16666666 ns (798783.7379166633) |
1.01 |
decompress time/TPC-H l_comment chunked without fsst throughput |
249197090 bytes |
249197090 bytes |
1 |
parquet_rs-zstd decompress time/TPC-H l_comment chunked without fsst |
250201123 ns (812756.2631250024) |
251050344.9 ns (817334.428124994) |
1.00 |
parquet_rs-zstd decompress time/TPC-H l_comment chunked without fsst throughput |
249197090 bytes |
249197090 bytes |
1 |
vortex:parquet-zstd size/TPC-H l_comment chunked without fsst |
4.6094514816250465 ratio |
4.6097550604180295 ratio |
1.00 |
vortex:raw size/TPC-H l_comment chunked without fsst |
1.0531659739686365 ratio |
1.053191142801868 ratio |
1.00 |
vortex size/TPC-H l_comment chunked without fsst |
262445896 bytes |
262452168 bytes |
1.00 |
compress time/TPC-H l_comment chunked |
1335140128.9 ns (5931160.307499886) |
1330648342.9 ns (4203783.149999976) |
1.00 |
compress time/TPC-H l_comment chunked throughput |
249197090 bytes |
249197090 bytes |
1 |
parquet_rs-zstd compress time/TPC-H l_comment chunked |
907558438 ns (2792470.992500007) |
909853894.2 ns (2810918.6225000024) |
1.00 |
parquet_rs-zstd compress time/TPC-H l_comment chunked throughput |
249197090 bytes |
249197090 bytes |
1 |
decompress time/TPC-H l_comment chunked |
156789142.00936508 ns (1173280.749357149) |
157892997.7545635 ns (427224.2606944591) |
0.99 |
decompress time/TPC-H l_comment chunked throughput |
249197090 bytes |
249197090 bytes |
1 |
parquet_rs-zstd decompress time/TPC-H l_comment chunked |
249643368.63333336 ns (907347.4166666716) |
252224453.15 ns (1122881.4381249994) |
0.99 |
parquet_rs-zstd decompress time/TPC-H l_comment chunked throughput |
249197090 bytes |
249197090 bytes |
1 |
vortex:parquet-zstd size/TPC-H l_comment chunked |
1.3591492412508186 ratio |
1.3521581212490283 ratio |
1.01 |
vortex:raw size/TPC-H l_comment chunked |
0.3105379761858375 ratio |
0.3089276845086754 ratio |
1.01 |
vortex size/TPC-H l_comment chunked |
77385160 bytes |
76983880 bytes |
1.01 |
compress time/TPC-H l_comment canonical |
1336973830.75 ns (2820770.606874943) |
1328352040.25 ns (4456466.226250052) |
1.01 |
compress time/TPC-H l_comment canonical throughput |
249197106 bytes |
249197106 bytes |
1 |
parquet_rs-zstd compress time/TPC-H l_comment canonical |
916667579.7 ns (4017024.6587499976) |
914826188.85 ns (1868717.762499988) |
1.00 |
parquet_rs-zstd compress time/TPC-H l_comment canonical throughput |
249197106 bytes |
249197106 bytes |
1 |
decompress time/TPC-H l_comment canonical |
157795692.39718255 ns (1278811.0523809195) |
158406123.25684524 ns (633223.0301614553) |
1.00 |
decompress time/TPC-H l_comment canonical throughput |
249197106 bytes |
249197106 bytes |
1 |
parquet_rs-zstd decompress time/TPC-H l_comment canonical |
249432568.1963889 ns (752616.5993263721) |
250283378.52470237 ns (492849.85571427643) |
1.00 |
parquet_rs-zstd decompress time/TPC-H l_comment canonical throughput |
249197106 bytes |
249197106 bytes |
1 |
vortex:parquet-zstd size/TPC-H l_comment canonical |
1.3591437986094663 ratio |
1.352162704925404 ratio |
1.01 |
vortex:raw size/TPC-H l_comment canonical |
0.3105379562473731 ratio |
0.3089276646736018 ratio |
1.01 |
vortex size/TPC-H l_comment canonical |
77385160 bytes |
76983880 bytes |
1.01 |
This comment was automatically generated by workflow using github-action-benchmark.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No pooling right now. I had implemented a naive buffer pool on a separate branch and it drastically hurt performance, so just getting this up as a first step