Skip to content

Commit

Permalink
address comments
Browse files Browse the repository at this point in the history
  • Loading branch information
lichunzhu committed Nov 23, 2022
1 parent 4c7eaf9 commit 46ddf53
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion br/pkg/lightning/mydump/region.go
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ const (
// the increment ratio of large CSV file size threshold by `region-split-size`
largeCSVLowerThresholdRation = 10
// TableFileSizeINF for compressed size, for lightning 10TB is a relatively big value and will strongly affect efficiency
// It's used to make sure compressed files can be read until EOF. Because we can't get the exact decompressed size of the compressed files.
TableFileSizeINF = 10 * 1024 * tableRegionSizeWarningThreshold
)

Expand Down Expand Up @@ -300,7 +301,7 @@ func MakeSourceFileRegion(
// set fileSize to INF to make sure compressed files can be read until EOF. Because we can't get the exact size of the compressed files.
// TODO: update progress bar calculation for compressed files.
if fi.FileMeta.Compression != CompressionNone {
rowIDMax = fileSize * 100 / divisor
rowIDMax = fileSize * 100 / divisor // FIXME: this is not accurate. Need more tests and fix solution.
fileSize = TableFileSizeINF
}
tableRegion := &TableRegion{
Expand Down

0 comments on commit 46ddf53

Please sign in to comment.