GC is wrongly triggered in dumpling+ million tables scenario

[lilinghai]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

millions tables
dumpling dump a database with 2 tables

tiup dumpling:nightly -u root -P 4000 -h [10.104.100.98](http://10.104.100.98/) --filetype csv -t 16 -r 200000 -F 256MiB --database sb10wnormal1
The component `dumpling` version v8.5.0-alpha-nightly is not installed; downloading from repository.
download https://tiup-mirrors.pingcap.com/dumpling-v8.5.0-alpha-nightly-linux-amd64.tar.gz 37.20 MiB / 37.20 MiB 100.00% 44.63 MiB/s
Starting component dumpling: /root/.tiup/components/dumpling/v8.5.0-alpha-nightly/dumpling -u root -P 4000 -h [10.104.100.98](http://10.104.100.98/) --filetype csv -t 16 -r 200000 -F 256MiB --database sb10wnormal1
Release version: v8.5.0-alpha-258-g6d7407156c
Git commit hash: 6d7407156c17ce9785c91d716a640d641feafc7a
Git branch:      HEAD
Build timestamp: 2024-12-02 05:17:01Z
Go version:      go version go1.23.3 linux/amd64

[2024/12/02 06:50:54.598 +00:00] [INFO] [versions.go:54] ["Welcome to dumpling"] ["Release Version"=v8.5.0-alpha-258-g6d7407156c] ["Git Commit Hash"=6d7407156c17ce9785c91d716a640d641feafc7a] ["Git Branch"=HEAD] ["Build timestamp"="2024-12-02 05:17:01"] ["Go Version"="go version go1.23.3 linux/amd64"]
[2024/12/02 06:50:54.600 +00:00] [INFO] [version.go:457] ["detect server version"] [type=TiDB] [version=8.5.0-alpha-256-gec288d95a3]
[2024/12/02 06:50:54.631 +00:00] [INFO] [pd_service_discovery.go:1025] ["[pd] switch leader"] [new-leader=http://tc-pd-0.tc-pd-peer.forge2-tps-7690491-1-273.svc:2379/] [old-leader=]
[2024/12/02 06:50:54.631 +00:00] [INFO] [pd_service_discovery.go:499] ["[pd] init cluster id"] [cluster-id=7441598078004266891]
[2024/12/02 06:50:54.631 +00:00] [INFO] [client.go:532] ["[pd] changing service mode"] [old-mode=UNKNOWN_SVC_MODE] [new-mode=PD_SVC_MODE]
[2024/12/02 06:50:54.631 +00:00] [INFO] [tso_client.go:296] ["[tso] switch dc tso global allocator serving url"] [dc-location=global] [new-url=http://tc-pd-0.tc-pd-peer.forge2-tps-7690491-1-273.svc:2379/]
[2024/12/02 06:50:54.632 +00:00] [INFO] [client.go:538] ["[pd] service mode changed"] [old-mode=UNKNOWN_SVC_MODE] [new-mode=PD_SVC_MODE]
[2024/12/02 06:50:54.632 +00:00] [INFO] [tso_dispatcher.go:140] ["[tso] start tso deadline watcher"] [dc-location=global]
[2024/12/02 06:50:54.632 +00:00] [INFO] [tso_dispatcher.go:198] ["[tso] tso dispatcher created"] [dc-location=global]
[2024/12/02 06:50:54.632 +00:00] [INFO] [tso_client.go:132] ["[tso] start tso dispatcher check loop"]
[2024/12/02 06:50:54.632 +00:00] [INFO] [tso_dispatcher.go:476] ["[tso] start tso connection contexts updater"] [dc-location=global]
[2024/12/02 06:50:54.632 +00:00] [INFO] [tso_dispatcher.go:723] ["[tso] switching tso rpc concurrency"] [old=0] [new=1]
[2024/12/02 06:50:54.632 +00:00] [INFO] [dump.go:1523] ["generate dumpling gc safePoint id"] [id=dumpling_1733122254632766329]
[2024/12/02 06:50:54.641 +00:00] [INFO] [dump.go:152] ["begin to run Dump"] [conf="{\"s3\":{\"endpoint\":\"\",\"region\":\"\",\"storage-class\":\"\",\"sse\":\"\",\"sse-kms-key-id\":\"\",\"acl\":\"\",\"access-key\":\"\",\"secret-access-key\":\"\",\"session-token\":\"\",\"provider\":\"\",\"force-path-style\":true,\"use-accelerate-endpoint\":false,\"role-arn\":\"\",\"external-id\":\"\",\"object-lock-enabled\":false},\"gcs\":{\"endpoint\":\"\",\"storage-class\":\"\",\"predefined-acl\":\"\",\"credentials-file\":\"\"},\"azblob\":{\"endpoint\":\"\",\"account-name\":\"\",\"account-key\":\"\",\"access-tier\":\"\",\"sas-token\":\"\",\"encryption-scope\":\"\",\"encryption-key\":\"\"},\"SpecifiedTables\":false,\"AllowCleartextPasswords\":false,\"SortByPk\":true,\"NoViews\":true,\"NoSequences\":true,\"NoHeader\":false,\"NoSchemas\":false,\"NoData\":false,\"CompleteInsert\":false,\"TransactionalConsistency\":true,\"EscapeBackslash\":true,\"DumpEmptyDatabase\":true,\"PosAfterConnect\":false,\"CompressType\":0,\"Host\":\"[10.104.100.98](http://10.104.100.98/)\",\"Port\":4000,\"Threads\":16,\"User\":\"root\",\"Security\":{\"CAPath\":\"\",\"CertPath\":\"\",\"KeyPath\":\"\"},\"LogLevel\":\"info\",\"LogFile\":\"\",\"LogFormat\":\"text\",\"OutputDirPath\":\"./export-2024-12-02T06:50:54Z\",\"StatusAddr\":\":8281\",\"Snapshot\":\"454327600314580994\",\"Consistency\":\"snapshot\",\"CsvNullValue\":\"\\\\N\",\"SQL\":\"\",\"CsvSeparator\":\",\",\"CsvDelimiter\":\"\\\"\",\"CsvLineTerminator\":\"\\r\\n\",\"Databases\":[\"sb10wnormal1\"],\"Where\":\"\",\"FileType\":\"csv\",\"ServerInfo\":{\"ServerType\":3,\"ServerVersion\":\"8.5.0-alpha-256-gec288d95a3\",\"HasTiKV\":true},\"Rows\":200000,\"ReadTimeout\":900000000000,\"TiDBMemQuotaQuery\":0,\"FileSize\":268435456,\"StatementSize\":1000000,\"SessionParams\":{\"tidb_snapshot\":\"454327600314580994\"},\"Tables\":{},\"CollationCompatible\":\"loose\",\"CsvOutputDialect\":0,\"IOTotalBytes\":null,\"Net\":\"\"}"]
[2024/12/02 07:05:42.660 +00:00] [INFO] [resource_manager_client.go:296] ["[resource manager] exit resource token dispatcher"]
[2024/12/02 07:05:42.660 +00:00] [INFO] [tso_stream.go:359] ["tsoStream.recvLoop ended"] [stream=[tc-pd-0.tc](http://tc-pd-0.tc/)-pd-peer.forge2-tps-7690491-1-273.svc:2379-1] [error="rpc error: code = Canceled desc = context canceled"] [errorVerbose="rpc error: code = Canceled desc = context canceled\[ngithub.com/tikv/pd/client.(*tsoStream).recvLoop\n\tgithub.com/tikv/pd/[email protected]/tso_stream.go:427\nruntime.goexit\n\truntime/asm_amd64.s:1700](http://ngithub.com/tikv/pd/client.(*tsoStream).recvLoop/n/tgithub.com/tikv/pd/[email protected]/tso_stream.go:427/nruntime.goexit/n/truntime/asm_amd64.s:1700)"]
[2024/12/02 07:05:42.661 +00:00] [INFO] [tso_dispatcher.go:490] ["[tso] exit tso connection contexts updater"] [dc-location=global]
[2024/12/02 07:05:42.660 +00:00] [INFO] [tso_dispatcher.go:264] ["[tso] stop fetching the pending tso requests due to context canceled"] [dc-location=global]
[2024/12/02 07:05:42.661 +00:00] [INFO] [tso_dispatcher.go:201] ["[tso] exit tso dispatcher"] [dc-location=global]
[2024/12/02 07:05:42.661 +00:00] [INFO] [pd_service_discovery.go:551] ["[pd] exit member loop due to context canceled"]
[2024/12/02 07:05:42.661 +00:00] [INFO] [tso_client.go:147] ["[tso] exit tso dispatcher check loop"]
[2024/12/02 07:05:42.661 +00:00] [INFO] [tso_dispatcher.go:158] ["[tso] exit tso deadline watcher"] [dc-location=global]
[2024/12/02 07:05:42.661 +00:00] [ERROR] [main.go:78] ["dump failed error stack info"] [error="sql: SELECT COUNT(1) as c FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_TYPE='SEQUENCE': Error 9006 (HY000): GC life time is shorter than transaction duration, transaction starts at 2024-12-02 06:50:38.718 +0000 UTC, GC safe point is 2024-12-02 06:50:54.618 +0000 UTC"] [errorVerbose="Error 9006 (HY000): GC life time is shorter than transaction duration, transaction starts at 2024-12-02 06:50:38.718 +0000 UTC, GC safe point is 2024-12-02 06:50:54.618 +0000 UTC\nsql: SELECT COUNT(1) as c FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_TYPE='SEQUENCE'\[ngithub.com/pingcap/tidb/dumpling/export.CheckIfSeqExists\n\tgithub.com/pingcap/tidb/dumpling/export/sql.go:932\ngithub.com/pingcap/tidb/dumpling/export.prepareTableListToDump\n\tgithub.com/pingcap/tidb/dumpling/export/dump.go:1153\ngithub.com/pingcap/tidb/dumpling/export.(*Dumper).Dump\n\tgithub.com/pingcap/tidb/dumpling/export/dump.go:218\nmain.main\n\t./main.go:75\nruntime.main\n\truntime/proc.go:272\nruntime.goexit\n\truntime/asm_amd64.s:1700](http://ngithub.com/pingcap/tidb/dumpling/export.CheckIfSeqExists/n/tgithub.com/pingcap/tidb/dumpling/export/sql.go:932/ngithub.com/pingcap/tidb/dumpling/export.prepareTableListToDump/n/tgithub.com/pingcap/tidb/dumpling/export/dump.go:1153/ngithub.com/pingcap/tidb/dumpling/export.(*Dumper).Dump/n/tgithub.com/pingcap/tidb/dumpling/export/dump.go:218/nmain.main/n/t./main.go:75/nruntime.main/n/truntime/proc.go:272/nruntime.goexit/n/truntime/asm_amd64.s:1700)"]

dump failed: sql: SELECT COUNT(1) as c FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_TYPE='SEQUENCE': Error 9006 (HY000): GC life time is shorter than transaction duration, transaction starts at 2024-12-02 06:50:38.718 +0000 UTC, GC safe point is 2024-12-02 06:50:54.618 +0000 UTC

2. What did you expect to see? (Required)

3. What did you see instead (Required)

4. What is your TiDB version? (Required)

master

[lance6716]

Dumpling's gc safepoint, and the "SELECT COUNT(1) as c FROM INFORMATION_SCHEMA.TABLES" query should block GC.

For dumpling optimization and workaround, I have separated #57902

[jebter]

/assign @tiancaiamao