sort operator did not clear the disk files after spilling

[guo-shaoge]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

1. tiup playground start 1 tidb/1 pd/1 tikv
2. adjust slow log threshold: set global tidb_slow_log_threshold = 1;
3. prepare tpcc to setup big slow log file(50M, it may takes 3mins to finish): tiup bench tpcc prepare -D tpcc -T 10
4. reset slow log threshold: set global tidb_slow_log_threshold = 300;
5. set mem quota: set @@tidb_mem_quota_query = 50000000;
6. run following query: select time,host host_ip,Query_time as exec_max_time,parse_time,compile_time,Query as sql_text,Digest as sql_id,is_internal,succ, Plan as plan_text,mem_max as mem_max,User as parse_user,DB as database_name,total_keys,request_count,process_time,process_keys from information_schema.cluster_SLOW_QUERY order by time desc, exec_max_time desc;
7. check tmp storage: ls /tmp/1000_tidb/XXXX/tmp-storage

2. What did you expect to see? (Required)

All tmp spilling files should be removed

3. What did you see instead (Required)

There are some spilling files still exists

4. What is your TiDB version? (Required)

v6.5.3

[guo-shaoge]

The following code will not be called if child.Next() return error.

tidb/executor/sort.go

Lines 236 to 239 in 71e6696

	if e.rowChunks.NumRow() > 0 {
	e.rowChunks.Sort()
	e.partitionList = append(e.partitionList, e.rowChunks)
	}

Thus, there might be missing RowContainer entries in sortExec.partitionList. In the sortExec.Close() function, only the RowContainer entries recorded in sortExec.partitionList are closed and cleaned up. This leads to some disk files not being deleted.

[guo-shaoge]

This bug doesn't reproduce in versions later than v8.0.0, because the sortExec code has already been refactored. But we still need to fix this for older versions.

[seiya-annie]

/report customer