Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The priority queue panic when refreshing the last analysis time #57648

Closed
Rustin170506 opened this issue Nov 23, 2024 · 0 comments · Fixed by #57649
Closed

The priority queue panic when refreshing the last analysis time #57648

Rustin170506 opened this issue Nov 23, 2024 · 0 comments · Fixed by #57649
Labels
affects-8.5 This bug affects the 8.5.x(LTS) versions. component/statistics impact/panic severity/major sig/planner SIG: Planner type/bug The issue is confirmed as a bug.

Comments

@Rustin170506
Copy link
Member

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

We found this problem on a very large cluster with a heavy workload.

2. What did you expect to see? (Required)

No panic

3. What did you see instead (Required)

tidb-2024-11-22T11-43-27.742.log:[2024/11/22 10:00:45.192 +00:00] [ERROR] [queue.go:302] ["Priority queue panicked"] [category=stats] [recover="runtime error: invalid memory address or nil pointer dereference"] [stack="github.com/pingcap/tidb/pkg/statistics/handle/autoanalyze/priorityqueue.(*AnalysisPriorityQueue).run.func1\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/statistics/handle/autoanalyze/priorityqueue/queue.go:302\nruntime.gopanic\n\t/usr/local/go/src/runtime/panic.go:785\nruntime.panicmem\n\t/usr/local/go/src/runtime/panic.go:262\nruntime.sigpanic\n\t/usr/local/go/src/runtime/signal_unix.go:917\ngithub.com/pingcap/tidb/pkg/statistics.(*Table).IsAnalyzed\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/statistics/table.go:720\ngithub.com/pingcap/tidb/pkg/statistics/handle/autoanalyze/priorityqueue.(*AnalysisJobFactory).FindLastAnalyzeTime\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/statistics/handle/autoanalyze/priorityqueue/analysis_job_factory.go:209\ngithub.com/pingcap/tidb/pkg/statistics/handle/autoanalyze/priorityqueue.(*AnalysisJobFactory).GetTableLastAnalyzeDuration\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/statistics/handle/autoanalyze/priorityqueue/analysis_job_factory.go:197\ngithub.com/pingcap/tidb/pkg/statistics/handle/autoanalyze/priorityqueue.(*AnalysisPriorityQueue).RefreshLastAnalysisDuration.func1\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/statistics/handle/autoanalyze/priorityqueue/queue.go:656\ngithub.com/pingcap/tidb/pkg/statistics/handle/util.WrapTxn\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/statistics/handle/util/util.go:206\ngithub.com/pingcap/tidb/pkg/statistics/handle/util.CallWithSCtx\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/statistics/handle/util/util.go:104\ngithub.com/pingcap/tidb/pkg/statistics/handle/autoanalyze/priorityqueue.(*AnalysisPriorityQueue).RefreshLastAnalysisDuration\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/statistics/handle/autoanalyze/priorityqueue/queue.go:624\ngithub.com/pingcap/tidb/pkg/statistics/handle/autoanalyze/priorityqueue.(*AnalysisPriorityQueue).run\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/statistics/handle/autoanalyze/priorityqueue/queue.go:323\ngithub.com/pingcap/tidb/pkg/util.(*WaitGroupWrapper).Run.func1\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/util/wait_group_wrapper.go:157"]
tidb-2024-11-22T11-43-27.742.log:[2024/11/22 10:04:17.666 +00:00] [INFO] [cache.go:119] ["reset the old infoschema after v1 v2 switch, using the stale object will panic"]
tidb-2024-11-22T11-43-27.742.log:[2024/11/22 10:20:00.976 +00:00] [ERROR] [queue.go:302] ["Priority queue panicked"] [category=stats] [recover="runtime error: invalid memory address or nil pointer dereference"] [stack="github.com/pingcap/tidb/pkg/statistics/handle/autoanalyze/priorityqueue.(*AnalysisPriorityQueue).run.func1\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/statistics/handle/autoanalyze/priorityqueue/queue.go:302\nruntime.gopanic\n\t/usr/local/go/src/runtime/panic.go:785\nruntime.panicmem\n\t/usr/local/go/src/runtime/panic.go:262\nruntime.sigpanic\n\t/usr/local/go/src/runtime/signal_unix.go:917\ngithub.com/pingcap/tidb/pkg/statistics.(*Table).IsAnalyzed\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/statistics/table.go:720\ngithub.com/pingcap/tidb/pkg/statistics/handle/autoanalyze/priorityqueue.(*AnalysisJobFactory).FindLastAnalyzeTime\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/statistics/handle/autoanalyze/priorityqueue/analysis_job_factory.go:209\ngithub.com/pingcap/tidb/pkg/statistics/handle/autoanalyze/priorityqueue.(*AnalysisJobFactory).GetTableLastAnalyzeDuration\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/statistics/handle/autoanalyze/priorityqueue/analysis_job_factory.go:197\ngithub.com/pingcap/tidb/pkg/statistics/handle/autoanalyze/priorityqueue.(*AnalysisPriorityQueue).RefreshLastAnalysisDuration.func1\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/statistics/handle/autoanalyze/priorityqueue/queue.go:656\ngithub.com/pingcap/tidb/pkg/statistics/handle/util.WrapTxn\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/statistics/handle/util/util.go:206\ngithub.com/pingcap/tidb/pkg/statistics/handle/util.CallWithSCtx\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/statistics/handle/util/util.go:104\ngithub.com/pingcap/tidb/pkg/statistics/handle/autoanalyze/priorityqueue.(*AnalysisPriorityQueue).RefreshLastAnalysisDuration\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/statistics/handle/autoanalyze/priorityqueue/queue.go:624\ngithub.com/pingcap/tidb/pkg/statistics/handle/autoanalyze/priorityqueue.(*AnalysisPriorityQueue).run\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/statistics/handle/autoanalyze/priorityqueue/queue.go:323\ngithub.com/pingcap/tidb/pkg/util.(*WaitGroupWrapper).Run.func1\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/util/wait_group_wrapper.go:157"]
tidb.log:[2024/11/23 03:03:22.126 +00:00] [ERROR] [worker.go:87] ["Auto analyze job execution failed"] [category=stats] [recover="assignment to entry in nil map"] [stack="github.com/pingcap/tidb/pkg/statistics/handle/autoanalyze/refresher.(*worker).SubmitJob.func2\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/statistics/handle/autoanalyze/refresher/worker.go:87\ngithub.com/pingcap/tidb/pkg/util.(*WaitGroupWrapper).RunWithRecover.func1.1\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/util/wait_group_wrapper.go:185\nruntime.gopanic\n\t/usr/local/go/src/runtime/panic.go:785\nruntime.mapassign_fast64\n\t/usr/local/go/src/runtime/map_fast64.go:115\ngithub.com/pingcap/tidb/pkg/statistics/handle/autoanalyze/priorityqueue.(*AnalysisPriorityQueue).Pop.func2\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/statistics/handle/autoanalyze/priorityqueue/queue.go:763\ngithub.com/pingcap/tidb/pkg/statistics/handle/autoanalyze/priorityqueue.(*DynamicPartitionedTableAnalysisJob).Analyze.func1\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/statistics/handle/autoanalyze/priorityqueue/dynamic_partitioned_table_analysis_job.go:110\ngithub.com/pingcap/tidb/pkg/statistics/handle/autoanalyze/priorityqueue.(*DynamicPartitionedTableAnalysisJob).Analyze\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/statistics/handle/autoanalyze/priorityqueue/dynamic_partitioned_table_analysis_job.go:115\ngithub.com/pingcap/tidb/pkg/statistics/handle/autoanalyze/refresher.(*worker).processJob\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/statistics/handle/autoanalyze/refresher/worker.go:102\ngithub.com/pingcap/tidb/pkg/statistics/handle/autoanalyze/refresher.(*worker).SubmitJob.func1\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/statistics/handle/autoanalyze/refresher/worker.go:83\ngithub.com/pingcap/tidb/pkg/util.(*WaitGroupWrapper).RunWithRecover.func1\n\t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/util/wait_group_wrapper.go:189"]

4. What is your TiDB version? (Required)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects-8.5 This bug affects the 8.5.x(LTS) versions. component/statistics impact/panic severity/major sig/planner SIG: Planner type/bug The issue is confirmed as a bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants