Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DATA RACE in the executor.(*IndexNestedLoopHashJoin) #49692

Closed
Tracked by #41316
hawkingrei opened this issue Dec 22, 2023 · 3 comments · Fixed by #55824
Closed
Tracked by #41316

DATA RACE in the executor.(*IndexNestedLoopHashJoin) #49692

hawkingrei opened this issue Dec 22, 2023 · 3 comments · Fixed by #55824
Labels
affects-6.1 This bug affects the 6.1.x(LTS) versions. affects-6.5 This bug affects the 6.5.x(LTS) versions. affects-7.1 This bug affects the 7.1.x(LTS) versions. affects-7.5 This bug affects the 7.5.x(LTS) versions. affects-8.1 This bug affects the 8.1.x(LTS) versions. component/test severity/major sig/execution SIG execution type/bug The issue is confirmed as a bug.

Comments

@hawkingrei
Copy link
Member

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

2. What did you expect to see? (Required)

3. What did you see instead (Required)

==================
WARNING: DATA RACE
Read at 0x00c006fd8bd8 by goroutine 47:
  github.com/pingcap/tidb/pkg/executor.(*IndexNestedLoopHashJoin).getResultFromChannel()
      pkg/executor/index_lookup_hash_join.go:294 +0x1a4
  github.com/pingcap/tidb/pkg/executor.(*IndexNestedLoopHashJoin).runUnordered()
      pkg/executor/index_lookup_hash_join.go:256 +0x88
  github.com/pingcap/tidb/pkg/executor.(*IndexNestedLoopHashJoin).Next()
      pkg/executor/index_lookup_hash_join.go:232 +0x1cf
  github.com/pingcap/tidb/pkg/executor/internal/exec.Next()
      pkg/executor/internal/exec/executor.go:309 +0x34b
  github.com/pingcap/tidb/pkg/executor.(*ExecStmt).next()
      pkg/executor/adapter.go:1245 +0x84
  github.com/pingcap/tidb/pkg/executor.(*recordSet).Next()
      pkg/executor/adapter.go:156 +0x19a
  github.com/pingcap/tidb/pkg/session.(*execStmtResult).Next()
      <autogenerated>:1 +0x6c
  github.com/pingcap/tidb/pkg/session.GetRows4Test()
      pkg/session/tidb.go:363 +0xd2
  github.com/pingcap/tidb/pkg/testkit.(*TestKit).QueryToErr()
      pkg/testkit/testkit.go:239 +0x2ea
  pkg/executor/test/jointest/jointest_test.TestIssue30211()
      pkg/executor/test/jointest/join_test.go:780 +0x284
  github.com/pingcap/tidb/pkg/session.runStmt()
      pkg/session/session.go:2386 +0x71a
  github.com/pingcap/tidb/pkg/session.(*session).ExecuteStmt()
      pkg/session/session.go:2228 +0x1af8
  github.com/pingcap/tidb/pkg/testkit.(*TestKit).ExecWithContext()
      pkg/testkit/testkit.go:364 +0x889
  github.com/pingcap/tidb/pkg/testkit.(*TestKit).MustExecWithContext()
      pkg/testkit/testkit.go:136 +0xab
  github.com/pingcap/tidb/pkg/testkit.(*TestKit).MustExec()
      pkg/testkit/testkit.go:131 +0x128
  pkg/executor/test/jointest/jointest_test.TestIssue30211()
      pkg/executor/test/jointest/join_test.go:776 +0x187
  github.com/pingcap/tidb/pkg/domain.(*Domain).LoadSysVarCacheLoop()
      pkg/domain/domain.go:1715 +0x93
  github.com/pingcap/tidb/pkg/session.bootstrapSessionImpl()
      pkg/session/session.go:3350 +0x724
  github.com/pingcap/tidb/pkg/domain.(*Domain).GetSessionCache()
      pkg/domain/sysvar_cache.go:62 +0x50
  github.com/pingcap/tidb/pkg/session.(*session).loadCommonGlobalVariablesIfNeeded()
      pkg/session/session.go:3722 +0xf4
  github.com/pingcap/tidb/pkg/session.(*session).ExecuteStmt()
      pkg/session/session.go:2088 +0x184
  github.com/pingcap/tidb/pkg/session.(*session).ExecuteInternal()
      pkg/session/session.go:1600 +0x3b9
  github.com/pingcap/tidb/pkg/domain.(*Domain).LoadPrivilegeLoop()
      pkg/domain/domain.go:1658 +0x108
  github.com/pingcap/tidb/pkg/session.bootstrapSessionImpl()
      pkg/session/session.go:3343 +0x6ca
  github.com/pingcap/tidb/pkg/session.BootstrapSession()
      pkg/session/session.go:3264 +0xad
  github.com/pingcap/tidb/pkg/testkit.bootstrap()
      pkg/testkit/mockstore.go:227 +0x94
  github.com/pingcap/tidb/pkg/testkit.CreateMockStoreAndDomain()
      pkg/testkit/mockstore.go:200 +0xd5
  github.com/pingcap/tidb/pkg/testkit.CreateMockStore()
      pkg/testkit/mockstore.go:68 +0x3d1
  pkg/executor/test/jointest/jointest_test.TestIssue30211()
      pkg/executor/test/jointest/join_test.go:754 +0x49
  testing.tRunner()
      GOROOT/src/testing/testing.go:1595 +0x261
  testing.(*T).Run.func1()
      GOROOT/src/testing/testing.go:1648 +0x44
Previous write at 0x00c006fd8bd8 by goroutine 42866:
  github.com/pingcap/tidb/pkg/executor.(*IndexNestedLoopHashJoin).finishJoinWorkers()
      pkg/executor/index_lookup_hash_join.go:204 +0x33b
  github.com/pingcap/tidb/pkg/executor.(*IndexNestedLoopHashJoin).finishJoinWorkers-fm()
      <autogenerated>:1 +0x47
  github.com/pingcap/tidb/pkg/util.WithRecovery.func1()
      pkg/util/misc.go:88 +0x5d
  runtime.gopanic()
      GOROOT/src/runtime/panic.go:920 +0x26f
  github.com/pingcap/tidb/pkg/executor.(*IndexNestedLoopHashJoin).newInnerWorker()
      pkg/executor/index_lookup_hash_join.go:469 +0xfca
  github.com/pingcap/tidb/pkg/executor.(*IndexNestedLoopHashJoin).startWorkers.func2()
      pkg/executor/index_lookup_hash_join.go:186 +0x58
  github.com/pingcap/tidb/pkg/util.WithRecovery()
      pkg/util/misc.go:96 +0x77
  github.com/pingcap/tidb/pkg/executor.(*IndexNestedLoopHashJoin).startWorkers.func4()
      pkg/executor/index_lookup_hash_join.go:186 +0x44
Goroutine 47 (running) created at:
  testing.(*T).Run()
      GOROOT/src/testing/testing.go:1648 +0x845
  testing.runTests.func1()
      GOROOT/src/testing/testing.go:2054 +0x84
  testing.tRunner()
      GOROOT/src/testing/testing.go:1595 +0x261
  testing.runTests()
      GOROOT/src/testing/testing.go:2052 +0x8ad
  testing.(*M).Run()
      GOROOT/src/testing/testing.go:1925 +0xcd7
  go.uber.org/goleak.VerifyTestMain()
      external/org_uber_go_goleak/testmain.go:53 +0x64
  pkg/executor/test/jointest/jointest_test.TestMain()
      pkg/executor/test/jointest/main_test.go:44 +0x676
  main.main()
      bazel-out/k8-fastbuild/bin/pkg/executor/test/jointest/jointest_test_/testmain.go:137 +0x5cb
Goroutine 42866 (finished) created at:
  github.com/pingcap/tidb/pkg/executor.(*IndexNestedLoopHashJoin).startWorkers()
      pkg/executor/index_lookup_hash_join.go:186 +0x8dc
  github.com/pingcap/tidb/pkg/executor.(*IndexNestedLoopHashJoin).Next()
      pkg/executor/index_lookup_hash_join.go:225 +0x84
  github.com/pingcap/tidb/pkg/executor/internal/exec.Next()
      pkg/executor/internal/exec/executor.go:309 +0x34b
  github.com/pingcap/tidb/pkg/executor.(*ExecStmt).next()
      pkg/executor/adapter.go:1245 +0x84
  github.com/pingcap/tidb/pkg/executor.(*recordSet).Next()
      pkg/executor/adapter.go:156 +0x19a
  github.com/pingcap/tidb/pkg/session.(*execStmtResult).Next()
      <autogenerated>:1 +0x6c
  github.com/pingcap/tidb/pkg/session.GetRows4Test()
      pkg/session/tidb.go:363 +0xd2
  github.com/pingcap/tidb/pkg/testkit.(*TestKit).QueryToErr()
      pkg/testkit/testkit.go:239 +0x2ea
  pkg/executor/test/jointest/jointest_test.TestIssue30211()
      pkg/executor/test/jointest/join_test.go:780 +0x284
  github.com/pingcap/tidb/pkg/session.runStmt()
      pkg/session/session.go:2386 +0x71a
  github.com/pingcap/tidb/pkg/session.(*session).ExecuteStmt()
      pkg/session/session.go:2228 +0x1af8
  github.com/pingcap/tidb/pkg/testkit.(*TestKit).ExecWithContext()
      pkg/testkit/testkit.go:364 +0x889
  github.com/pingcap/tidb/pkg/testkit.(*TestKit).MustExecWithContext()
      pkg/testkit/testkit.go:136 +0xab
  github.com/pingcap/tidb/pkg/testkit.(*TestKit).MustExec()
      pkg/testkit/testkit.go:131 +0x128
  pkg/executor/test/jointest/jointest_test.TestIssue30211()
      pkg/executor/test/jointest/join_test.go:776 +0x187
  github.com/pingcap/tidb/pkg/domain.(*Domain).LoadSysVarCacheLoop()
      pkg/domain/domain.go:1715 +0x93
  github.com/pingcap/tidb/pkg/session.bootstrapSessionImpl()
      pkg/session/session.go:3350 +0x724
  github.com/pingcap/tidb/pkg/domain.(*Domain).GetSessionCache()
      pkg/domain/sysvar_cache.go:62 +0x50
  github.com/pingcap/tidb/pkg/session.(*session).loadCommonGlobalVariablesIfNeeded()
      pkg/session/session.go:3722 +0xf4
  github.com/pingcap/tidb/pkg/session.(*session).ExecuteStmt()
      pkg/session/session.go:2088 +0x184
  github.com/pingcap/tidb/pkg/session.(*session).ExecuteInternal()
      pkg/session/session.go:1600 +0x3b9
  github.com/pingcap/tidb/pkg/domain.(*Domain).LoadPrivilegeLoop()
      pkg/domain/domain.go:1658 +0x108
  github.com/pingcap/tidb/pkg/session.bootstrapSessionImpl()
      pkg/session/session.go:3343 +0x6ca
  github.com/pingcap/tidb/pkg/session.BootstrapSession()
      pkg/session/session.go:3264 +0xad
  github.com/pingcap/tidb/pkg/testkit.bootstrap()
      pkg/testkit/mockstore.go:227 +0x94
  github.com/pingcap/tidb/pkg/testkit.CreateMockStoreAndDomain()
      pkg/testkit/mockstore.go:200 +0xd5
  github.com/pingcap/tidb/pkg/testkit.CreateMockStore()
      pkg/testkit/mockstore.go:68 +0x3d1
  pkg/executor/test/jointest/jointest_test.TestIssue30211()
      pkg/executor/test/jointest/join_test.go:754 +0x49
  testing.tRunner()
      GOROOT/src/testing/testing.go:1595 +0x261
  testing.(*T).Run.func1()
      GOROOT/src/testing/testing.go:1648 +0x44
==================

4. What is your TiDB version? (Required)
@hawkingrei hawkingrei added the type/bug The issue is confirmed as a bug. label Dec 22, 2023
@hawkingrei hawkingrei changed the title DATA RACE in the executor.(*IndexNestedLoopHashJoin DATA RACE in the executor.(*IndexNestedLoopHashJoin) Dec 22, 2023
@hawkingrei
Copy link
Member Author

https://prow.hawkingrei.com/view/gs/prowpingcap/logs/tidb_race/1737931843371536384#1:build-log.txt%3A2577

@hawkingrei hawkingrei mentioned this issue Dec 22, 2023
Open
@ti-chi-bot ti-chi-bot added the affects-8.1 This bug affects the 8.1.x(LTS) versions. label Apr 9, 2024
@jebter jebter added the sig/planner SIG: Planner label Apr 29, 2024
@qw4990
Copy link
Contributor

qw4990 commented May 20, 2024

IndexNestedLoopHashJoin.panicErr is not thread-safe:
image

Transfer this issue to Executor Team.

@qw4990 qw4990 added sig/execution SIG execution and removed sig/planner SIG: Planner labels May 20, 2024
@qw4990
Copy link
Contributor

qw4990 commented May 20, 2024

PTAL @XuHuaiyu

@solotzg solotzg mentioned this issue Sep 3, 2024
Merged
13 tasks
@windtalker windtalker added affects-6.1 This bug affects the 6.1.x(LTS) versions. affects-6.5 This bug affects the 6.5.x(LTS) versions. affects-7.1 This bug affects the 7.1.x(LTS) versions. affects-7.5 This bug affects the 7.5.x(LTS) versions. and removed may-affects-5.4 This bug maybe affects 5.4.x versions. may-affects-6.1 may-affects-6.5 may-affects-7.1 may-affects-7.5 labels Sep 4, 2024
@ti-chi-bot ti-chi-bot bot closed this as completed in 8cdd449 Sep 4, 2024
This was referenced Sep 6, 2024
Merged
Merged
Merged
Merged
Merged
ti-chi-bot bot pushed a commit that referenced this issue Sep 9, 2024
@ti-chi-bot
executor: fix data race in IndexNestedLoopHashJoin (#55824) (#55909)
58bb25d 
close #49692
ti-chi-bot bot pushed a commit that referenced this issue Sep 10, 2024
@ti-chi-bot
executor: fix data race in IndexNestedLoopHashJoin (#55824) (#55908)
5665343 
close #49692
ti-chi-bot bot pushed a commit that referenced this issue Sep 10, 2024
@ti-chi-bot
executor: fix data race in IndexNestedLoopHashJoin (#55824) (#55911)
854bf3c 
close #49692
ti-chi-bot bot pushed a commit that referenced this issue Sep 10, 2024
@ti-chi-bot
executor: fix data race in IndexNestedLoopHashJoin (#55824) (#55910)
acc2a65 
close #49692
ti-chi-bot bot pushed a commit that referenced this issue Sep 10, 2024
@ti-chi-bot
executor: fix data race in IndexNestedLoopHashJoin (#55824) (#55907)
92acec0 
close #49692
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
affects-6.1 This bug affects the 6.1.x(LTS) versions. affects-6.5 This bug affects the 6.5.x(LTS) versions. affects-7.1 This bug affects the 7.1.x(LTS) versions. affects-7.5 This bug affects the 7.5.x(LTS) versions. affects-8.1 This bug affects the 8.1.x(LTS) versions. component/test severity/major sig/execution SIG execution type/bug The issue is confirmed as a bug.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

executor: fix data race in IndexNestedLoopHashJoin solotzg/tidb
6 participants
@windtalker @hawkingrei @jebter @qw4990 @seiya-annie @ti-chi-bot