internal error: entered unreachable code in single_stream.rs

[tomaka]

Got another crash

$ node demo/demo.mjs 
JSON-RPC server now listening on port 9944
Please visit one of:
- westend2: https://polkadot.js.org/apps/?rpc=ws%3A%2F%2F127.0.0.1%3A9944%2Fwestend2
- westmint: https://polkadot.js.org/apps/?rpc=ws%3A%2F%2F127.0.0.1%3A9944%2Fwestmint
- polkadot: https://polkadot.js.org/apps/?rpc=ws%3A%2F%2F127.0.0.1%3A9944%2Fpolkadot
- acala: https://polkadot.js.org/apps/?rpc=ws%3A%2F%2F127.0.0.1%3A9944%2Facala
- ksmcc3: https://polkadot.js.org/apps/?rpc=ws%3A%2F%2F127.0.0.1%3A9944%2Fksmcc3
- statemine: https://polkadot.js.org/apps/?rpc=ws%3A%2F%2F127.0.0.1%3A9944%2Fstatemine
- karura: https://polkadot.js.org/apps/?rpc=ws%3A%2F%2F127.0.0.1%3A9944%2Fkarura
- rococo_v2_2: https://polkadot.js.org/apps/?rpc=ws%3A%2F%2F127.0.0.1%3A9944%2Frococo_v2_2
- adz_rococo: https://polkadot.js.org/apps/?rpc=ws%3A%2F%2F127.0.0.1%3A9944%2Fadz_rococo
- canvas-rococo: https://polkadot.js.org/apps/?rpc=ws%3A%2F%2F127.0.0.1%3A9944%2Fcanvas-rococo

[21:00:16.012] [smoldot] Smoldot v0.6.27
[21:00:16.086] [smoldot] Chain initialization complete for westend2. Name: "Westend". Genesis hash: 0xe143…423e. State root hash: 0x7e92439a94f79671f9cade9dff96a094519b9001a7432244d46ab644bb6f746f. Network identity: 12D3KooWRse9u6Z9ukP4C92YCCH2gXziNm8ThRch2owaaFh9H6D1. Chain specification or database starting at: 0xf55e…a836 (#11771600)
[21:00:16.086] [smoldot] Chain westend2 has bad blocks in its chain specification. Bad blocks are not implemented in the light client
[21:00:21.779] [sync-service-westend2] GrandPa warp sync finished to #12074567 (0x958e…1c2b)
[21:00:22.263] [runtime-westend2] Finalized block runtime ready. Spec version: 9270. Size of `:code`: 1.1 MiB.
(demo) New JSON-RPC client connected: ::ffff:127.0.0.1.
[21:00:28.937] [smoldot] Chain initialization complete for ksmcc3. Name: "Kusama". Genesis hash: 0xb0a8…dafe. State root hash: 0xb0006203c3a6e6bd2c6a17b1d4ae8ca49a31da0f4579da950b127774b44aef6b. Network identity: 12D3KooWQBvigEVJTUXd3PH76tDK1JNU4pCNnP1Hydy517Ww1vYg. Chain specification or database starting at: 0xb6ca…fe2a (#13626505)
[21:00:28.937] [smoldot] Chain ksmcc3 has bad blocks in its chain specification. Bad blocks are not implemented in the light client
[21:00:44.212] [sync-service-ksmcc3] GrandPa warp sync still in progress and taking a long time
[21:00:56.463] [sync-service-ksmcc3] GrandPa warp sync finished to #13914535 (0xc72d…52c1)
[21:00:57.327] [runtime-ksmcc3] Finalized block runtime ready. Spec version: 9260. Size of `:code`: 1.3 MiB.
[21:00:57.337] [smoldot] Parachain initialization complete for statemine. Name: "Statemine". Genesis hash: 0x4823…771a. State root hash: 0x299ac939ea4722d1fcf7f4873675040ebc83d144620c76206dbeb7468cf30cdf. Network identity: 12D3KooWNKTx2E4g65inx6nb8BX4efprbPVJyvSMQbj6PRH6uTau. Relay chain: ksmcc3 (id: 1000)
[21:01:09.080] [runtime-statemine] Successfully compiled runtime. Spec version: 2. Size of `:code`: 755 kiB.
[21:01:16.085] [smoldot] Smoldot v0.6.27. Current memory usage: 304 MiB. Average download: 421 kiB/s. Average upload: 1.3 kiB/s.
[21:02:16.088] [smoldot] Smoldot v0.6.27. Current memory usage: 305 MiB. Average download: 49.1 kiB/s. Average upload: 1.0 kiB/s.
[21:03:16.090] [smoldot] Smoldot v0.6.27. Current memory usage: 305 MiB. Average download: 43.9 kiB/s. Average upload: 1.3 kiB/s.
(demo) New JSON-RPC client connected: ::ffff:127.0.0.1.
[21:03:19.040] [smoldot] Chain initialization complete for polkadot. Name: "Polkadot". Genesis hash: 0x91b1…90c3. State root hash: 0x29d0d972cd27cbc511e9589fcb7a4506d5eb6a9e8df205f00472e5ab354a4e17. Network identity: 12D3KooWLpEHXXNU4hxRm8cEYR6KacAFCtw8tcsjmAD4VUmormUE. Chain specification or database starting at: 0xd359…ac08 (#9768333)
[21:03:34.179] [sync-service-polkadot] GrandPa warp sync still in progress and taking a long time
[21:03:46.127] [sync-service-polkadot] GrandPa warp sync finished to #11513966 (0x2185…b8a2)
[21:03:46.422] [runtime-polkadot] Finalized block runtime ready. Spec version: 9250. Size of `:code`: 1.2 MiB.
[21:03:46.444] [smoldot] Parachain initialization complete for acala. Name: "Acala". Genesis hash: 0xfc41…a64c. State root hash: 0x010c5745a5d42bcfbe0a644d5a2a4e22e2ff0fd378d48208ecfacea5b7e05a74. Network identity: 12D3KooWRUec2csLQZRMupQMokTgBcUaPasFtc4MHeVQVkoBD2bh. Relay chain: polkadot (id: 2000)
[21:03:46.445] [smoldot] Chain acala has bad blocks in its chain specification. Bad blocks are not implemented in the light client
[21:03:58.231] [runtime-acala] Successfully compiled runtime. Spec version: 2000. Size of `:code`: 1.1 MiB.
[21:04:16.095] [smoldot] Smoldot v0.6.27. Current memory usage: 558 MiB. Average download: 261 kiB/s. Average upload: 1.9 kiB/s.
[21:05:16.099] [smoldot] Smoldot v0.6.27. Current memory usage: 558 MiB. Average download: 69.3 kiB/s. Average upload: 1.8 kiB/s.
[21:06:16.101] [smoldot] Smoldot v0.6.27. Current memory usage: 558 MiB. Average download: 60.4 kiB/s. Average upload: 1.7 kiB/s.
[21:07:16.104] [smoldot] Smoldot v0.6.27. Current memory usage: 558 MiB. Average download: 60.0 kiB/s. Average upload: 1.7 kiB/s.
[21:08:16.121] [smoldot] Smoldot v0.6.27. Current memory usage: 559 MiB. Average download: 56.3 kiB/s. Average upload: 1.7 kiB/s.
[21:09:16.124] [smoldot] Smoldot v0.6.27. Current memory usage: 560 MiB. Average download: 51.2 kiB/s. Average upload: 2.4 kiB/s.
Smoldot has panicked while executing task `connections`. This is a bug in smoldot. Please open an issue at https://github.com/paritytech/smoldot/issues with the following message:
panicked at 'internal error: entered unreachable code', /Users/xiliangchen/projects/smoldot/src/libp2p/collection/single_stream.rs:384:18
(demo) Error during JSON-RPC request: Error: panicked at 'internal error: entered unreachable code', /Users/xiliangchen/projects/smoldot/src/libp2p/collection/single_stream.rs:384:18

Originally posted by @xlc in #2593 (comment)

[tomaka]

It is completely unclear to me what is happening.
By looking at the code of single_stream.rs, the only way this code path could be reached is if StartShutdown is sent twice by the coordinator (which is forbidden). However, looking at the code of the coordinator this can't possible happen as we keep track of when we've sent this message.