[POA][Clique] Private network stop at shanghaiTime when run more than 1 node

[anhnh131]

System information

Geth version: 1.13.4
OS & Version: Docker

Expected behaviour

Upgrade to the Shanghai, mining block & synchronizing the block on the network as normal

Actual behaviour

Stop at Shanghai time

Steps to reproduce the behaviour

Run 1 private network with config below:

• genesis config

{
    "config": {
    "chainId": 1337,
    "homesteadBlock": 0,
    "eip150Block": 0,
    "eip150Hash": "0x0000000000000000000000000000000000000000000000000000000000000000",
    "eip155Block": 0,
    "eip158Block": 0,
    "byzantiumBlock": 0,
    "constantinopleBlock": 0,
    "petersburgBlock": 0,
    "istanbulBlock": 0,
    "muirGlacierBlock": 0,
    "berlinBlock": 0,
    "londonBlock": 0,
    "shanghaiTime": 1711159980,
    "clique": {
      "period": 5,
      "epoch": 30000
    }
  }
}

• Geth run command

geth --networkid=1337 -unlock 0xXXX --password /root/files/password --allow-insecure-unlock --http.api "clique,personal,eth,net,web3,miner,admin" --http.corsdomain="*" --http --http.addr="0.0.0.0" --http.port=8546 --mine --miner.etherbase="0xXXX"  --snapshot=false --syncmode="full"

Backtrace

node1  | INFO [03-23|09:12:56.003] Successfully sealed new block            number=18 sealhash=2050e4..52b874 hash=9c52c0..b3985c elapsed=4.977s
node1  | INFO [03-23|09:12:56.003] Commit new sealing work                  number=19 sealhash=3e257f..176e67 txs=0 gas=0 fees=0 elapsed="319.584µs"
node1  | WARN [03-23|09:12:56.003] Block sealing failed                     err="signed recently, must wait for others"
node3  | INFO [03-23|09:12:56.005] Imported new chain segment               number=18 hash=9c52c0..b3985c blocks=1 txs=0 mgas=0.000 elapsed=1.430ms     mgasps=0.000 triedirty=0.00B
node3  | INFO [03-23|09:12:56.005] Commit new sealing work                  number=19 sealhash=3e257f..176e67 txs=0 gas=0 fees=0 elapsed="213.125µs"
node2  | INFO [03-23|09:12:56.508] Imported new chain segment               number=18 hash=9c52c0..b3985c blocks=1 txs=0 mgas=0.000 elapsed=2.371ms     mgasps=0.000 triedirty=0.00B
node2  | INFO [03-23|09:12:56.508] Commit new sealing work                  number=19 sealhash=df675b..ef94ab txs=0 gas=0 fees=0 elapsed="69.792µs"
node2  | INFO [03-23|09:13:01.015] Successfully sealed new block            number=19 sealhash=df675b..ef94ab hash=47249f..c861da elapsed=4.506s
node1  | INFO [03-23|09:13:01.017] Looking for peers                        peercount=1 tried=0 static=0
node2  | INFO [03-23|09:13:01.017] Commit new sealing work                  number=20 sealhash=fe8dc0..2c9ea4 txs=0 gas=0 fees=0 elapsed=1.075ms
node2  | WARN [03-23|09:13:01.017] Block sealing failed                     err="signed recently, must wait for others"
node3  | INFO [03-23|09:13:01.112] Successfully sealed new block            number=19 sealhash=3e257f..176e67 hash=86d843..624d9a elapsed=5.107s
node3  | INFO [03-23|09:13:01.113] Commit new sealing work                  number=20 sealhash=0624f0..b32e12 txs=0 gas=0 fees=0 elapsed="499.166µs"
node3  | WARN [03-23|09:13:01.113] Block sealing failed                     err="signed recently, must wait for others"
node2  | INFO [03-23|09:13:01.159] New local node record                    seq=1,711,159,891,225 id=6dbca6a593ee5a43 ip=127.0.0.1 udp=30303 tcp=30303
node3  | INFO [03-23|09:13:01.518] New local node record                    seq=1,711,159,891,199 id=b13d9b8cfdab8a24 ip=127.0.0.1 udp=30303 tcp=30303
node3  | INFO [03-23|09:13:02.247] Looking for peers                        peercount=0 tried=160 static=0
node2  | INFO [03-23|09:13:02.983] Looking for peers                        peercount=0 tried=162 static=0
node3  | INFO [03-23|09:13:12.400] Looking for peers                        peercount=0 tried=170 static=0
node1  | INFO [03-23|09:13:12.982] Looking for peers                        peercount=1 tried=0 static=0
node1  | INFO [03-23|09:13:12.982] Block synchronisation started
node1  | INFO [03-23|09:13:12.983] Mining aborted due to sync
node1  | WARN [03-23|09:13:12.986] Failed to load old bad blocks            error="pebble: not found"
node1  | ERROR[03-23|09:13:12.989] 
node1  | ########## BAD BLOCK #########
node1  | Block: 19 (0x47249fa199de2dbfada859ce0776d3ec30f5d6a9bfc3a4cd1903c08f66c861da)
node1  | Error: clique does not support shanghai fork
node1  | Platform: geth (devel) go1.21.7 arm64 linux
node1  | VCS: 2bd6bd01-
node1  | Chain config: &params.ChainConfig{ChainID:1337, HomesteadBlock:0, DAOForkBlock:<nil>, DAOForkSupport:false, EIP150Block:0, EIP155Block:0, EIP158Block:0, ByzantiumBlock:0, ConstantinopleBlock:0, PetersburgBlock:0, IstanbulBlock:0, MuirGlacierBlock:0, BerlinBlock:0, LondonBlock:0, ArrowGlacierBlock:<nil>, GrayGlacierBlock:<nil>, MergeNetsplitBlock:<nil>, ShanghaiTime:(*uint64)(0x4000672c18), CancunTime:(*uint64)(nil), PragueTime:(*uint64)(nil), VerkleTime:(*uint64)(nil), TerminalTotalDifficulty:<nil>, TerminalTotalDifficultyPassed:false, Ethash:(*params.EthashConfig)(nil), Clique:(*params.CliqueConfig)(0x4000672c30)}
node1  | Receipts: 
node2  | INFO [03-23|09:13:12.990] Looking for peers                        peercount=0 tried=139 static=0
node1  | ##############################
node1  | 
node1  | ERROR[03-23|09:13:12.989] Failed to retrieve beacon bounds for bad block reporting err="beacon sync not yet started"
node1  | WARN [03-23|09:13:12.989] Synchronisation failed, dropping peer    peer=6dbca6a593ee5a4364b74f9e3f8f02a218c4fac639a1f7e873d980530ffb14bc err="retrieved hash chain is invalid: clique does not support shanghai fork"
node1  | INFO [03-23|09:13:12.990] Commit new sealing work                  number=19 sealhash=4d2c7d..7125a8 txs=0 gas=0 fees=0 elapsed="211.208µs"
node1  | WARN [03-23|09:13:12.990] Block sealing failed                     err="signed recently, must wait for others"
node3  | INFO [03-23|09:13:22.485] Looking for peers                        peercount=0 tried=117 static=0
node2  | INFO [03-23|09:13:23.087] Looking for peers                        peercount=0 tried=143 static=0
node1  | INFO [03-23|09:13:23.776] Looking for peers                        peercount=1 tried=0 static=0
node1  | INFO [03-23|09:13:23.778] Mining aborted due to sync
node1  | ERROR[03-23|09:13:23.783] 
node1  | ########## BAD BLOCK #########
node1  | Block: 19 (0x86d8435f974a665ff0a2d4e9defb577d50e6a007cd7d53afdf3e0a3027624d9a)
node1  | Error: clique does not support shanghai fork
node1  | Platform: geth (devel) go1.21.7 arm64 linux
node1  | VCS: 2bd6bd01-
node1  | Chain config: &params.ChainConfig{ChainID:1337, HomesteadBlock:0, DAOForkBlock:<nil>, DAOForkSupport:false, EIP150Block:0, EIP155Block:0, EIP158Block:0, ByzantiumBlock:0, ConstantinopleBlock:0, PetersburgBlock:0, IstanbulBlock:0, MuirGlacierBlock:0, BerlinBlock:0, LondonBlock:0, ArrowGlacierBlock:<nil>, GrayGlacierBlock:<nil>, MergeNetsplitBlock:<nil>, ShanghaiTime:(*uint64)(0x4000672c18), CancunTime:(*uint64)(nil), PragueTime:(*uint64)(nil), VerkleTime:(*uint64)(nil), TerminalTotalDifficulty:<nil>, TerminalTotalDifficultyPassed:false, Ethash:(*params.EthashConfig)(nil), Clique:(*params.CliqueConfig)(0x4000672c30)}
node1  | Receipts: 
node1  | ##############################
node1  | 
node1  | ERROR[03-23|09:13:23.783] Failed to retrieve beacon bounds for bad block reporting err="beacon sync not yet started"
node1  | WARN [03-23|09:13:23.783] Synchronisation failed, dropping peer    peer=b13d9b8cfdab8a242116cb6b2de687adfd8fe872052321a2839d47ea1fc1193a err="retrieved hash chain is invalid: clique does not support shanghai fork"

I think this problem comes from #26484 (comment).
Is Shanghai not supported on Clique? How can I use the new features on Clique?

[karalabe]

You need to transition your network over to PoS. Shanghai depends on features that need a beacon chain.