ESP32-S2 read_flash fails on linux (ESPTOOL-590)

[overflo23]

Operating System

Ubuntu 22.04.1 LTS

Esptool Version

3.0 - 4.4

Python Version

Python 3.10.6

Chip Description

ESP32-S2FH4 (revision v0.0) / ESP32-S2FNR2 (revision v0.0)

Device Description

ESP32-S2 mini

Hardware Configuration

no

How is Esptool Run

shell

Full Esptool Command Line that Was Run

esptool.py read_flash 0 0x400000 /tmp/foo.bin

Esptool Output

stall while reading flash at a certain page
running with --no-stub works till 0x200000 then it hangs.
reading from > 0x200000 with --no-stub returns:
A fatal error occurred: Failed to read flash block (result was 01090000: CRC or checksum was invalid)

when running in normal mode it goes like this:

esptool.foo.py  read_flash 0 0x400000 /tmp/foo.bin 
esptool.py v4.4
Found 1 serial ports
Serial port /dev/ttyACM0
Connecting...
Detecting chip type... Unsupported detection protocol, switching and trying again...
Detecting chip type... ESP32-S2
Chip is ESP32-S2FNR2 (revision v0.0)
Features: WiFi, Embedded Flash 4MB, Embedded PSRAM 2MB, ADC and temperature sensor calibration in BLK2 of efuse V1
Crystal is 40MHz
MAC: 84:f7:03:eb:86:80
Uploading stub...
Running stub...
Stub running...
933888 (22 %)

stalled, hangs right there, wont continue.
--
after posting on the adafruit circuitpython forums i was sent to a similar bug report here:
https://github.com/espressif/esptool/issues/658

this seems to be the same bug.

the weirdest thing:
if i run it on a mac with OSX, it works.
i can read0->0x400000 with no errors.

More Information

i tried different things.
flash_erase -> read -> works retuns 4 MB of 0xff as expected
installing tinyuf2-lolin_s2_mini-0.11.0 with write_flash works, starts up, opens its volume..
after that i can read back all data from 0->0x400000 without error including the tinyuf2 bootloader
than i installed circuitpython-8.0-beta6 (but this doesnt really matter)
the led flashed as the flash is written, reboot -> works as expected.

but now i cant dump the flash anymore.
it hangs at 933888

manually i can read till 937984 after that the read hangs a random locations.
937984 - 933888 = 4095
is this a pagesize?

Other Steps to Reproduce

see "more information" for steps o reproduce

I Have Read the Troubleshooting Guide

• I confirm I have read the troubleshooting guide.

[dhalbert]

I have also been testing this. The problem occurs even if the board is started up in bootloader mode on power up (hold down BOOT button and plug in), so it doesn't have to do with any odd setup the firmware may do.

I tested on an Adafruit Feather ESP32-S2, on two Windows 11 machines, Ubuntu 22.04, and an M1 Mac running macOS 13.1. On the Windows and Ubuntu machines, the stall described above occurs at the same location. On the Mac the read_flash runs to completion.

I tested with esptool versions back to 3.0 and up to 4.4 on Linux, with the same behavior.

My guess is that the contents of flash does not matter, as long as the non-all-1's part is large enough (CircuitPython is pretty big). If flash is completely erased, is the data sent back to the host pretty short (e.g., is it run-length encoded?)?

[radimkarnis]

Hi @overflo23 @dhalbert,
thank you both for the report and investigation! I will try to reproduce this and look into it.

The fact that this is OS-dependent may point to a driver issue, we'll see.

[Jason2866]

@radimkarnis Maybe related to the Flash speed increase to 240Mhz in stub loader.
Same issue as for S3?

[radimkarnis]

@Jason2866 this report states that this happens even in esptool v3 - the CPU freq increase is not implemented in that version. So 100% not related.

[trodery]

I am having the same issue on Windows (10), Linux (Ubuntu 22.04), and macOS (Ventura?). I've tried several different versions of esptool from pip going back as far as 3.3.3.

I'm using a Bus Pirate v3.6 for UART, which is in passthrough mode. I've ordered a different device to see if it still happens.

The really interesting thing is that by removing --no-stub, I can read up to exactly 174 bytes beyond offset 0x200000. Reading a size greater than or equal to 175 bytes using the stub will hang. I could recover some pieces of flash by painfully iterating over the flash 174 bytes at a time...

Using --no-stub

>esptool -p COM6 -b 115200 --no-stub read_flash 0x200000 0x200000 output.bin
esptool.py v4.2.1
Serial port COM6
Connecting....................
Detecting chip type... Unsupported detection protocol, switching and trying again...
Connecting...
Detecting chip type... ESP32
Chip is ESP32-D0WDQ6 (revision 1)
Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None
WARNING: Detected crystal freq 38.48MHz is quite different to normalized freq 40MHz. Unsupported crystal in use?
Crystal is 40MHz
MAC: XX:XX:XX:XX:XX:XX
Enabling default SPI flash mode...

A fatal error occurred: Failed to read flash block (result was 01090000: CRC or checksum was invalid)

Using stub

>esptool -p COM6 -b 115200  read_flash 0x199999 0x200 output.bin
esptool.py v4.2.1
Serial port COM6
Connecting...
Detecting chip type... Unsupported detection protocol, switching and trying again...
Connecting...
Detecting chip type... ESP32
Chip is ESP32-D0WDQ6 (revision 1)
Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None
WARNING: Detected crystal freq 38.48MHz is quite different to normalized freq 40MHz. Unsupported crystal in use?
Crystal is 40MHz
MAC: XX:XX:XX:XX:XX:XX
Stub is already running. No upload is necessary.

A fatal error occurred: Corrupt data, expected 0x1000 bytes but received 0x1ed bytes

Using stub with less than 174 bytes

>esptool -p COM6 -b 115200  read_flash 0x199999 100 output.bin
esptool.py v4.2.1
Serial port COM6
Connecting....
Detecting chip type... Unsupported detection protocol, switching and trying again...
Connecting...
Detecting chip type... ESP32
Chip is ESP32-D0WDQ6 (revision 1)
Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None
WARNING: Detected crystal freq 38.48MHz is quite different to normalized freq 40MHz. Unsupported crystal in use?
Crystal is 40MHz
MAC: XX:XX:XX:XX:XX:XX
Stub is already running. No upload is necessary.
100 (100 %)
100 (100 %)
Read 100 bytes at 0x00199999 in 0.0 seconds (18.4 kbit/s)...
Hard resetting via RTS pin...

[trodery]

I tried this again with a HiLetgo CP2102, and it worked flawlessly on Windows, Ubuntu, and macOS. It's really interesting that this problem only occurs at offsets >2MB and the default baud rate for the Bus Pirate. Per the Bus Pirate's documentation, 115200 should be stable, while any rate greater than 115200 may encounter some buffer issues. So it was a cheap fix but definitely disappointing results for such a widely revered specialized piece of hardware.

[Foschia]

I am also having trouble with the read_flash command with the 2MB boundary read error.

The operating environment in which the error is reproduced is ESP-TOOL Version 4.7.0 running at the CMD prompt on Windows 10 Pro (22H2 ).
The hardware modules reproducing the problem are the XIAO ESP32-C3 and XIAO ESP32-C6 from Seeed Studio, as well as the XIAO ESP32-S3.
Reading 1 byte from 0x1fffff in Flash memory is no problem, but reading 1 bye from 0x200000 causes an error.
A fatal error occurred: Failed to read flash block (result was 01090000: CRC or checksum was invalid)

The following log is a record of the error.
It seems to be a functional problem within ESP-TOOL.
ーーーーーーーーーーーーー
D:>esptool chip_id
esptool.py v4.7.0
Loaded custom configuration from D:\esptool.cfg
Found 3 serial ports
Serial port COM16
Connecting...
Detecting chip type... ESP32-S3
Chip is ESP32-S3 (QFN56) (revision v0.2)
Features: WiFi, BLE, Embedded PSRAM 8MB (AP_3v3)
Crystal is 40MHz
MAC: ????????????????????????????????
Uploading stub...
Running stub...
Stub running...
Warning: ESP32-S3 has no Chip ID. Reading MAC instead.
MAC: ????????????????????????????????
Hard resetting via RTS pin...

D:>
D:>esptool --no-stub --baud 115200 read_flash 0x200000 1 flash-c6.bin
esptool.py v4.7.0
Loaded custom configuration from D:\esptool.cfg
Found 3 serial ports
Serial port COM16
Connecting...
Detecting chip type... ESP32-S3
Chip is ESP32-S3 (QFN56) (revision v0.2)
Features: WiFi, BLE, Embedded PSRAM 8MB (AP_3v3)
Crystal is 40MHz
MAC: ?????????????????????????
Enabling default SPI flash mode...

A fatal error occurred: Failed to read flash block (result was 01090000: CRC or checksum was invalid)

D:>esptool --no-stub --baud 115200 read_flash 0x1fffff 1 flash-c6.bin
esptool.py v4.7.0
Loaded custom configuration from D:\esptool.cfg
Found 3 serial ports
Serial port COM16
Connecting...
Detecting chip type... ESP32-S3
Chip is ESP32-S3 (QFN56) (revision v0.2)
Features: WiFi, BLE, Embedded PSRAM 8MB (AP_3v3)
Crystal is 40MHz
MAC: ?????????????????????????
Enabling default SPI flash mode...
1 (100 %)
Read 1 bytes at 0x001fffff in 0.1 seconds (0.2 kbit/s)...
Hard resetting via RTS pin...

D:>

ーーーーーーーーーーーーーーーーーーーーーーーーーーーーーー
D:>esptool chip_id
esptool.py v4.7.0
Loaded custom configuration from D:\esptool.cfg
Found 3 serial ports
Serial port COM15
Connecting...
Detecting chip type... ESP32-C6
Chip is ESP32-C6FH4 (QFN32) (revision v0.1)
Features: WiFi 6, BT 5, IEEE802.15.4
Crystal is 40MHz
MAC: ?????????????????????????
BASE MAC: ?????????????????????????
MAC_EXT: ff:fe
Uploading stub...
Running stub...
Stub running...
Warning: ESP32-C6 has no Chip ID. Reading MAC instead.
MAC: ?????????????????????????
BASE MAC: ?????????????????????????
MAC_EXT: ff:fe
Hard resetting via RTS pin...

D:>
D:>esptool --no-stub --baud 115200 read_flash 0x1fffff 1 flash-c6.bin
esptool.py v4.7.0
Loaded custom configuration from D:\esptool.cfg
Found 3 serial ports
Serial port COM15
Connecting...
Detecting chip type... ESP32-C6
Chip is ESP32-C6FH4 (QFN32) (revision v0.1)
Features: WiFi 6, BT 5, IEEE802.15.4
Crystal is 40MHz
MAC: ?????????????????????????
BASE MAC: ?????????????????????????
MAC_EXT: ff:fe
Enabling default SPI flash mode...
1 (100 %)
Read 1 bytes at 0x001fffff in 0.1 seconds (0.1 kbit/s)...
Hard resetting via RTS pin...

D:>esptool --no-stub --baud 115200 read_flash 0x200000 1 flash-c6.bin
esptool.py v4.7.0
Loaded custom configuration from D:\esptool.cfg
Found 3 serial ports
Serial port COM15
Connecting...
Detecting chip type... ESP32-C6
Chip is ESP32-C6FH4 (QFN32) (revision v0.1)
Features: WiFi 6, BT 5, IEEE802.15.4
Crystal is 40MHz
MAC: ?????????????????????????
BASE MAC: ?????????????????????????
MAC_EXT: ff:fe
Enabling default SPI flash mode...

A fatal error occurred: Failed to read flash block (result was 01090000: CRC or checksum was invalid)

D:>

ーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーーー
D:>esptool chip_id
esptool.py v4.7.0
Loaded custom configuration from D:\esptool.cfg
Found 3 serial ports
Serial port COM10
Connecting...
Detecting chip type... ESP32-C3
Chip is ESP32-C3 (QFN32) (revision v0.4)
Features: WiFi, BLE, Embedded Flash 4MB (XMC)
Crystal is 40MHz
MAC: ?????????????????????
Uploading stub...
Running stub...
Stub running...
Warning: ESP32-C3 has no Chip ID. Reading MAC instead.
MAC: ?????????????????????????
Hard resetting via RTS pin...

D:>
D:>
D:>esptool --no-stub --baud 115200 read_flash 0x1fffff 1 flash-c6.bin
esptool.py v4.7.0
Loaded custom configuration from D:\esptool.cfg
Found 3 serial ports
Serial port COM10
Connecting...
Detecting chip type... ESP32-C3
Chip is ESP32-C3 (QFN32) (revision v0.4)
Features: WiFi, BLE, Embedded Flash 4MB (XMC)
Crystal is 40MHz
MAC: ?????????????????????????
Enabling default SPI flash mode...
1 (100 %)
Read 1 bytes at 0x001fffff in 0.0 seconds (0.2 kbit/s)...
Hard resetting via RTS pin...

D:>esptool --no-stub --baud 115200 read_flash 0x200000 1 flash-c6.bin
esptool.py v4.7.0
Loaded custom configuration from D:\esptool.cfg
Found 3 serial ports
Serial port COM10
Connecting...
Detecting chip type... ESP32-C3
Chip is ESP32-C3 (QFN32) (revision v0.4)
Features: WiFi, BLE, Embedded Flash 4MB (XMC)
Crystal is 40MHz
MAC: ?????????????????????????
Enabling default SPI flash mode...

A fatal error occurred: Failed to read flash block (result was 01090000: CRC or checksum was invalid)

D:>

ーーーーーーーーーーーーー
D:>
D:>type esptool.cfg
[esptool]
timeout = 30
max_timeout = 240
erase_write_timeout_per_mb = 40
mem_end_rom_timeout = 10
serial_write_timeout = 10
D:>