Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Can't upgrade Zigbee Firmware #1042

Closed
dsimoes opened this issue Apr 15, 2023 · 16 comments
Closed

Can't upgrade Zigbee Firmware #1042

dsimoes opened this issue Apr 15, 2023 · 16 comments
Labels
bug Something isn't working

Comments

@dsimoes
Copy link

dsimoes commented Apr 15, 2023

I've setup 3.2.0 integration with ZHA mode (gateway ZNDMWG03LM Firmware: 1.5.0_0102)

When running the command Zigbee Flash EZSP (custom) It fails:

2023-04-15 19:24:11,041 192.168.3.83 [FWUP] Target zigbee firmware v6.7.10.0
2023-04-15 19:24:12,282 192.168.3.83 [FWUP] Can't update firmware
Traceback (most recent call last):
File "/config/custom_components/xiaomi_gateway3/core/ezsp.py", line 52, in update_zigbee_firmware
cur_fw = await read_firmware(host)
File "/config/custom_components/xiaomi_gateway3/core/ezsp.py", line 104, in read_firmware
await asyncio.wait_for(ezsp._probe(), timeout=10)
File "/usr/local/lib/python3.10/asyncio/tasks.py", line 445, in wait_for
return fut.result()
File "/usr/local/lib/python3.10/site-packages/bellows/ezsp/init.py", line 84, in _probe
await self.connect(use_thread=False)
File "/usr/local/lib/python3.10/site-packages/bellows/ezsp/init.py", line 126, in connect
self._gw = await bellows.uart.connect(self._config, self, use_thread=use_thread)
File "/usr/local/lib/python3.10/site-packages/bellows/uart.py", line 414, in connect
protocol, _ = await _connect(config, application)
File "/usr/local/lib/python3.10/site-packages/bellows/uart.py", line 385, in _connect
transport, protocol = await zigpy.serial.create_serial_connection(
File "/usr/local/lib/python3.10/site-packages/zigpy/serial.py", line 31, in create_serial_connection
transport, protocol = await loop.create_connection(
File "/usr/local/lib/python3.10/asyncio/base_events.py", line 1076, in create_connection
raise exceptions[0]
File "/usr/local/lib/python3.10/asyncio/base_events.py", line 1060, in create_connection
sock = await self._connect_sock(
File "/usr/local/lib/python3.10/asyncio/base_events.py", line 969, in _connect_sock
await self.sock_connect(sock, address)
File "/usr/local/lib/python3.10/asyncio/selector_events.py", line 501, in sock_connect
return await fut
File "/usr/local/lib/python3.10/asyncio/selector_events.py", line 541, in _sock_connect_cb
raise OSError(err, f'Connect call failed {address}')
ConnectionRefusedError: [Errno 111] Connect call failed ('192.168.3.83', 8889)

Looking at the device logs, I see some errors which I assume are related:

Apr 15 19:24:10 rlxlinux auth.info login[2400]: root login on 'ttyp0'
Apr 15 19:24:10 rlxlinux user.info zigbee_gw[1364]: onSocketClosed: /tmp/miio_agent.socket closed by peer (tcp_connection.cpp:77)
Apr 15 19:24:10 rlxlinux user.info zigbee_gw[1364]: onTcpDisconnected: AgentConnection disconnected! (agent_connection.cpp:113)
Apr 15 19:24:10 rlxlinux user.info zigbee_gw[1364]: enterConnectingState: connect fail, errno:146, Connection refused (tcp_connector.cpp:200)
Apr 15 19:24:10 rlxlinux user.info zigbee_gw[1364]: enterReconnectDelayState: enter reconnect delay state, delay: 1 (tcp_connector.cpp:249)
Apr 15 19:24:10 rlxlinux user.notice : ^[[1;32m[D] : check wifi status 8
Apr 15 19:24:10 rlxlinux user.notice : ^[[0m [E][agent_client] client_message_callback 835: sock
Apr 15 19:24:10 rlxlinux user.notice : [E][agent_client] client_connect 894: connect
Apr 15 19:24:10 rlxlinux user.notice : ^[[0m^[[32m Info miio_agent_socket_callback: Epoll event error: Bad file descriptor
Apr 15 19:24:10 rlxlinux user.notice : ^[[0m^[[31m Error miio_agent_socket_callback: miio_agent disconnected!
Apr 15 19:24:10 rlxlinux user.notice : 2023-04-15 19:24:10 info miio::Agent: Lost connection with miio_agent
Apr 15 19:24:10 rlxlinux user.notice : 2023-04-15 19:24:10 warn miio::Agent: connect(): Connection refused
Apr 15 19:24:11 rlxlinux user.info zigbee_gw[1364]: exitReconnectDelayState: exit reconnect delay state (tcp_connector.cpp:265)
Apr 15 19:24:11 rlxlinux user.info zigbee_gw[1364]: enterConnectingState: connect fail, errno:146, Connection refused (tcp_connector.cpp:200)
Apr 15 19:24:11 rlxlinux user.info zigbee_gw[1364]: enterReconnectDelayState: enter reconnect delay state, delay: 1 (tcp_connector.cpp:249)
Apr 15 19:24:12 rlxlinux user.notice : 2023-04-15 19:24:12 warn miio::Agent: connect(): No such file or directory
Apr 15 19:24:12 rlxlinux user.info zigbee_gw[1364]: exitReconnectDelayState: exit reconnect delay state (tcp_connector.cpp:265)
Apr 15 19:24:12 rlxlinux user.info zigbee_gw[1364]: enterConnectingState: connect fail, errno:2, No such file or directory (tcp_connector.cpp:200)
Apr 15 19:24:12 rlxlinux user.info zigbee_gw[1364]: enterReconnectDelayState: enter reconnect delay state, delay: 1 (tcp_connector.cpp:249)
Apr 15 19:24:13 rlxlinux user.info zigbee_gw[1364]: exitReconnectDelayState: exit reconnect delay state (tcp_connector.cpp:265)
Apr 15 19:24:13 rlxlinux user.info zigbee_gw[1364]: exitConnectingState: exit connecting state (tcp_connector.cpp:230)
Apr 15 19:24:13 rlxlinux user.info zigbee_gw[1364]: onTcpConnected: AgentConnection connected. (agent_connection.cpp:131)
Apr 15 19:24:13 rlxlinux user.info zigbee_gw[1364]: operator(): connected to agent server. (cloud_miot.cpp:71)
@RedFox-Fr
Copy link

It was very complicated for me to update the Zigbee firmware because Lumi_Z3GatewayHost_MQTT process interfers with the update.
But when Lumi_Z3GatewayHost_MQTT is killed and the process that restart Lumi_Z3GatewayHost_MQTT was also killed (app_monitor.sh and openmiio_agent), the firmware update process don´t works either.

Here my poor solution...

Open 2 telnet sessions on the gateway (ZNDMWG03LM)
On first session:

# killall app_monitor.sh
# killall openmiio_agent
# killall Lumi_Z3GatewayHost_MQTT

On second session:

# wget -O /tmp/zigbee_flash_mgl03.zip "http://master.dl.sourceforge.net/project/mgl03/zigbee/zigbee_flash.zip?viasf=1" && unzip -o /tmp/zigbee_flash_mgl03.zip -d /tmp && cd /tmp && sh /tmp/mgl3_zigbee_flash.sh

Choose the fimware version 6.7.8
The update seems to freeze after displaying "Give your local XMODEM receive command now"
Return to first session and launch

# Lumi_Z3GatewayHost_MQTT -n 1 -b 115200 -p /dev/ttyS2 -d /data/silicon_zigbee_host/

I don't understand why, but this allow upgrade in the second session to start.
If the upgrade fails, ie you don't see "Transfert complete... Detected EZSP v8"
just retry

# cd /tmp
# sh /tmp/mgl3_zigbee_flash.sh

and relaunch Lumi_Z3GatewayHost_MQTT in the first session with the "right timing" :/
When update to 6.7.8 was sucessfull, you can use the same method to update to 6.7.10.

For me, with this method, update to 6.7.8 was successfull at second attempt, and update to 6.7.10 at first attempt...

@RedFox-Fr
Copy link

I discovered this "method" because, after upgrading the main Realtek firmware, upgrading the Zigbee firmware failed randomly at differents steps but without going to the end of the upload. Then I see in htop that Lumi_Z3* process constantly respawned while eating nearly 100% CPU...
After killed all processes involved in the launch of Lumi_Z3, sx was unable to start the upload of the new firmware, then, just to see, I launch Lumi_Z3 manually and sx start to upload...
When Lumi_Z3 quit itself, upload of the firmware can be interrupted or not depending of the "timing"...

@szymucha94
Copy link

It was very complicated for me to update the Zigbee firmware because Lumi_Z3GatewayHost_MQTT process interfers with the update. But when Lumi_Z3GatewayHost_MQTT is killed and the process that restart Lumi_Z3GatewayHost_MQTT was also killed (app_monitor.sh and openmiio_agent), the firmware update process don´t works either.

Here my poor solution...

Open 2 telnet sessions on the gateway (ZNDMWG03LM) On first session:

# killall app_monitor.sh
# killall openmiio_agent
# killall Lumi_Z3GatewayHost_MQTT

On second session:

# wget -O /tmp/zigbee_flash_mgl03.zip "http://master.dl.sourceforge.net/project/mgl03/zigbee/zigbee_flash.zip?viasf=1" && unzip -o /tmp/zigbee_flash_mgl03.zip -d /tmp && cd /tmp && sh /tmp/mgl3_zigbee_flash.sh

Choose the fimware version 6.7.8 The update seems to freeze after displaying "Give your local XMODEM receive command now" Return to first session and launch

# Lumi_Z3GatewayHost_MQTT -n 1 -b 115200 -p /dev/ttyS2 -d /data/silicon_zigbee_host/

I don't understand why, but this allow upgrade in the second session to start. If the upgrade fails, ie you don't see "Transfert complete... Detected EZSP v8" just retry

# cd /tmp
# sh /tmp/mgl3_zigbee_flash.sh

and relaunch Lumi_Z3GatewayHost_MQTT in the first session with the "right timing" :/ When update to 6.7.8 was sucessfull, you can use the same method to update to 6.7.10.

For me, with this method, update to 6.7.8 was successfull at second attempt, and update to 6.7.10 at first attempt...

doesn't work for me. Z3GatewayHost throws assertion after attempting to "reset NCP"

@dsimoes
Copy link
Author

dsimoes commented Apr 18, 2023

Hi,

Well, I wanted to try to get the flash via Integration working and avoid manually doing It on the shell.
I looked at the current code and noticed there is a sleep(1) after running the necessary commands before the flash.. Debug logs shows the process would stop after this.

So I assumed this delay might not be sufficient. I increased to 15 secs just for testing.
Also noticed the commands are repeated again here and here. so I also added the same sleep(15) to give some time for the execution of the command before continuing.

This worked!
Of course, 15secs is too much, I just wanted to give some extra time to check via shell the processes. So i would increase that 1sec wait time a bit more time and perform the sleep in all 3 spots. But I'll let @AlexxIT pitch in on this!

@RedFox-Fr
Copy link

RedFox-Fr commented Apr 18, 2023 •
edited
Loading

Heyy,

The firmware used by ezsp.py at line 70 is
mgl03_ncp_6_7_10_b38400_sw.gbl
but the firmware in the zip file is
ncp-uart-sw_mgl03_6_7_10_z2m.gbl with same md5sum than mgl03_ncp_6_7_10_b115200_sw.gbl

What is the better firmware? It's not the faster one?

@kkmaslowski
Copy link

@szymucha94 I tried your method without success. I see error about missing /data/sx binary:

You selected: ncp-uart-sw_mgl03_6_7_8_z2m.gbl

Preparing
Killing default zigbee software
killall: socat: no process killed
killall: daemon_app.sh: no process killed
killall: Lumi_Z3GatewayHost_MQTT: no process killed
Done

Press Ctrl+C to cancel in 5 seconds!

Detected EZSP v7
Rebooting to bootloader
Sending upload command
/tmp/mgl3_zigbee_flash.sh: line 185: /data/sx: not found
Sending restart command
Detected EZSP v7

Flashing completed!

@szymucha94
Copy link

/tmp/mgl3_zigbee_flash.sh: line 185: /data/sx: not found

@kkmaslowski that one seems pretty obvious :)
just download sx manually and place it inside /data. Remember to "chmod +x /data/sx". You can find the command for download inside mgl3_zigbee_flash.sh - just open it with nano or cat.

@kkmaslowski
Copy link

kkmaslowski commented May 2, 2023 •
edited
Loading

I can't revert to original firmware, flashing stops, please help:

Xiaomi Gateway 3 (mgl3) Zigbee firmware flashing script

Found firmware files:
1: ncp-uart-sw_mgl03_6_6_2_stock.gbl
2: ncp-uart-sw_mgl03_6_7_10_z2m.gbl
3: ncp-uart-sw_mgl03_6_7_8_z2m.gbl
Select firmware: 1
You selected: ncp-uart-sw_mgl03_6_6_2_stock.gbl

Preparing
Killing default zigbee software
killall: socat: no process killed
killall: daemon_app.sh: no process killed
Done

Press Ctrl+C to cancel in 5 seconds!

Detected EZSP v7
Rebooting to bootloader
Sending upload command
Sending ncp-uart-sw_mgl03_6_6_2_stock.gbl, 1443 blocks: Give your local XMODEM receive command now.
Xmodem sectors/kbytes sent:   0/ 0kRetry 0: Got 65 for sector ACK
Retry 0: NAK on sector
Xmodem sectors/kbytes sent:  16/ 2kRetry 0: Got 0d for sector ACK

@mihaiaka
Copy link

I can't revert to original firmware, flashing stops, please help:

Xiaomi Gateway 3 (mgl3) Zigbee firmware flashing script

Found firmware files:
1: ncp-uart-sw_mgl03_6_6_2_stock.gbl
2: ncp-uart-sw_mgl03_6_7_10_z2m.gbl
3: ncp-uart-sw_mgl03_6_7_8_z2m.gbl
Select firmware: 1
You selected: ncp-uart-sw_mgl03_6_6_2_stock.gbl

Preparing
Killing default zigbee software
killall: socat: no process killed
killall: daemon_app.sh: no process killed
Done

Press Ctrl+C to cancel in 5 seconds!

Detected EZSP v7
Rebooting to bootloader
Sending upload command
Sending ncp-uart-sw_mgl03_6_6_2_stock.gbl, 1443 blocks: Give your local XMODEM receive command now.
Xmodem sectors/kbytes sent:   0/ 0kRetry 0: Got 65 for sector ACK
Retry 0: NAK on sector
Xmodem sectors/kbytes sent:  16/ 2kRetry 0: Got 0d for sector ACK

Hello gents,

Did you find any resolution for this? I'm also in the same situation.
Did Alex have any opinion on it yet?

@kkmaslowski
Copy link

@mihaiaka
AlexxIT fixed zigbee firmware remotely on my gateway. We tested that flashing zigbee firmware from HA GUI works normally on HassWP (https://github.com/AlexxIT/HassWP - HA with AlexxIT addon running on windows) and don't work on my HA in docker. He also fixed flashing zigbee firmware when current version is unknown (like mine above - after uncomplete flash) in commit: ebae56f

I think you should try to do the following steps:

@mihaiaka
Copy link

Thank you @kkmaslowski . On a Windows 10 laptop, I've downloaded the latest HassWP. I've replaced the ezsp.py file. I've turned on HA via the hass.cmd file. made a test account. When i try to add the GW3 integration i get an error and the cmd window logs this:

2023-05-15 10:05:21,780.780 ERROR (MainThread) [homeassistant.config_entries] Error occurred loading configuration flow for integration xiaomi_gateway3: cannot import name 'OPENMIIO_CMD' from 'custom_components.xiaomi_gateway3.core.shell.base' (C:\Users\User\Downloads\HassWP_2023.3.6\config\custom_components\xiaomi_gateway3\core\shell\base.py)

If i try to do a similar thing and copy the base.py from https://github.com/AlexxIT/XiaomiGateway3/blob/ebae56f907b6e446408d4739d94e825f6c25d496/custom_components/xiaomi_gateway3/core/shell/base.py the error changes, but still doesn't load the integration, below is the log from the cmd window:

2023-05-15 10:22:20,330.330 ERROR (MainThread) [homeassistant.config_entries] Error occurred loading configuration flow for integration xiaomi_gateway3: cannot import name 'URL_ARM' from 'custom_components.xiaomi_gateway3.core.shell.base' (C:\Users\User\Downloads\HassWP_2023.3.6\config\custom_components\xiaomi_gateway3\core\shell\base.py)

Any idea what to try next @AlexxIT ?

@mihaiaka
Copy link

Hi,

Any recommendations?
Thank you.

@kkmaslowski
Copy link

@mihaiaka maybe try from the beggining, but do not replace any files manually and add gateway to see if it's working. Next try to replace file, restart HA, test if it's ok and try to flash from gui.

@kossnikita
Copy link

@mihaiaka AlexxIT fixed zigbee firmware remotely on my gateway. We tested that flashing zigbee firmware from HA GUI works normally on HassWP (https://github.com/AlexxIT/HassWP - HA with AlexxIT addon running on windows) and don't work on my HA in docker. He also fixed flashing zigbee firmware when current version is unknown (like mine above - after uncomplete flash) in commit: ebae56f

I think you should try to do the following steps:

Just got in WP

[custom_components.xiaomi_gateway3.core.ezsp] 192.168.0.42 [FWUP] Not in flash mode

and on telnet

Preparing
Killing default zigbee software
killall: socat: no process killed
killall: daemon_app.sh: no process killed
killall: Lumi_Z3GatewayHost_MQTT: no process killed
Done

Press Ctrl+C to cancel in 5 seconds!

Detected EZSP v7
Rebooting to bootloader
Sending upload command
Sending ncp-uart-sw_mgl03_6_7_10_z2m.gbl, 1548 blocks: Give your local XMODEM receive command now.
Xmodem sectors/kbytes sent:   0/ 0kRetry 0: Got 65 for sector ACK
Retry 0: NAK on sector
Xmodem sectors/kbytes sent:  65/ 8kRetry 0: Got 42 for sector ACK

No matter with fix ezsp.py or not

@kkmaslowski
Copy link

@kossnikita @mihaiaka maybe try to contact Alexx directly on Telegram group https://t.me/xiaomi_gw_v3_hack

@AlexxIT AlexxIT added the bug Something isn't working label Jun 5, 2023
@AlexxIT
Copy link
Owner

AlexxIT commented Jun 12, 2023

https://github.com/AlexxIT/XiaomiGateway3/releases/tag/v3.3.0

@AlexxIT AlexxIT closed this as completed Jun 12, 2023
@ztdan4ik ztdan4ik mentioned this issue Oct 29, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@AlexxIT @kkmaslowski @dsimoes @kossnikita @mihaiaka @RedFox-Fr @szymucha94