Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compact WS data #1387

Merged
merged 9 commits into from
Feb 21, 2019
Merged

Compact WS data #1387

merged 9 commits into from
Feb 21, 2019

Conversation

mcspr
Copy link
Collaborator

@mcspr mcspr commented Dec 1, 2018

For #1367

Do not duplicate keys for relay, sensor, schedule, rfbridge, thingspeak & domoticz ws objects
Convert bool -> int (3 and 4 char shorter!) for status arrays
Minor edit _magnitudes.size() -> magnitudeCount()
Reuse single JsonObject for all callbacks

code/html/custom.js Show resolved Hide resolved
code/espurna/sensor.ino Show resolved Hide resolved
code/espurna/sensor.ino Show resolved Hide resolved
@mcspr
Copy link
Collaborator Author

mcspr commented Dec 11, 2018

I was distracted while doing the first merge.

Quick testing 4d5c0d7 with Windows / Chrome and 16 switch rfbridge. Compared to this, it takes full 1-2 seconds for all of the rfb messages to come through.
*edit: * In theory tcp layer would pack the data without nodelay set, but this does not happen somehow (or it is set somewhere). Need to check espasynctcp again...
edit2: tcp part is right here. next thing to try is to make websocket code write but not send immediately. ws is framed so browser will figure it out. Will update #1300 if this works

Also, notable comment from here: Links2004/arduinoWebSockets#187 (comment)

the 200ms from windows can be workaround-ed when you send a small answer at the moment when you get first message in the browser.
when you send data back from windows to the ESP windows will send the ACK immediately ;)

(which was referenced in #1300 by replacing module init code to send messages to the esp asking for each config part)

@mcspr
Copy link
Collaborator Author

mcspr commented Jan 27, 2019

Will force-push dev...mcspr:ws/optimize here to avoid doing merge + reduced buffer growth when using ws

Tangentially, this also lead up to trying to fix ws logging for the latest ESPAsyncWebServer by manually queueing messages. I wouldn't call it finished, as that approach only matters for ws. Telnet blows up with large payloads when using buffer and probably needs queuing like asyncwebsocket does internally. And it does not handle special case for '\r' char yet (OTA updates progress will fill the buffer with garbage)

ws/optimize branch now has an actually working buffered logger, but that still needs update to avoid duplicating messages when timestamping.

* send all ws config at once on start
* reduce ws json overhead by creating message buffer manually
* use k:[values] instead of k1:value1, k2:value2 etc. for lists
This was referenced Feb 4, 2019
mcspr added 3 commits February 7, 2019 17:12
* Simply print timestamp for supported destinations
* Update ws destination to support 'prefix' sub-key
* Avoid void casting in telnet module
@mcspr
Copy link
Collaborator Author

mcspr commented Feb 7, 2019

I would argue that logging changes are related, but they can be split into a separate pr with ESPAsyncTCP / ESPAsyncWebServer versions update and general clean-up (like '\r' chars sent to ws when doing OTA and changes from ws/optimize to use application-level buffer for all the logging)

Also, missing rfm69 wsOnSend.

@mcspr
Copy link
Collaborator Author

mcspr commented Feb 12, 2019

@xoseperez Generally, if you do not like approach at packing everything inside single ws message, there are two solutions for the ack problem at hand:

  • force webui to keep state of which cfg it loaded, sending something like {action: "load_cfg", {module: "relay"}} per each module and receive response from the esp. that way, only single module data is in the memory at once and queue is not kept filled. each time client responds with "action", ack is piggybacked inside, so the last message is removed from the queue whenever app yields into the system
  • fork espasyncwebserver 🤷‍♂️ and introduce additional user callback that is called when tcp ACK is really received (and not guess with delay()). this will also help the solution from Print each HA config entry separately #1535 by making printer decisively send the data

@xoseperez xoseperez merged commit 27dcd15 into xoseperez:dev Feb 21, 2019
@mcspr mcspr deleted the compact-ws branch February 21, 2019 22:29
@mcspr mcspr mentioned this pull request Oct 17, 2019
34 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants