Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement URC Handling. (IDFGH-8812) #180

Closed
diplfranzhoepfinger opened this issue Nov 24, 2022 · 53 comments · Fixed by #620
Closed

Implement URC Handling. (IDFGH-8812) #180

diplfranzhoepfinger opened this issue Nov 24, 2022 · 53 comments · Fixed by #620

Comments

@diplfranzhoepfinger
Copy link
Contributor

diplfranzhoepfinger commented Nov 24, 2022

also discussed in:

#156

#168

#179

@diplfranzhoepfinger
Copy link
Contributor Author

@david-cermak

@github-actions github-actions bot changed the title Implement URC Handling. Implement URC Handling. (IDFGH-8812) Nov 24, 2022
@franz-ms-muc
Copy link
Contributor

franz-ms-muc commented Dec 1, 2022

also in the Modem Console URC might be helpful:
see here:
image

issuing a extra cmd AT does not really look smart.

@david-cermak
Copy link
Collaborator

This could be implemented the same way as #156, i.e. installing a permanent callback which would check for a global set of URC first and only after that for replies to requests. So we could still create a DCE class that's command-able (=could add commands) and at the same time handles the URCs.

I'll prepare a simple demo.

@franz-ms-muc
Copy link
Contributor

franz-ms-muc commented Dec 1, 2022 via email

@franz-ms-muc
Copy link
Contributor

franz-ms-muc commented Dec 13, 2022

I'll prepare a simple demo.

any Progress @david-cermak ?

@david-cermak
Copy link
Collaborator

Added as c61fe1f

it's very simple and WIP. It only adds a user callback that could handle all received data. (needs to be removed before switching modes and applied again after)

The other option is to reuse CMUX mode for that, we already have two virtual terminals, so we could make another one for handling URC. Unfortunately the DTE/CMUX classes are not fully designed to support variable number of virtual terminals, but that would be an interesting addition.

@diplfranzhoepfinger
Copy link
Contributor Author

i will test it.
have a Good Test-case here:

a SIMCOM A7672E-FASE

we can send here a AT+CGNSSPWR=1 command,
and it will answer with a URC of +CGNSSPWR: READY!

will test now...

@diplfranzhoepfinger
Copy link
Contributor Author

your URC Handler works:

image

@franz-ms-muc
Copy link
Contributor

Hi,

Question: (from Vasil Nikolov)

I understand the URC implementation following way:

  1. You can switch on/off the reaction on URC
    a. This is nice
  2. ESP Modem receives all of the URC
    a. For me it is important to understand how modem implementation differentiates between URCs and a normal response to AT commands
    i. There are commands which require longer time until a response is received.
    b. Can we receive URCs while we are waiting for a response to a certain AT Command?
    i. If yes then how can we be sure that responses to the AT Commands are forwarded to the URC handler instead to the command handler
  3. In our user code we shall define one URC handler with a switch case based on the data and length of received URC
    a. This is a nice approach, we have full control over all URCs we are interested in

@VasilNikolovRilabs
Copy link

@david-cermak could you please provide some feedback about the last questions from Franz?
We would like to really assure that the URC implementation wont impact the normal AT command request/response handling

Thank you!

@david-cermak
Copy link
Collaborator

Hi Franz and Vasil,

sorry for not responding earlier.
The example of injecting a URC callback doesn't really change anything in the esp_modem layers, it just overrides the default command implementation using these two methods:

  1. Traditional method for sending commands makes the class command-able, so it could be employed by the command library to run all/defined AT commands:

command_result Shiny::DCE::command(const std::string &cmd, got_line_cb got_line, uint32_t time_ms, const char separator)

  1. The data callback that processes the replies

command_result Shiny::DCE::handle_data(uint8_t *data, size_t len)

This way, we're able to

  • first call the user urc-callback (and after that)
  • continue with processing the replies by the command library

a. For me it is important to understand how modem implementation differentiates between URCs and a normal response to AT commands

we process the user callback first

if (handle_urc) {
handle_urc(data, len);
}

...and then let the command library to parse the same data.

if (handle_cmd) {
auto ret = handle_cmd(data, len);
if (ret == esp_modem::command_result::TIMEOUT) {
return command_result::TIMEOUT;
}
if (ret == esp_modem::command_result::OK) {
signal.set(1);
}
if (ret == esp_modem::command_result::FAIL) {
signal.set(2);
}
}

i. There are commands which require longer time until a response is received.

You need to be aware that the urc handler would still received all the replies, including the ones that belong to unrelated commands issued by the command library (by user)

b. Can we receive URCs while we are waiting for a response to a certain AT Command?

Yes, the urc-handler gets always called (if enabled). The other part (lib-command processing) is called only if an AT command is in progress.

i. If yes then how can we be sure that responses to the AT Commands are forwarded to the URC handler instead to the command handler

This works like a fork (tee command in bash :-), we pass the same data to two handlers

The only trouble might be the time, physically spent on processing the urc responses, since it actually delays processing the simultaneously active AT command. This wouldn't delay data reception, but might influence timeouts of the defined commands.
(for example, if we're sending AT+CGC\r with timeout of 500ms and spend 600ms on urc processing, then it would fail, but should still work if we spend like 450ms on urc-processing, as the actual reply would simply queue so the data handler should make it in time)

@VasilNikolovRilabs
Copy link

VasilNikolovRilabs commented Mar 2, 2023 via email

@franz-ms-muc
Copy link
Contributor

https://gist.github.com/franz-ms-muc/b28a70d42a86bb2ed292e638a11fa152

i test the URC HAndler now in the CMUX Sample.
it is installed in this Line:
https://gist.github.com/franz-ms-muc/b28a70d42a86bb2ed292e638a11fa152#file-gistfile1-txt-L257

but then never called. However, as i have extended Logging active, i see the URCs are coming in.

however when i was testing it with the Console Sample it was working.

think there is something to fix still.

@franz-ms-muc
Copy link
Contributor

@franz-ms-muc
Copy link
Contributor

@david-cermak did you look into this ?

@david-cermak
Copy link
Collaborator

@franz-ms-muc Could you please also share the code where you install the handler?
Note that the change has already been merged to master (and release as modem-v1.0.0), so you should be able to play with URC directly in the console example. Does it work for you in the default example?

@franz-ms-muc
Copy link
Contributor

franz-ms-muc commented Mar 21, 2023 via email

@david-cermak
Copy link
Collaborator

Could you please close this issue if it work for you? Or comment and share the details if URC handling doesn't work as expected.

@diplfranzhoepfinger
Copy link
Contributor Author

hi,

i tested like this:

use this Commit:
diplfranzhoepfinger@0a33ce5

and the Outcome is this Log:
https://gist.github.com/diplfranzhoepfinger/73da3e69cbee3edbaaa61def2f50eba1

+CGNSSPWR: READY!

is a URC.
it is not handeled by the URC Handler.

@diplfranzhoepfinger
Copy link
Contributor Author

making another Try:

https://gist.github.com/diplfranzhoepfinger/6a3062d8861535f2b3927cdc6e5d299f

there the URC Handler works.

it seems the URC HAndler only works on "cmd xx"
and not on "build in" commands.

pls look into this.
Thanks !
Franz

@diplfranzhoepfinger
Copy link
Contributor Author

@david-cermak can you have a look into this ?

Thanks!

@diplfranzhoepfinger
Copy link
Contributor Author

@diplfranzhoepfinger
Copy link
Contributor Author

@diplfranzhoepfinger
Copy link
Contributor Author

@diplfranzhoepfinger
Copy link
Contributor Author

@diplfranzhoepfinger
Copy link
Contributor Author

and YEAH !!!! the URC Handler runs on the A7672

log here:

https://gist.github.com/diplfranzhoepfinger/2d663b68e47f5cbf7a11a980548166ee

i think it is a cool Feature ! 

now must implement the 2nd Layer above the URC Handler, a "URC Handler Parser" 

@diplfranzhoepfinger
Copy link
Contributor Author

@timbo100 you can see my Cmux - Example, there i got it working.

@timbo100
Copy link

@diplfranzhoepfinger and team, thanks.

while connected to a SIM7600 modem, I changed modem type to SHINY and it worked... I got connected, IP address, and tested the comm with mqtt broker.

I really want to test sending an SMS Text message, does anyone know how to use the console to do this since that command is rather complicated???

It's unclear to me, other than new functionality added to custom modem, what the difference is between a SIM7600 vs SHINY. It would seem then that the Generic Modem is very close to SIM7600(?).

Very much appreciated.

@pavellevitsky
Copy link

Switch modem from SIM76000 to SHINY - how?
Using menuconfig (Development Framework Configuration) --> Connect configuration ?
There is no SHINY in the list...

@david-cermak david-cermak added this to the modem-v1.2.0 milestone Nov 28, 2023
@david-cermak
Copy link
Collaborator

While this is supported in the example by creating a custom module, it's not very user friendly.
I've added a new milestone and will support URC in a better way, directly in the library.

@diplfranzhoepfinger
Copy link
Contributor Author

Hi @david-cermak

Added as c61fe1f

it's very simple and WIP. It only adds a user callback that could handle all received data. (needs to be removed before switching modes and applied again after)

this was working well with the CMUX Example.
but now we switched to the Dual Channel USB Example.
how to add the same URC Handler in this Example ?

https://github.com/espressif/esp-protocols/tree/master/components/esp_modem/examples/pppos_client

I mean one Problem is it is C, so the Syntax will be different.
But we would more like to stay in C.

Thanks,
Franz

@david-cermak
Copy link
Collaborator

Hi Franz,

If you want to use the idea in the modem example, you'd have to stay with C++.
You can use Dual channel mode in C++, just need to create the DTE and DCE separately (same as in other C++ examples). You can check the USB example and the C-API wrappers.

(as mentioned above, I'd like to add URC handler support to the library itself, so those overrides and customized factories won't be needed, planned for modem-v1.2 )

@diplfranzhoepfinger
Copy link
Contributor Author

does this mean:
if we want to stay with C, it will be possible with 1.2
if we swith the pppos_client Sample to C++ we can do it like now.

@david-cermak
Copy link
Collaborator

does this mean: if we want to stay with C, it will be possible with 1.2 if we swith the pppos_client Sample to C++ we can do it like now.

Yes, it means that the URC will be an official API. The solution I offered before was just a workaround, and since the C++ implementation is very modular, we were able to simply inject a handler without changing the library code (100% user space)

david-cermak added a commit to david-cermak/esp-protocols that referenced this issue Jul 24, 2024
david-cermak added a commit to david-cermak/esp-protocols that referenced this issue Sep 16, 2024
@VasilNikolovRilabs
Copy link

VasilNikolovRilabs commented Sep 17, 2024 via email

embedcat pushed a commit to embedcat/esp-protocols that referenced this issue Sep 17, 2024
david-cermak added a commit to david-cermak/esp-protocols that referenced this issue Nov 7, 2024
1.2.0
Features
- Add support for guessing mode (52598e5)
- Delete CMUX internal implementation even if terminal exit fails (0e0cbd6)
- Add support for handling URC (1b6a3b3, espressif#180)
- add ability to change ESP_MODEM_C_API_STR_MAX from Kconfig (1790989)
- Added target test config with CHAP authentication (f8ae7de)
- example add esp32p4 usb support (adafeae)
- Publish mbedtls component (0140455)
- host test support of the latest ESP-IDF release (3f74b4e)
Bug Fixes
- Fix console example to use urc/detect features (1a9eaf3)
- Update target test builds to use external Catch2 (554f022)
- Fix arguments names when spawn esp_modem_xxx declarations (b6792c5)
- Remove catch dependency (c348076)
- Examples: use local configs for MQTT topic/data (f5c13b9)
- Fixed clang-tidy warnings (70fa3af)
- Fix CI build per IDFv5.3 (d0c17ef)
- Fixed UART task to check for buffered data periodically (4bdd90c, espressif#536)
- Cleanup unused configs from PPPoS example (08a62cc)
- Update CMUX example with SIM7070_gnss cleaned-up (56fe532)
- Update console example with SIM7070_gnss format comments (5baaf54)
- Fix remaining print format warnings (3b80181)
Updated
- docs(modem): Fix esp_modem_at_raw() description (C-API) (492a6a0)
- ci(common): updated github actions(checkout, upload, download) v3 to 4, Ubuntu 20.04 to v22.04 (a23a002)
@diplfranzhoepfinger
Copy link
Contributor Author

Hello @david-cermak
i do not find the C-API URC Handler for ESP-Modem in Version 1.2.0
did you forget to add it ?

Thanks,
Franz

@diplfranzhoepfinger
Copy link
Contributor Author

@david-cermak

also next, we need to implement a URC Parser.
so that we can react on the received URCs.

Thanks,
Franz

@david-cermak
Copy link
Collaborator

i do not find the C-API URC Handler for ESP-Modem in Version 1.2.0
did you forget to add it ?

I did forget, sorry. Reopening -- will create v1.2.1 this week.

@david-cermak david-cermak reopened this Nov 18, 2024
@david-cermak
Copy link
Collaborator

@diplfranzhoepfinger Franz, since you're a regular user and also a contributor to esp-modem I'd like to ask you to check this PR: #685
Did you or your team have any problem using IDE navigation and auto code expansion with the defined AT commands?

david-cermak added a commit to david-cermak/esp-protocols that referenced this issue Nov 20, 2024
1.2.1
Bug Fixes
- Use higher GPIO range to support new chips (428fdbb, espressif#558)
- Remove tests and support for IDFv4.4, added IDFv5.4 (433a033)
- Fix typo GENETIC -> GENERIC in mode types (090b1ff, espressif#667)
- Add support for URC handler into C-API (295d99d, espressif#180)
@skibz
Copy link

skibz commented Nov 20, 2024

This is an epic addition to the C API and is going to drastically simplify several systems I'm maintaining. Thank you @david-cermak ! 🙏 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants