13. August 2024

[dimakuv]

Agenda

(please write your proposed agenda items in comments under this discussion)

• Dmitrii and Kailun: Discuss "EDMM lazy allocation" PR
  • Time to engage plan B -- remove the relevant portions of the PR that trigger the Linux driver bugs?
    • Surround the madvise(DONTNEED) with #ifdef LINUX_KERNEL_PATCHED -- so we circumvent the second Linux driver bug
    • Re-instate the workaround (checking if #PF happened on EAUG instruction) for the first Linux driver bug
• Mwk: discuss [RFC] epoll-like PAL interface #1971
• Dmitrii: AEX Notify flows, continued ([PAL/Linux-SGX] Add AEX-Notify support #1530 and [PAL/Linux-SGX] Add AEX-Notify support in exception handling flow #1531)
  • Dmitrii will present his findings and proposals as described in SGX signal handling flows #1948
• Dmitrii: discuss Env ulimit/prlimit not inherited by Gramine process #1576
  • Priority bumped; simply hard-coding a higher value (e.g. 4096) should be enough for the customer

Dmitrii and Kailun: Discuss "EDMM lazy allocation" PR

• Woju: what is the impact of the workaround for the first Linux driver bug?

  • Dmitrii: no particular performance impact, it's a couple lines of if-then code in the exception handler
  • Final decision: yes, can re-instate the workaround

• Woju doesn't like commenting out (with an undeclared macro) for the second Linux driver bug

  • Woju: propose to add a second commit that removes this currently-not-working code, and then revert this second commit
    • Michal: but then Gramine will have one (first) commit that breaks on current Linux
  • Kailun: maybe split that madvise(DONTNEED) in a separate PR
    • Michal: will be annoying to review again...
  • Final decision: keep that code commented out (no need to re-review, and enough tests to notice the issues in the future when we comment-in back)

TODO(Kailun): modify the PR according to the decisions.

mwk: discuss #1971

• Dmitrii wrote his objections in [RFC] epoll-like PAL interface #1971 (comment)
  • Everyone agrees that the epoll PAL API solution is complex and hard to implement in other PALs
• Dmitrii: we can try to do the same trick as for emulating EPOLLET flag in epoll:
  • If object is not "epollet-ready" (not yet read or written), then temporarily remove from the list of pollable events in Async Thread
  • See e.g.

    gramine/libos/include/libos_internal.h

    Line 230 in 0c0971c

    * \brief Check if next `epoll_wait` with `EPOLLET` should trigger for this handle.

  • mwk: agreed, may be workable

TODO(mwk): can take this task

Dmitrii: discuss #1576

• Michal: we can use the proposal in Env ulimit/prlimit not inherited by Gramine process #1576 (comment)

• Woju: these limits are totally unrelated to the host OS

  • So if the host OS has a lower limit, then the app will be surprised
  • If the host OS has some limit, but Gramine sets this limit higher with setrlimit(), then the app is again surprised
  • This is user-unfriendly, and it will be cumbersome to explain to customers who will run into this "dummy but functional" implementation

• Michal: RLIMIT_NOFILE is the only one from the supported rlimits used in practice, let's add a special manifest option to let users hard-code the value.

  • Michal: we won't allow "passthrough" for several reasons (1) it is hard/impossible to implement on non-Linux hosts, (2) <don't remember the other one>.
  • Final decision: Add a special manifest option to change RLIMIT_NOFILE at startup; no other code changes needed.

TODO(Dmitrii): will do this.

[mwkmwkmwk]

I'd like to nominate #1971