23. August 2022

[dimakuv]

Agenda

(please write your proposed agenda items in comments under this discussion)

• Vijay: Local Attestation and UNIX Domain Sockets, see RFC: Support for Local Attestation (LA-TLS) between Enclaves #814

The whole meeting was spent on discussing the above item from Vijay. The summary of the discussions can be found in #814. See #814 (comment)

[vans163]

Can anyone go a bit indepth on these 3rd gen Xeon bios options?

• SGX Package Info In-Band access (?)
• PRMRR Size (this is just mem reserved to SGX at boot, not the max mem per enclave?)
• SGX QoS (?)
• Epoch 1/2 (extra entropy per enclave key?)
• PUBKEYHASH0-4 (?)
• SGX Auto MP Registration (?)

[dimakuv]

What about if PRMRR is 2GB and 10 enclaves with 256MB are started+use_256MB, will the last few fail?

This depends on the total amount of available physical memory (RAM + swap file), not on the size of PRMRR. As soon as PRM (more specifically, its EPC part that holds the 4KB enclave pages) starts to become full, the Linux kernel starts swapping out the least-recently-used enclave pages out of the EPC and into RAM (and if there is not enough free pages in RAM, then the Linux kernel further tries to swap out from RAM to the swap file on hard drive).

Of course, the swap operation is very slow. So when your SGX enclaves eat up all PRM/EPC memory, this swapping process starts, and the whole system's performance drops. I suggest to also read our explanation here: https://gramine.readthedocs.io/en/latest/manifest-syntax.html#enclave-size

But yes, if you start 10 256MB enclaves (i.e., 2GB of enclave size in total) and your system has only 2GB of RAM + swap file, then you're doomed -- Linux has no other choice but to kill one of these SGX enclaves.

Is it not possible to use DCAP without registering the platform? I thought the new attestation in SGX2 could be done without intel in the loop (or was planned to).

Hm, I'm not sure about this question. I hope @ScottR-Intel can reply here.

[vans163]

So system has 256GB of ram in these examples (128GB ram not feasible for 3rd Gen Xeon because you need to populate atleast 8 dimm slots, and 16GB sticks are only about 30-50% cheaper than 32GB sticks [not the usual ~90% cheaper due to 2x less density]).

Swap is disabled, 0 bytes available for swap.

Keeping PRMRR at 2GB should be fine then, even if starting enclaves that will eventually physically consume 32G of ram?

[dimakuv]

Keeping PRMRR at 2GB should be fine then, even if starting enclaves that will eventually physically consume 32G of ram?

It will be fine from the functionality perspective (SGX enclaves will run correctly, and Linux's OOM killer won't destroy these enclaves due to insufficient physical memory).

But it will be bad from the performance perspective. 32GB of enclaves backed by 2GB of PRMRR will lead to a lot of swap-from-EPC-to-RAM operations, and these operations are expensive. So the performance of your enclave execution will degrade significantly.

TLDR: It depends on your needs. If you are Ok to lose some enclave performance when your enclaves start to exceed 2GB, then keep PRMRR=2GB (and you'll have a lot of RAM available for other, non-SGX-enclave workloads). If the only thing that you plan to run on this system are SGX enclaves, then it makes sense to maximize your PRMRR setting.

[ScottR-Intel]

Hi @vans163.

First, a little terminology clarification... In the Intel SDM, the only feature the term "SGX2" refers to is the Enhanced Dynamic Memory Management (EDMM) feature. It does not refer to any other features like flexible launch control, key separation and sharing, DCAP attestation support, etc.

As for your DCAP question... On 3rd Gen Xeon Scalable (Ice Lake Server) and later platforms, you will always have to "register" the CPUs/platform with Intel Registration Service to be able to generate and verify quotes using ECDSA based SGX remote attestation (RA). This is a hard requirement and RA will not work without this step. This can be done in an air-gapped environment if need be, though, utilizing our PCK Cert ID Retrieval on the air-gapped platform and a simple curl command (see the API docs at the Intel Registration Service page) on a platform that is connected to the internet. You will also, even in an air-gapped situation, have to periodically return to Intel's Provisioning Certification Services to download updated evidence required to verify quotes. As an FYI, neither Xeon-E based server nor Core consumer based platforms with SGX support have to do this registration... only the Scalable CPUs do.

Hope this helps clarify.

Scott

[vans163]

@dimakuv Thanks understood.
@ScottR-Intel Thanks understood.

[jsun39]

SGX Package Info In-Band access (?)
**When you enable this knob, you can get the the UEFI variable file: /sys/firmware/efi/efivars/SgxRegistrationPackageInfo-ac406deb-ab92-42d6-aff7-0d78e0826c68. It will contain the per-package key blob information, by default this information is saved in NVRAM, when you enable this knob, you can access it OS. Of course, the context is encrypted.
PRMRR Size (this is just mem reserved to SGX at boot, not the max mem per enclave?)
** No, it means the whole EPC size.
SGX QoS (?)
** SGX Quality of service
Epoch 1/2 (extra entropy per enclave key?)
** different EPOCH will result different sgx key. So you can set your owner EPOCH value, when you want to retire this platform or transfer this platform to others, to protect your data, you can just change the EPOCH value, then others could NOT decrypt your data.
PUBKEYHASH0-4 (?)
** it will be filled with the loaded enclave's MRsigner. By the way, it should be 03, not 04.
SGX Auto MP Registration (?)
** this know will enable the automatic registration or not. When this knob is enable, you can install Multi-package Agent, then when system boot into OS, the Multi-package agent server will automatically register your platform to Intel Registration Server.