Skip to content

Commit

Permalink
Add doc article
Browse files Browse the repository at this point in the history
Signed-off-by: amaslennikov <[email protected]>
  • Loading branch information
almaslennikov committed Jun 27, 2024
1 parent 9e88ac4 commit 418c6bc
Showing 1 changed file with 80 additions and 0 deletions.
80 changes: 80 additions & 0 deletions doc/ib-vf-guid-configuration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
# Infiniband VF GUID Configuration

SR-IOV Network Operator is able to use a static configuration file from the host filesystem to assign GUIDs to IB VFs.

## Prerequisites

- Nvidia Infiniband NICs
- Static GUID configuration file on the host filesystem

### Configuration file

Config file should be stored at `/etc/sriov-operator/infiniband/guids`. This location is writable across most cloud platforms.

VF to GUID assignment, based on this file, is ordered. VF0 takes the GUID, VF1 takes the GUID1 etc.
For each PF

Example of the config file:

```json
[
{
"pci_address": "<pci_address_1>",
"guids": [
"02:00:00:00:00:00:00:00",
"02:00:00:00:00:00:00:01"
]
},
{
"pf_guid": "<pf_guid_2>",
"guidsRange": {
"start": "02:00:00:00:00:aa:00:02",
"end": "02:00:00:00:00:aa:00:0a"
}
}
]
```

Requirements for the config file:

* `pci_adress` and `pf_guid` cannot be set at the same time for a single device - should return an error
* if the list contains multiple entries for the same device, the first one shall be taken
* `rangeStart` and `rangeEnd` are both included in the range
* `guids` list and range cannot be both set at the same time for a single device - should return an error
* GUIDs are assigned once and not change throughout the lifecycle of the host

### Deploy SriovNetworkNodePolicy

```yaml
apiVersion: sriovnetwork.openshift.io/v1
kind: SriovNetworkNodePolicy
metadata:
name: ib-policy
namespace: sriov-network-operator
spec:
nodeSelector:
feature.node.kubernetes.io/network-sriov.capable: "true"
resourceName: ib_vfs
priority: 10
numVfs: 2
nicSelector:
vendor: "15b3"
rootDevices:
- 0000:d8.00.0
deviceType: netdevice
```
## Verify assigned GUIDs
Run ip link tool to verify assigned GUIDs:
```bash
ip link

...
6: ibp216s0f0: <BROADCAST,MULTICAST> mtu 4092 qdisc noop state UP mode DEFAULT group default qlen 256
link/infiniband 00:00:05:a9:fe:80:00:00:00:00:00:00:b8:59:9f:03:00:f9:8f:86 brd 00:ff:ff:ff:ff:12:40:1b:ff:ff:00:00:00:00:00:00:ff:ff:ff:ff
vf 0 link/infiniband ... NODE_GUID 02:00:00:00:00:00:00:00, PORT_GUID 02:00:00:00:00:00:00:00, link-state enable, trust off, query_rss off
vf 1 link/infiniband ... NODE_GUID 02:00:00:00:00:00:00:01, PORT_GUID 02:00:00:00:00:00:00:01, link-state enable, trust off, query_rss off
```

0 comments on commit 418c6bc

Please sign in to comment.