[Donut Merge] Support for off-switch buffering in SPGW pipeline with reordering prevention

[ccascone]

This pull request was originally opened by @robertmacdavid in #54. Carmelo re-created this to be based on the new main branch. This PR describes a potential solution to the packet re-ordering problem in downlink buffering. We leave it open for reference, it is not intended to be merged as-is. Some comments can be found in the old PR.

Below is the original description from Robert.

---

This branch makes drastic changes to the SPGW pipeline with several goals in mind.

The forwarding model of distinct "PDRs" and "FARs" is removed from the pipeline, to break away from the associated restrictions and permit tofino-specific optimizations. Now there are simply UE flows, which are amalgamates of PDRs and FARs.
The entirety of the standard fabric.p4 pipeline cannot execute until the last table of the spgw pipeline has finished. The deeper the SPGW pipeline, the less resources available for all other modules. Operations have been reshuffled to reduce the pipeline depth to fewer stages. In some cases, the reshuffling is ugly. An attempt was made to document the reason behind the reshuffling in such cases.
The new pipline supports redirecting downlink UE flows to an off-switch buffering device, assumed to be dbuf. A custom GTPU extension header was added for preserving some pipeline metadata for when the packet returns from dbuf. Registers were added to avoid packet reordering.
There is also one minor change to the INT pipeline, which fixes a bug introduced by fixing an inconsistent behavior in the
SPGW pipeline. When GTPU encapsulation was performed, fabric_md.bridged.l4_sport and fabric_md.bridged.l4_dport were erroneously not updated to refer to the new outer headers (compare to fabric_md.ipv4_src and fabric_md.ipv4_dst), meaning these fields would refer to the innermost L4 ports for the downlink case, but the outermost for the uplink case. INT relied upon this (what I believe to be an) incorrect assumption. New metadata fields fabric_md.bridged.innermost_l4_sport and fabric_md.bridged.innermost_l4_dport were added as a fix.

The primary drawback of this approach is that a large amount of changes will need to be made to the UP4 app. There is currently a one-to-one and almost-stateless mapping between up4.p4 table entries and fabric.p4 entries. That will no longer be the case if this branch is merged as-is.