pact plugin message contents differ in pact ffi 0.4.6+ (now base64 encoded)

[YOU54F]

During some testing and upgrading of pact-js/pact-go from 0.4.5 -> 0.4.12 (latest at the time of testing), there is a difference in the Pact content, which is now recorded as base64 encoded. These contents are consumed by the matt plugin 0.0.9

1. pact-go https://github.com/pact-foundation/pact-go/tree/master/examples/plugin
2. pact-js https://github.com/pact-foundation/pact-js/tree/master/examples/v4/plugins/test

I'm trying to work out if this is intended, and if as plugin authors, we should be considering the encoded value and decoding the base64 content before sending it to the system under test. Currently to get the tests to pass, I would have to modify the clients, as they are now receiving base64 encoded content where they aren't expecting it.

     {
       "request": {
         "contents": {
-          "content": "MATThellotcpMATT",
-          "encoded": false
+          "content": "TUFUVGhlbGxvdGNwTUFUVA==",
+          "encoded": "base64"
         }
       },
       "response": [
         {
           "contents": {
-            "content": "MATTtcpworldMATT",
-            "encoded": false
+            "content": "TUFUVHRjcHdvcmxkTUFUVA==",
+            "encoded": "base64"
           }
         }
       ]

Cookie trail

1. libpact_ffi Release 0.4.6 https://github.com/pact-foundation/pact-reference/releases/tag/libpact_ffi-v0.4.6
2. plugin-driver release 0.4.5 https://github.com/pact-foundation/pact-plugins/releases/tag/pact-rust-driver-v0.4.5

0.4.5 Pact

{
 "consumer": {
   "name": "matttcpconsumer"
 },
 "interactions": [
   {
     "description": "Matt message",
     "pending": false,
     "providerStates": [
       {
         "name": "the world exists"
       }
     ],
     "request": {
       "contents": {
         "content": "MATThellotcpMATT",
         "contentType": "application/matt",
         "contentTypeHint": "DEFAULT",
         "encoded": false
       }
     },
     "response": [
       {
         "contents": {
           "content": "MATTtcpworldMATT",
           "contentType": "application/matt",
           "contentTypeHint": "DEFAULT",
           "encoded": false
         }
       }
     ],
     "transport": "matt",
     "type": "Synchronous/Messages"
   }
 ],
 "metadata": {
   "pactRust": {
     "ffi": "0.4.5",
     "mockserver": "1.1.1",
     "models": "1.1.2"
   },
   "pactSpecification": {
     "version": "4.0"
   },
   "plugins": [
     {
       "configuration": {},
       "name": "matt",
       "version": "0.0.9"
     }
   ]
 },
 "provider": {
   "name": "matttcpprovider"
 }
}

0.4.6 Pact

{
  "consumer": {
    "name": "matttcpconsumer"
  },
  "interactions": [
    {
      "description": "Matt message",
      "pending": false,
      "providerStates": [
        {
          "name": "the world exists"
        }
      ],
      "request": {
        "contents": {
          "content": "TUFUVGhlbGxvdGNwTUFUVA==",
          "contentType": "application/matt",
          "contentTypeHint": "DEFAULT",
          "encoded": "base64"
        }
      },
      "response": [
        {
          "contents": {
            "content": "TUFUVHRjcHdvcmxkTUFUVA==",
            "contentType": "application/matt",
            "contentTypeHint": "DEFAULT",
            "encoded": "base64"
          }
        }
      ],
      "transport": "matt",
      "type": "Synchronous/Messages"
    }
  ],
  "metadata": {
    "pactRust": {
      "ffi": "0.4.6",
      "mockserver": "1.2.2",
      "models": "1.1.8"
    },
    "pactSpecification": {
      "version": "4.0"
    },
    "plugins": [
      {
        "configuration": {},
        "name": "matt",
        "version": "0.0.9"
      }
    ]
  },
  "provider": {
    "name": "matttcpprovider"
  }
}

[YOU54F]

failing and passing ci runs to repro

https://github.com/pact-foundation/pact-go/actions/runs/7617794146
https://github.com/pact-foundation/pact-go/actions/runs/7617813884

[mefellows]

I've taken a look at the MATT plugin to try and work out what's going on.

I thought perhaps I could force the content not to be encoded by explicitly setting ContentTypeHint to TEXT but that doesn't help (it still encodes it).

Curiously, the CompareContents API call has the unencoded details in it:

2024/01/23 18:27:57 Received CompareContents request: expected:{contentType:"application/matt"  content:{value:"MATTworldMATT"}}  actual:{contentType:"application/matt;charset=utf-8"  content:{value:"MATTworldMATT"}}  allow_unexpected_keys:true  pluginConfiguration:{interactionConfiguration:{}  pactConfiguration:{}}

[rholshausen]

I don't see what the issue is. Both forms are valid. TUFUVGhlbGxvdGNwTUFUVA== == MATThellotcpMATT

[YOU54F]

the client isn't sending or expecting a base64 response, something in Pact, in 0.4.6+ of the FFI is encoding the payload as base64 in the pact.

That would be okay, if on the provider side, it decoded the content before sending the request to the provider.

Currently that isn't the case, the provider service receives a base64 encoded payload, from the pact verifier, and I don't know if the pact client library (pact-go/pact-js), or the plugin author (Matt in this case), needs to make changes to their code.

The fact an end-users test needs to change seems like a code smell

[rholshausen]

Ok, that is a different problem. The Pact file has how it has been encoded, and it should be decoded when the Pact is loaded before being used.

[YOU54F]

I saw from the cookie trail of commits, that there was a fix put in for v3 contents to decode the body, let me find the commit (this is a v4 pact)

[rholshausen]

It must be a plugin issue, I think it passes everything on to the plugin to setup the request to make

[YOU54F]

Here is the plugin code for PrepareInteractionForVerification that is used in the test

this is the v3 commit fix: V3 message binary content was not being base64 decoded correctly when loaded from a Pact file which came after 0.4.6 of the pact ffi, so assumed this was as a result of the change to the encoding

a03fc5f

[rholshausen]

That commit is for V3 Pacts, this one is V4

[YOU54F]

wasn't sure if it needed to be applied for v4 too, or if that is/was a red herring. I didn't dig into too much, just an eyeball

[YOU54F]

So in pact ffi 0.4.5 the key generation was removed, and were no longer stored in the Pact (it was causing issues when uploading Pacts to a broker, and performing comparisons) as part of this commit

Related issues

• Key generation in Pact V4 interactions is unstable across multiple runs #264
• key field's value should not change in generated Pact file when running same consumer test multiple times for Pact v4 pact-go#230

The plugin errors seen here, use a version which was built and relied on FFI 0.3.15

#278

Updating the plugin to solve the issues documented in the OP required

1. removing the interaction key check against the pact (now not there) against the interaction key generated by plugin driver
2. base64 decoding the content provided to PrepareInteractionForVerification here

This leaves me with working tests in pact-go and pact-js which utilise this plugin, but leaves open the question in this issue around

How can a plugin author lookup an Interaction in the Pact struct during PrepareInteractionForVerification?

But I think once I've applied the update to the matt plugin, and got that rolled out in the pact-go and pact-js example. I can close this ticket and follow it on the other raised issue