Comparing two fields with diffirent types that are semantically equivalent

[daenney]

This one is a bit odd, but I'm trying to test if a payload can successfully round-trip from the original JSON, through my type's Unmarshal+Marshal cycle and come back out the semantically the same on the other end. (the semantic is the tricky bit)

The way I test that is by unmarshalling the original payload into a any as well as the payload after the Unmarshal+Marshal and then calling cmp.Diff.

This works, except that for some fields there's 2 possible encodings for them that are semantically equivalent. A field may come in as either a slice of strings, or a singular string. I deal with this in my Unmarshal method, but my Marshaler has no way of knowing what the original "shape" was and in the case of the single element slice outputs the string instead. This is semantically correct, but obviously go-cmp is unhappy about that.

So, if you get this in:

{
  "a": "b"
}

Or:

{
  "a": ["b"]
}

My Marshaler will output the equivalent and "most compact" form:

{
  "a": "b"
}

What I'm trying to figure out is if I can deal with that situation in go-cmp. The Transformer doesn't seem like it would fit, since I only need to apply this to some specific fields. I've taken a look at FilterPath, but I'm not sure how to apply it.

I suspect the answer is "can't be done", in which case I can reprocess the resulting map[string]any to deal with the difference. It's very few fields that exhibit this characteristic in practice so that's doable. But if anyone has any ideas on how to do it in go-cmp that would be helpful.

[dsnet]

Hi, I have a general sense of the problem your describing, but the solution is sensitive to the exact Go types you're dealing with, could you provide a simple reproduction?

[daenney]

Here's a basic case:

package main

import (
	"encoding/json"
	"fmt"

	"github.com/google/go-cmp/cmp"
)

func main() {
	val1 := []byte(`{
		"object": {
			"to": [
				"string"
			]
		},
		"to": "string"
	}`)

	var res Payload
	if err := json.Unmarshal(val1, &res); err != nil {
		panic(err)
	}
	data, err := json.MarshalIndent(res, "", "    ")
	if err != nil {
		panic(err)
	}

	fmt.Println(string(data) + "\n")

	var orig any
	var tripped any

	if err := json.Unmarshal(val1, &orig); err != nil {
		panic(err)
	}

	if err := json.Unmarshal(data, &tripped); err != nil {
		panic(err)
	}

	fmt.Println(cmp.Diff(orig, tripped))
}

type Payload struct {
	To     W3String `json:"to,omitempty"`
	Object struct {
		To W3String `json:"to,omitempty"`
	} `json:"object,omitempty"`
}

type W3String []string

func (w *W3String) UnmarshalJSON(data []byte) error {
	switch data[0] {
	case '"':
		var s string
		if err := json.Unmarshal(data, &s); err != nil {
			return err
		}
		*w = W3String{s}
	case '[':
		type alias W3String
		var res alias
		if err := json.Unmarshal(data, &res); err != nil {
			return err
		}
		*w = W3String(res)
	default:
		panic("please no")
	}
	return nil
}

func (w W3String) MarshalJSON() ([]byte, error) {
	switch len(w) {
	case 0:
		return []byte(`null`), nil
	case 1:
		return json.Marshal(w[0])
	default:
		type alias W3String
		return json.Marshal(alias(w))
	}
}

So in both cases I end up with "to": "string" in the output, except that the to in the object member initially came in as ["string"]. So for some fields, I want to treat slice with 1 element string vs string, as long as it's the same string, as equivalent for the purpose of Diff.

{
    "to": "string",
    "object": {
        "to": "string"
    }
}

  map[string]any{
- 	"object": map[string]any{"to": []any{string("string")}},
+ 	"object": map[string]any{"to": string("string")},
  	"to":     string("string"),
  }

[daenney]

For now I've gone the way of making an internal type that is a map[string]any and correcting the fields when it runs into them during a json.Unmarshal. It's hideous but it's a test helper and it's hard to argue with results. That way cmp doesn't have to do any work.

If there's still some ideas on how to solve it in go-cmp I'm all ears. Even just a description of the general approach / what to chain would be helpful. I can probably figure it out from there.