Question: What would be needed in order to create dynamic messages with proto imports

[gabrielerzinger]

First of all, thank you very much for your work on this lib. It's trully awesome and definitely something that is missing on Go. Given that, i've been struggling with the following task: how can i make dynamic messages from protos that import other protos?

Lets say i have Proto A, which imports Proto b in its file, i've already seen that simply trying to use the proto descriptor of A isn't enough, since it says not enough dependencies, so I wanted to know, is it already supported to - somehow - use descriptors to create dyn. messages of protos with import? And if not, what could I study in order to do so?

Again, thank you very much, this repo is awesome!

[jhump]

@gabrielerzinger, I am assuming you are getting an error when you try to create the descriptor (not when creating the message), right?

IIUC, you are trying to use desc.CreateFileDescriptor(...) and are supplying the descriptor proto for proto A. Is that right? If so, take a close look at the signature for desc.CreateFileDescriptor: it accepts both the descriptor proto and all of its dependencies (which must already be upgraded to "rich" desc.FileDescriptor instances).

To make it more concrete, let's assume we have the following situation:

1. We want a descriptor for file-a.proto.
2. This file imports file-b.proto and file-c.proto.
3. The lattermost file, c, imports file-d.proto.

In order to create a descriptor for a, we have to construct descriptors for its dependencies first (in reverse topological order: leaves first).

So the following sequence would be necessary for the given example:

// for brevity, omitting error handling (but real code should
// always check the error returned)
fdD, _ := desc.CreateFileDescriptor(fileDProto)
fdC, _ := desc.CreateDescriptor(fileCProto, fdD)
fdB, _ := desc.CreateDescriptor(fileBProto)
fdA, _ := desc.CreateDescriptor(fileAProto, fdB, fdC)

So as you can see, we had to supply the dependencies in that last call, when creating a descriptor for file-a.proto.

[gabrielerzinger]

First of all, thank you very much for the quick response. Do you think there is any way I could make it so I could extract the dependencies from .pb.go ? Because what I have right now, is the ability to, given a proto message name, retrieve it's descriptor (as follow) and build a dynamic message from it (not a big deal, simply using your lib and reflection):

     [...]
     protoReflectTypePointer := proto.MessageType(protoName)
     protoReflectType := protoReflectTypePointer.Elem()
     protoValue := reflect.New(protoReflectType)
     descriptorMethod, ok := protoReflectTypePointer.MethodByName("Descriptor")
     descriptorValue := descriptorMethod.Func.Call([]reflect.Value{protoValue})
     protoDescriptor := descriptorValue[0].Bytes()
     return protoDescriptor

If it were possible to get those dependencies from .pb.go, this method could work for any proto message. But maybe this isnt the right place to ask this question, unless you think this would be a good feature for the lib, and I could try to do it and make a PR. What are your thoughts on that?

Again, tyvm for the attention

[jhump]

If you have the fully-qualified name of the message, this should be easy:

md, err := desc.LoadMessageDescriptor(msgName)

If that isn't working, I suspect the error message is different.

The most common issue is when you've compiled your protos (to .pb.go files) in a way that the path you supplied to protoc differs from the path you use to import the proto from other files. You can read more about this issue here: golang/protobuf#567. There's also an existing issue in this library to provide API that allows users to work-around these issues (#147).

To resolve, you will need to modify your invocations of protoc such that the path you supply to the source protos is identical to how you then import those same files from others. For example, if you have a file foo/bar/baz.proto and it has an import statement for foo/bar/snafu.proto, then you should make sure that the invocation of the latter uses foo/bar/snafu.proto as the argument to protoc (and adjust proto path arguments and/or PWD for the invocation so it works).

[gabrielerzinger]

The most common issue is when you've compiled your protos (to .pb.go files) in a way that the path you supplied to protoc differs from the path you use to import the proto from other files. You can read more about this issue here: [golang/protobuf#567]

I've runned some pre-tests and Im almost sure that was the problem! Thank you very much and thank you for the great work with this lib. (This trully should be added to golang/protobuff sometime)!

I think we can close this and sorry if it was not appropriate for an issue :)

[jhump]

This trully should be added to golang/protobuff sometime

Most of the functionality in this repo will become part of the core runtime. Here's a design doc for v2 API for the core protobuf library, and here's the in-progress work.

sorry if it was not appropriate for an issue :)

No worries!