[Feature Request] Provide a legit way to override HTTP Response.

[nguyentranbao-ct]

🚀 Feature Request

Issue with error logs for ForwardResponseOptions

PR #4245 changed how errors are logged, which is cool. But it also started logging a bunch of stuff we already aware.

My situation:

I need to override the response sometimes. To do that, I use ForwardResponseOptions with a custom function that might return an error. This tells grpc-gateway to chill and not write anything else. Works great, but then it fills the logs with these messages after v2.20.0:

Error handling ForwardResponseOptions: xxx

Tried a few things:

• Setting GRPC_GO_LOG_SEVERITY_LEVEL=none would stop all logs, but I still need to see other errors.
• Overriding grpc.Logger is also an option, but that's kinda overkill just to omit the simple log.

Proposal:

Add a new variable like ErrResponseHandled. This would tell grpc-gateway to:

• Stop logging "Error handling ForwardResponseOptions" for these cases.
• Not rewrite the response body (which I already took care of).
• Less code clutter by avoiding unnecessary logging.

Similar issues but have been closed:

• Override HTTP response code for particular service method? #517
• How to elegant rewrite/custom resp body from gRPC resp ?  #1610

Code examples (in case they help):

Here's what the change in grpc-gateway/runtime/handler.go would look like:

// runtime/handler.go
package runtime

var ErrResponseHandled = errors.New("response handled")

func handleForwardResponseOptions(ctx context.Context, w http.ResponseWriter, resp proto.Message, opts []func(context.Context, http.ResponseWriter, proto.Message) error) error {
  if len(opts) == 0 {
    return nil
  }
  for _, opt := range opts {
    if err := opt(ctx, w, resp); err != nil {
      if !errors.Is(err, ErrResponseHandled) {
        grpclog.Errorf("Error handling ForwardResponseOptions: %v", err)
      }
      return err
    }
  }
  return nil
}

And here's how I use it in my code:

// in your files
func forwardResponse(ctx context.Context, w http.ResponseWriter, m protoreflect.ProtoMessage) error {
  switch v := m.(type) {
  case A, B, C:
    w.WriteHeader(http.StatusNoContent)
	// response has been written, don't write anything else
    return runtime.ErrResponseHandled
  case X, Y, Z:
    resp := map[string]any{
      "Success": true,
      "Data":    m,
    }
    w.WriteHeader(http.StatusOK)
    if err := json.NewEncoder(w).Encode(resp); err != nil {
      return err
    }
    // response has been written, don't write anything else
    return runtime.ErrResponseHandled
  default:
    // continue to process response normally
    return nil
  }
}

Let me know what you think!

[johanbrandhorst]

Hi, thanks for your well written issue. I'm not sure that this meets the threshold of utility for a new feature (though minor in cost). I usually like to hear from two separate users that are having the same issue before considering it. To rephrase your request (correct me if I'm wrong):

1. You have a custom forward response handler that sometimes but not always takes care of writing the whole response. This is similar to an example from our docs: https://grpc-ecosystem.github.io/grpc-gateway/docs/mapping/customizing_your_gateway/#mutate-response-messages-or-set-response-headers.
2. In order to stop handleForwardResponseOptions from executing, you return a sentinel error.
3. This error is now logged by the grpclogger when it's not actually an error to you.

Now, as an alternative, have you tried just not returning an error from the response option? Since you've called w.WriteHeader, only the body can be mutated further, so maybe nothing else happens in your application?

[nguyentranbao-ct]

Yup, you're absolutely right about my request. But it's not just me, our organization uses this approach as well. Additionally, two similar issues I mentioned earlier requested similar functionality but haven't reached a conclusion yet.

Currently, the WithForwardResponseOptions allows modifying the response value but not able to rewrite the entire body, which includes structural changes. While returning nil prevents an error, grpc-gateway still writes the original message.

For example:

message GetNameResponse {
  string name = 1;
}

Here's the relevant server code snippet:

mux := runtime.NewServerMux(
  runtime.WithHealthEndpointAt(grpc_health.NewHealthClient(conn), "/health"),
  runtime.WithMarshalerOption(runtime.MIMEWildcard, &runtime.JSONBuiltin{}),
  runtime.WithForwardResponseOption(forwardResponse),
)

func forwardResponse(ctx context.Context, w http.ResponseWriter, m protoreflect.ProtoMessage) error {
  switch v := m.(type) {
  case *pb.GetNameResponse:
    resp := map[string]any{
      "success": true,
      "data":    m,
    }
    return runtime.ErrResponseHandled // (1)
  default:
    return nil
  }
}

Issue with Returning nil:

Returning nil at point (1) results in the client receiving an invalid JSON response because the original payload is appended to the custom response:

{"success":true,"data":{"name": "John Doe"}}{"name": "John Doe"}

But if we return error at (1), it works as expected:

{"success":true,"data":{"name": "John Doe"}}

[johanbrandhorst]

Yep, I see the problem. I think the workflow of having ultimate control to override the HTTP response is something we do want to support. Let me propose some other options (not saying no to this feature yet, just exploring the problem space):

1. Have you tried using a custom marshaler? You could combine a forwardResponseOption that sets headers with a custom marshaler that writes data in a specific way. Of course, if you need to alter the output based on some information that isn't available to the marshaler (such as headers in the gRPC response) then you're out of luck.
2. Have you tried using the google/api/httpbody.proto HttpBody response type? That allows you to set an arbitrary response in the gRPC handler. This compromises the client side generation type safety, of course.
3. Lastly, have you tried using google/protobuf/struct.proto Struct type for arbitrary JSON output? This has the same problem as above.

Let me know what you think!

[nguyentranbao-ct]

1. The Custom Marshaller works as expected, just as you mentioned. Although it requires extra effort, we can use it along with forwardResponse to fully override responses.

   type CustomPB struct {
   	runtime.JSONPb
   }
   
   func (c *CustomPB) Marshal(data any) ([]byte, error) {
   	resp := data
   	switch v := data.(type) {
   	case *X, *Y:
   		resp = map[string]any{
   			"success": true,
   			"data":    data,
   		}
   	}
   	return json.Marshal(resp)
   }

2.3. It’s okay to use these approaches, but my aim is to override HTTP responses. We want to maintain a structured response with gRPC calls.

[johanbrandhorst]

Do you think this is sufficient for now? If so, would you be willing to contribute an update to our docs that mentions this pattern for fully overriding responses? We already have a section for setting headers but it would be good to complement it with a custom marshaler example. Thanks!

[nguyentranbao-ct]

Yep, I agree, this is sufficient for now! I'll squeeze in some time to update the docs with that custom marshaler example.

[nguyentranbao-ct]

It took a while but I finally have time for this simple task. I've now added the section on fully overriding custom HTTP responses using both a Forward Response Option and a Custom Marshaller. Please review the changes #4564 and let me know if any further adjustments are needed.