Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

unable to configure protobuf deserialization , error ,".google.protobuf.Timestamp" is not defined. #1055

Open
hari819 opened this issue Mar 22, 2022 · 7 comments
Labels
backend Need a backend update bug Something isn't working topic data Kafka Topic data

Comments

@hari819
Copy link

hari819 commented Mar 22, 2022

I have converted my proto files to desc files and followed the link https://akhq.io/docs/configuration/protobuf.html , but could not get it working ,

syntax = "proto3";

import "google/protobuf/timestamp.proto";



message ForElastic {
  google.protobuf.Timestamp eventDate = 2;

i am continuously getting this timestamp error , please can someone guide me on this ,

2022-03-22 15:21:05,494 �[1;31mERROR�[0;39m �[35mr-thread-3�[0;39m �[36mo.a.c.ErrorController �[0;39m Cannot build Protobuf descriptors for the topics regex [ForElastic.*]
java.lang.RuntimeException: Cannot build Protobuf descriptors for the topics regex [ForElastic.*]
at org.akhq.utils.ProtobufToJsonDeserializer.buildAllDescriptors(ProtobufToJsonDeserializer.java:63)
at org.akhq.utils.ProtobufToJsonDeserializer.<init>(ProtobufToJsonDeserializer.java:40)
at org.akhq.repositories.CustomDeserializerRepository.getProtobufToJsonDeserializer(CustomDeserializerRepository.java:26)
at org.akhq.repositories.RecordRepository.newRecord(RecordRepository.java:446)
at org.akhq.repositories.RecordRepository.lambda$getLastRecord$3(RecordRepository.java:105)
at java.base/java.lang.Iterable.forEach(Unknown Source)
at org.akhq.repositories.RecordRepository.getLastRecord(RecordRepository.java:103)
at org.akhq.controllers.TopicController.lastRecord(TopicController.java:223)
at org.akhq.controllers.$TopicController$Definition$Exec.dispatch(Unknown Source)
at io.micronaut.context.AbstractExecutableMethodsDefinition$DispatchedExecutableMethod.invoke(AbstractExecutableMethodsDefinition.java:351)
at io.micronaut.context.DefaultBeanContext$4.invoke(DefaultBeanContext.java:583)
at io.micronaut.web.router.AbstractRouteMatch.execute(AbstractRouteMatch.java:303)
at io.micronaut.web.router.RouteMatch.execute(RouteMatch.java:111)
at io.micronaut.http.context.ServerRequestContext.with(ServerRequestContext.java:103)
at io.micronaut.http.server.RouteExecutor.lambda$executeRoute$14(RouteExecutor.java:656)
at reactor.core.publisher.FluxDeferContextual.subscribe(FluxDeferContextual.java:49)
at reactor.core.publisher.InternalFluxOperator.subscribe(InternalFluxOperator.java:62)
at reactor.core.publisher.FluxSubscribeOn$SubscribeOnSubscriber.run(FluxSubscribeOn.java:194)
at io.micronaut.reactive.reactor.instrument.ReactorInstrumentation.lambda$null$0(ReactorInstrumentation.java:62)
at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84)
at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37)
at io.micrometer.core.instrument.composite.CompositeTimer.recordCallable(CompositeTimer.java:68)
at io.micrometer.core.instrument.Timer.lambda$wrap$1(Timer.java:171)
at io.micronaut.scheduling.instrument.InvocationInstrumenterWrappedCallable.call(InvocationInstrumenterWrappedCallable.java:53)
at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)
Caused by: com.google.protobuf.Descriptors$DescriptorValidationException: ForElastic.eventDate: **".google.protobuf.Timestamp" is not defined.**
at com.google.protobuf.Descriptors$DescriptorPool.lookupSymbol(Descriptors.java:2563)
at com.google.protobuf.Descriptors$FieldDescriptor.crossLink(Descriptors.java:1559)
at com.google.protobuf.Descriptors$FieldDescriptor.access$1400(Descriptors.java:1057)
at com.google.protobuf.Descriptors$Descriptor.crossLink(Descriptors.java:1000)
at com.google.protobuf.Descriptors$Descriptor.access$1200(Descriptors.java:648)
at com.google.protobuf.Descriptors$FileDescriptor.crossLink(Descriptors.java:600)
at com.google.protobuf.Descriptors$FileDescriptor.buildFrom(Descriptors.java:321)
at com.google.protobuf.Descriptors$FileDescriptor.buildFrom(Descriptors.java:290)
at org.akhq.utils.ProtobufToJsonDeserializer.buildAllDescriptorsForDescriptorFile(ProtobufToJsonDeserializer.java:94)
at org.akhq.utils.ProtobufToJsonDeserializer.buildAllDescriptors(ProtobufToJsonDeserializer.java:61)
... 27 common frames omitted
Logs from Mar 22, 202
@tchiotludo
Copy link
Owner

Seems that a } is missing on the proto desc ...

@hari819
Copy link
Author

hari819 commented Mar 22, 2022

Seems that a } is missing on the proto desc ...

no i did not copy the full proto file here, the actual file has the brackets closed

@tchiotludo
Copy link
Owner

I'm a noob on protobuf but the implementation is talking about that here

@hari819
Copy link
Author

hari819 commented Mar 22, 2022

i can see the alhq.jar file has all the proto files including timestamp.proto , but no clue why it is saying timestamp not found,

@hari819
Copy link
Author

hari819 commented Mar 23, 2022

i will try to dig more on this , but would like to know what value to pass here into "value schema" field while pushing data into topic ,
image

please could you help me on this,

@MilkyWay-core
Copy link

got the same problem

@tchiotludo
Copy link
Owner

The PR is here: #543 but won't merged due to conflict, PR are welcome

@tchiotludo tchiotludo added bug Something isn't working backend Need a backend update topic data Kafka Topic data labels May 15, 2022
@tchiotludo tchiotludo moved this to Backlog in Backlog Jun 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend Need a backend update bug Something isn't working topic data Kafka Topic data
Projects
Status: Backlog
Development

No branches or pull requests

3 participants