Intra-process communication latency

[alsora]

Bug report

Required Info:

• Operating System:
  • Ubuntu 16.04
• Installation type:
  -from source
• Version or commit hash:
  • 0.5.1
• DDS implementation:
  • Fast-RTPS
• Client library (if applicable):
  • rclcpp

Steps to reproduce issue

I slightly modified the demos/intra_process_demo/src/two_node_pipeline/two_node_pipeline.cpp example.

Changing the message type to something "bigger" such as

std_msgs/Header header
byte[4096] array

Moreover I recorded the communication latency as the difference between the timestamp in the message header and the current time when the subscriber callback is triggered.

Expected behavior

Intra-Process communication avoids copying object around and just share a pointer, thus reducing latency.

Actual behavior

When use_intra_process_comms is set to true in the node constructors, the average latency is almost identical or even slightly bigger.

[wjwwood]

I believe this might be a duplicate, or at least related to, this pr (and discussion):

ros2/rclcpp#504 (comment)

Basically the issue comes down to this:

In order to avoid a copy between the publisher and subscription callback, a unique_ptr is published which is then passed into the middleware to be delivered (uncopied) to the single subscription callback in the same process.

However, because the ownership of this pointer is not in the control of the publishing function, a copy has to be made first so that this copy may be sent to the middleware (so it can be sent over the network). This copy is likely where the latency you're seeing is coming from.

Unfortunately this must be done right now, for two reasons:

• we didn't know when someone is listening on the topic from a different process
• if the QoS for durability is not volatile (the topic is latched) then the message needs to be stored by the middleware for late subscriptions

Since that pr's discussion, we now have the ability to see how many subscriptions are matched to our publisher, see:

ros2/rcl#326

But that still doesn't let us see the GUID of the subscriptions, and therefore we cannot tell the difference between a subscription in our process (which will be serviced by intra process) and a subscription in a separate process which will require sending the data over the middleware.

If we could address that, then we could avoid this copy and the latency you're seeing iff you're not using transient local durability (if you're not latching).

[wjwwood]

But that still doesn't let us see the GUID of the subscriptions, and therefore we cannot tell the difference between a subscription in our process (which will be serviced by intra process) and a subscription in a separate process which will require sending the data over the middleware.

I was thinking about this, and if we compare the matched count for the intra process topic and the inter process topic and they were the same size, then I think it would be safe to say all subscriptions are in the same process so it is not necessary to publish to the inter process one as well.