feature request: ability to add python modules to extend functionality

[wx4cb]

not sure if it's at all possible, but would it be possible to add functionality to extend it with python modules. i've written a maestro servo controller module (same as using in ardupilot) in python so would be cool to be able to use that as an antenna tracker module.

[stronnag]

Interesting idea; unfortunately, as an ex-FORTRAN programmer, I have a severe pathological aversion to languages that are dependent on significant white space which I'm unlikely to be able to overcome, even with medical or psychological help.

However, mwp already provides a dbus instance; extending that to provide the information that an antenna tracker needs is a really interesting idea. Consider me interested, as it would be language / application independent.

• Home position
• Location of vehicle
• Range, bearing and azimuth of vehicle

(for example).

[wx4cb]

i know what you mean, but i'm trying to force myself to learn python. (i prefer perl).

as far as what's needed, yea, home position (or theoretically would be armed position), current location and height, and you can work out range bearing from the two known gps coords - again another python module i've made :D

[stronnag]

perhaps 10 years ago, this perl user decided he needed to learn something more readable / modern. I selected ruby over python.

Anyway, I'll extend the dbus implementation to provide a (flight) status, home / armed location and current location / altitude. Seems like an interesting project (for both of us).

[wx4cb]

cool.... yea i was the same way... way back when yahoo messenger was a thing, i used to help maintain a ruby chat program. worked through 5 revisions of the protocol (got to the point they changed it every 6 months) until they finally pulled the pin on messenger. that was all written in ruby.

been a while since i wrote in ruby. used to all the time. maybe i might figure it out again.

[stronnag]

OK, so I propose to add a few new dbus methods, synchronous call to get the home and vehicle location, and async (dbus signals) to push event by event changes to home or vehicle locations. The mwp dbus API introspection will then return (something like):

   <!DOCTYPE node PUBLIC "-//freedesktop//DTD D-BUS Object Introspection 1.0//EN"
                      "http://www.freedesktop.org/standards/dbus/1.0/introspect.dtd">
<!-- GDBus 2.60.2 -->
<node>
  <interface name="org.freedesktop.DBus.Properties">
    <method name="Get">
      <arg type="s" name="interface_name" direction="in"/>
      <arg type="s" name="property_name" direction="in"/>
      <arg type="v" name="value" direction="out"/>
    </method>
    <method name="GetAll">
      <arg type="s" name="interface_name" direction="in"/>
      <arg type="a{sv}" name="properties" direction="out"/>
    </method>
    <method name="Set">
      <arg type="s" name="interface_name" direction="in"/>
      <arg type="s" name="property_name" direction="in"/>
      <arg type="v" name="value" direction="in"/>
    </method>
    <signal name="PropertiesChanged">
      <arg type="s" name="interface_name"/>
      <arg type="a{sv}" name="changed_properties"/>
      <arg type="as" name="invalidated_properties"/>
    </signal>
  </interface>
  <interface name="org.freedesktop.DBus.Introspectable">
    <method name="Introspect">
      <arg type="s" name="xml_data" direction="out"/>
    </method>
  </interface>
  <interface name="org.freedesktop.DBus.Peer">
    <method name="Ping"/>
    <method name="GetMachineId">
      <arg type="s" name="machine_uuid" direction="out"/>
    </method>
  </interface>
  <interface name="org.mwptools.mwp">
    <method name="SetMission">
      <arg type="s" name="msg" direction="in"/>
      <arg type="i" name="result" direction="out"/>
    </method>
    <method name="GetHome">
      <arg type="y" name="fix" direction="out"/>
      <arg type="d" name="lat" direction="out"/>
      <arg type="d" name="lon" direction="out"/>
      <arg type="d" name="alt" direction="out"/>
      <arg type="y" name="nsats" direction="out"/>
    </method>
    <method name="GetLocation">
      <arg type="y" name="fix" direction="out"/>
      <arg type="d" name="lat" direction="out"/>
      <arg type="d" name="lon" direction="out"/>
      <arg type="d" name="alt" direction="out"/>
      <arg type="y" name="nsats" direction="out"/>
    </method>
    <method name="DeviceNames">
      <arg type="as" name="dd" direction="out"/>
      <arg type="i" name="result" direction="out"/>
    </method>
    <method name="UploadMission">
      <arg type="s" name="mn" direction="in"/>
      <arg type="i" name="result" direction="out"/>
    </method>
    <signal name="HomeChanged">
      <arg type="y" name="fix"/>
      <arg type="d" name="lat"/>
      <arg type="d" name="lon"/>
      <arg type="d" name="alt"/>
      <arg type="y" name="nsats"/>
    </signal>
    <signal name="LocationChanged">
      <arg type="y" name="fix"/>
      <arg type="d" name="lat"/>
      <arg type="d" name="lon"/>
      <arg type="d" name="alt"/>
      <arg type="y" name="nsats"/>
    </signal>
  </interface>
</node>

And there will be ruby and python examples in the mwp samples directory (perl is left as an exercise for the reader), the ruby sample looking like:

#!/usr/bin/ruby

require 'dbus'

# Create bus and service object
bus = DBus::SessionBus.instance
service = bus.service("org.mwptools.mwp")
mwp = service.object("/org/mwptools/mwp")

# Test if it's up.
# Rather than abort, we could start an instance of mwp if appropriate
#
begin
  pif = mwp["org.freedesktop.DBus.Peer"]
  pif.Ping
rescue
  abort "Service unavailable"
end

# Set the default interface
mwp.default_iface = "org.mwptools.mwp"

# dump out the interface definitions
#puts mwp.introspect
home = mwp.GetHome
puts "Init Home: #{home.join(' ')}"
loc= mwp.GetLocation
puts "Init Location: #{loc.join(' ')}"

mwp.on_signal("HomeChanged") do |fix,lat,lon,alt,sats|
  puts "Home changed: #{[fix,lat,lon,alt,sats].join(' ')}";
end

mwp.on_signal("LocationChanged") do |fix,lat,lon,alt,sats|
  puts "Vehicle changed: #{[fix,lat,lon,alt,sats].join(' ')}";
end

loop = DBus::Main.new
loop << bus
loop.run

And example output via a simulator that generates fake home every 60s and fake vehicle every 5s:

$ ./mwp-dbus-loc.rb
Init Home: 3 50.9 -1.59 19.0 16
Init Location: 3 50.9 -1.59 19.0 16
Vehicle changed: 3 50.945015355548584 -1.4601697740090613 21.716540006689726 11
Vehicle changed: 3 50.94108892810509 -1.4014042345671474 20.939402571380626 16
Vehicle changed: 3 50.80284383448028 -1.6345107612273868 15.159930316041079 17
Vehicle changed: 3 51.08448400353352 -1.771021846669922 16.435789963737783 16
Vehicle changed: 3 50.92242669253433 -1.659899352430048 22.899273211744973 16
Vehicle changed: 3 50.970202085634945 -1.4781194321035498 18.221376853044518 13
Vehicle changed: 3 51.04022325861279 -1.5524190268238307 21.9209228563129 18
Vehicle changed: 3 51.01170726016206 -1.6359415533180686 18.741003196005934 14
Vehicle changed: 3 50.767065657226816 -1.6377800630202473 19.536465550561992 11
Vehicle changed: 3 51.050143681623716 -1.6509550586624886 17.700590629974375 19
Vehicle changed: 3 50.934047492113876 -1.4986648896248247 16.11604373472268 18
Home changed: 3 50.900155514699314 -1.5896578908616374 19.000100321661417 11
Vehicle changed: 3 50.99164227553212 -1.6965933150301253 16.8075266415225 15
Vehicle changed: 3 50.90528612227922 -1.7643818830410445 21.42753603938019 15
Vehicle changed: 3 50.998604225310785 -1.4765781641225324 21.934331066854007 16
Vehicle changed: 3 50.80777197281351 -1.681904476451735 21.185685372659098 11

Expect a test branch in github in the next few days.

A client application can either poll for location changes or subscribe to the async signals.

Will this meet the requirement ?

[wx4cb]

dang dude.... that was quick lol. i've just been reading up on dbus signals etc (which im assuming is what you mean by events).

but yea all you need really is lat/long/alt for the home and the aircraft, you can figure out the rest relaively simple

[stronnag]

And here's the naive python example:

#!/usr/bin/python

# Simple python example for mwp dbus

import sys
import dbus
import dbus.mainloop.glib

from gi.repository import GLib

def loc_handler(*args):
    print('sig loc: ', args)

def home_handler(*args):
    print('sig home: ', args)

dbus.mainloop.glib.DBusGMainLoop(set_as_default=True)
try:
    bus = dbus.SessionBus()
    obj = bus.get_object("org.mwptools.mwp", "/org/mwptools/mwp")
    mwp = dbus.Interface(obj, "org.mwptools.mwp")

except dbus.DBusException as e:
        print(str(e))
        sys.exit(255)

home  = mwp.GetHome()
print("Init Home: ", home)
loc  = mwp.GetLocation()
print("Init Vehicle: ", loc)

obj.connect_to_signal('LocationChanged', loc_handler)
obj.connect_to_signal('HomeChanged', home_handler)
loop = GLib.MainLoop()
loop.run()

$ ./mwp-dbus-loc.py 
Init Home:  (dbus.Byte(3), dbus.Double(50.9), dbus.Double(-1.59), dbus.Double(19.0), dbus.Byte(16))
Init Vehicle:  (dbus.Byte(3), dbus.Double(50.9), dbus.Double(-1.59), dbus.Double(19.0), dbus.Byte(16))
sig loc:  (dbus.Byte(3), dbus.Double(51.011882911679294), dbus.Double(-1.3953381997145167), dbus.Double(18.451714331902558), dbus.Byte(19))
sig loc:  (dbus.Byte(3), dbus.Double(50.903136071159466), dbus.Double(-1.660467977539013), dbus.Double(20.663030946279914), dbus.Byte(16))
sig loc:  (dbus.Byte(3), dbus.Double(50.94129574607298), dbus.Double(-1.7155204481849222), dbus.Double(22.67078450233807), dbus.Byte(14))
sig loc:  (dbus.Byte(3), dbus.Double(50.8410343148923), dbus.Double(-1.712678462519394), dbus.Double(18.82139269508565), dbus.Byte(14))
sig loc:  (dbus.Byte(3), dbus.Double(50.91803884153409), dbus.Double(-1.3953096462349568), dbus.Double(22.54083267662535), dbus.Byte(17))
sig loc:  (dbus.Byte(3), dbus.Double(51.064393853883274), dbus.Double(-1.6692160618909773), dbus.Double(22.879821726013557), dbus.Byte(11))
sig loc:  (dbus.Byte(3), dbus.Double(50.8013383630179), dbus.Double(-1.6465535869731465), dbus.Double(19.19094609269579), dbus.Byte(14))
sig loc:  (dbus.Byte(3), dbus.Double(50.80527005847673), dbus.Double(-1.3965491737743803), dbus.Double(22.234457267654935), dbus.Byte(16))
sig home:  (dbus.Byte(3), dbus.Double(50.89973072647667), dbus.Double(-1.59051715402584), dbus.Double(18.999520334384666), dbus.Byte(13))
sig loc:  (dbus.Byte(3), dbus.Double(51.088982987516005), dbus.Double(-1.5807091351965856), dbus.Double(19.66594826038754), dbus.Byte(19))

[wx4cb]

dayum....

[wx4cb]

i would suggest (and i assume you're doing it) would be to force send a "home changed" and "Location Changed" event on arming, as I can for see a situation where it has a location for the uav, but not a home location on startup. if that makes sense

[stronnag]

The signals will be pushed whenever the data changes and on vehicle state changes (including arm). I either need to have a vehicle state changed signal, or include it in the other signals ....

[wx4cb]

I would suggest a separate one to.be honest. Juat then you're not tying yourself in to having to potentially make changes to the format in the future id you tie them into the location data string.

[stronnag]

Sure, makes the implementation easier too.

[stronnag]

There's now an initial implementation in development 3ba8220
I still have to write some documentation, however the example samples/mwp-dbus-loc.rb exercises the whole API e.g.

# iinstall ruby-dbus 
# e.g. Ubuntu `sudo apt install ruby-dbus`
# Arch `sudo pacman -S ruby-dbus`
# or `sudo gem install ruby-dbus` if the distro doesn't provide a package  
# in one terminal, start mwp
$ mwp 
# in a second terminal, start the sample:
$ samples/mwp-dbus-loc.rb

Then replay a log file (Blackbox or mwp log) in the mwp instance. You should see a stream of status / location / sat coverage signals displayed in the second terminal.

Note that samples/mwp-dbus-loc.rb dumps out the Dbus API. You can also dump it out by:

# Note mwp must be running, so the bus is defined ....
$ samples/mwp-dbus-test.sh introspect

There's also a less capable python example.

None of this is final, so any suggestions for improvement are welcome.

[wx4cb]

cool... im working on nit now... you wouldnt happen to have a logfile i can replay by chance?

[stronnag]

Attached a zip file with a mwp (not blackbox) log and the output captured from it by samples/mwp-dbus-loc.rb

mwplog.zip

[wx4cb]

thanks..... btw

> ./mwp_at.py /dev/ttyACM0
Using /dev/ttyACM0 for Maestro controller
Centering all channels on /dev/ttyACM0
Cycling servo on channel 0
Set servo channel 0 to -90 degrees (600 ms)
Set servo channel 0 to 90 degrees (2400 ms)
Set servo channel 0 to 0 degrees (1600 ms)
Cycling servo on channel 1
Set servo channel 1 to -90 degrees (1000 ms)
Set servo channel 1 to 90 degrees (2000 ms)
Set servo channel 1 to 0 degrees (1500 ms)
Creating DBUS Loop
Attempting DBUS Connection to MWP
Getting MWP Device List
Found Device: /dev/ttyUSB0@57600
Found Device: /dev/ttyACM1
Found Device: /dev/ttyACM0
Found Device: /dev/rfcomm0
Init Home: (dbus.Double(0.0), dbus.Double(0.0), dbus.Double(0.0))
Init Vehicle: (dbus.Double(0.0), dbus.Double(0.0), dbus.Double(0.0))
Init Sats: (dbus.Byte(0), dbus.Byte(0))

[stronnag]

As Dbus is not a high volume interface, I've added a position update rate limit property. This is not a problem for real world telemetry protocols, however for pre-arm, USB / MSP polling, mwp can potentially generate a lot of Dbus traffic.

There is now a Dbus property 'DBusPosInterval' to (optionally) rate limit position updates. Value is the minimum update interval in 0.1s units; 0 disables rate limiting, the default is 1 (10Hz), a value of 10 would give a 1Hz positional update rate.

Ruby and Python examples updated.

[stronnag]

Here's three more useful logs culled from public sources (RCG, Github) issues that will be more representative for a longer range antenna tracker: http://www.zen35309.zen.co.uk/bbl/lr-logs.tar.xz

[wx4cb]

yea tbh, i think 10hz is more than enough for updates.

I went out today and the dbus interface seemed to work great. I havent finished the actual servo control yet, but it kinda works. i have to figure out how to get the bearing normalised into -90/90. which is what the controller takes. plus i havent yet finished building the hardware yet - it's still running through the cnc :)

[stronnag]

LTM at best is 5Hz, so I might drop the default to that. Don't want to wear out your servos.

[wx4cb]

I think 5 times/s is adequate. You're not doing 200mph in small circles after all :)

[wx4cb]

I think it works :D

now to make it pretty 👍 and see if i can get the servos to move right :D

Home: +51.4081 +11.3445 - UAV: +51.4050 +11.3410 - Distance: +0.4255 - Elevation: +0.0038 - Bearing: -3
Home: +51.4081 +11.3445 - UAV: +51.4049 +11.3409 - Distance: +0.4274 - Elevation: +0.0038 - Bearing: -3
Home: +51.4081 +11.3445 - UAV: +51.4049 +11.3409 - Distance: +0.4293 - Elevation: +0.0039 - Bearing: -3
Home: +51.4081 +11.3445 - UAV: +51.4049 +11.3409 - Distance: +0.4313 - Elevation: +0.0039 - Bearing: -3
Home: +51.4081 +11.3445 - UAV: +51.4049 +11.3409 - Distance: +0.4332 - Elevation: +0.0039 - Bearing: -3
('State Changed: ', (dbus.Int32(5),))
Home: +51.4081 +11.3445 - UAV: +51.4049 +11.3409 - Distance: +0.4353 - Elevation: +0.0039 - Bearing: -3
Home: +51.4081 +11.3445 - UAV: +51.4049 +11.3409 - Distance: +0.4373 - Elevation: +0.0039 - Bearing: -3
Home: +51.4081 +11.3445 - UAV: +51.4049 +11.3408 - Distance: +0.4394 - Elevation: +0.0040 - Bearing: -3
Home: +51.4081 +11.3445 - UAV: +51.4048 +11.3408 - Distance: +0.4415 - Elevation: +0.0040 - Bearing: -3
Home: +51.4081 +11.3445 - UAV: +51.4048 +11.3408 - Distance: +0.4436 - Elevation: +0.0040 - Bearing: -3
Home: +51.4081 +11.3445 - UAV: +51.4048 +11.3408 - Distance: +0.4457 - Elevation: +0.0040 - Bearing: -3
Home: +51.4081 +11.3445 - UAV: +51.4048 +11.3408 - Distance: +0.4479 - Elevation: +0.0040 - Bearing: -3

[stronnag]

Nice

[wx4cb]

do you have a list of what the status numbers are (i assume 1 is armed)

[stronnag]

There's a DBus API for that (from the ruby sample, 0 indexed).

state_name = mwp.GetStateNames[0]
puts "Available states #{state_name.inspect}\n"
## giving ...
Available states ["DISARMED", "MANUAL", "ACRO", "HORIZON", "ANGLE", "CRUISE", "RTH", "LAND", "WP", "HEADFREE", "POSHOLD", "ALTHOLD", "LAUNCH", "AUTOTUNE", "UNDEFINED"]

Anything non-zero is armed.

[stronnag]

@stronnag slinks off to write some documentation ....
I was going to do the inav-radar stuff, but needs must. Dbus it is.

[wx4cb]

@stronnag slinks off to write some documentation ....
I was going to do the inav-radar stuff, but needs must. Dbus it is.

I'm supposed to be working too lol...

[stronnag]

and following from the previous example, display *man-readable states:

state = mwp.GetState[0]
puts "Inital state #{state_name[state]}"

[stronnag]

AFAIK, the ruby example exercises the whole API. I know it's not python but ....

[wx4cb]

that's fine, i got it

[wx4cb]

how would I access the bearing the MWP is using in the mwp-dbus.vala file to include it in the signal but im a little confused as to where they're actually would be getting set so I could get the correct (class?) variables

i was thinking of adding something like:

    internal double v_bearing;
     internal double v_heading;

to the top and then changing the loc signal:

    public void get_location(out double latitude, out double longitude,
                           out double altitude, out double bearing, out double heading) throws GLib.Error
      {
          latitude = v_lat;
          longitude = v_long;
          altitude = v_alt;
  
          bearing = v_bearing;
          heading = v_heading;
     }
 

[wx4cb]

im' trying to figure out why mine is way off compared to yours

[stronnag]

I'll add them ... tomorrow, need to get up stupidly early in the morning to get Mrs Stronnag on her train for the Chelsea Flower Show.

[stronnag]

OK, new commit and new methods / signals:

mwp.on_signal("PolarChanged") do |range, bearing, azimuth|
  puts "Polar changed: #{[range, bearing, azimuth].join(' ')}"
end

mwp.on_signal("VelocityChanged") do |speed, course|
  puts "Velocity changed: #{[speed, course].join(' ')}"
end

And similar Get* methods. The *Polar method /signal report:

• range from ground station to vehicle (m)
• bearing from ground station to vehicle (°)
• azimuth of vehicle (vertical angle, °)

I think this alone might be enough to drive a tracker?

[wx4cb]

shouldn't the direction and azimuth have "out" before them in the definition or am i reading it wrong

as i'm getting an error when i call GetPolarCoordinates - tried that even in d-feet

"g-dbus-error-quark: GDBus.Error:org.freedesktop.DBus.Error.InvalidArgs: Type of message, '()', does not match expected type '(qq)' (16)"

mwp_dbus.vala line 82

public void get_polar_coordinates(out uint16 range, uint16 direction, uint16 azimuth) throws GLib.Error
{
range = v_range;
direction = v_direction;
azimuth = v_azimuth;
}

[stronnag]

Damn it, yes. I'm fixing it. I'll write a test this time.

[wx4cb]

yup... modified mine to include the out now it seems like it's working :D check velocity too :D

[stronnag]

done

[wx4cb]

am i missing something. the bearing still doesn't match up with what is shown on mwp in the flight view tab

[stronnag]

It's the reciprocal (+/- minor latency). The default implementation follows the MSP COMP_GPS message which provides the range and bearing from the vehicle to the ground station (which always struck me as useless: I'm at the ground station and I want to know where the vehicle is! (and the direction to walk if I crash it at range). The Flight View widget and speech is output is controlled by:

gsettings set org.mwptools.planner audio-bearing-is-reciprocal true
# default is false

If you use Bing Maps for better zoom (or the other BB logs posted above) you'll see the affect.

[stale]

This issue / pull request has been automatically marked as stale because it has not had any activity in 30 days. The resources of the mwp team are limited, and inactive issues/PR are a burden on the project.
This issue / pull request will be closed if no further activity occurs within two weeks.

[wx4cb]

Thought ypu might be interested in the progress ive been making.

Local gps and mag and a general working model for the "plugin", the bearing and correction work well enough on the bench with the mag stuck to a servo lol

[stronnag]

Nice, very very neat.

[wx4cb]

@stronnag ty. beauty is, it'll work with anything that can send a particular packet format.... plain text CSV 🍭

[wx4cb]

@stronnag another update for you. it seems to work good (dbus side) only thing i've noted is that the bearing seems to be 180 out as noted in #62 and i was trying to figure out what units it was sending some things out in as velocity wasn't what it showed - figured out it was m/s :D

but it all seems good. only need to get some other things like arming state added _ (in another request)

https://www.youtube.com/watch?v=WT5rb5nPlEo (ignore the audio. it's way out of sync - gg YT)

[stale]

This issue / pull request has been automatically marked as stale because it has not had any activity in 30 days. The resources of the mwp team are limited, and inactive issues/PR are a burden on the project.
This issue / pull request will be closed if no further activity occurs within two weeks.

[stale]

Automatically closing as inactive.