is there any example on how to ingest GPS data from GPS sensor connected to Jetson NX into MQQT?

[AndreV84]

Hi guys, How is it going?
I am just wondering if you could share an example on how to send GPS data from Jetson with MQQT.
Thank you very much!

[rdejana]

I don't have a GPS, but it would be pretty much like any other data. You'd need a library to read your GPS info and then craft a "normal" MQTT message with your information being the payload.

[AndreV84]

@rdejana Thank you for your reply!
I do not have a bare metal GPS unit.
However, there is a way to simulate GPS :
terminal 1:

nvidia@nx_xavier:~$  socat -d -d pty,raw,echo=0 pty,raw,echo=0
2021/05/06 13:13:55 socat[23720] N PTY is /dev/pts/25
2021/05/06 13:13:55 socat[23720] N PTY is /dev/pts/26
2021/05/06 13:13:55 socat[23720] N starting data transfer loop with FDs [5,5] and [7,7]

terminal 2:

sudo apt install csh -y
wget https://iweb.dl.sourceforge.net/project/gpsfeed/gpsfeed-latest.zip
unzip gpsfeed-latest.zip
nvidia@nx_xavier:~$ tclsh8.6 gpsfeed+.tcl 

From this point there will be opened GUI of the gpsfeed. - two GUI windows;
e.g. in serial section goes insteead of default COM1 -> /dev/pts/25; baudrate to ->9600
Then in second GUI window = pressing play [ in the top left - a circle icon with a dot in the center;]
Now from the third terminal the GPS feed is available

:~$ cat /dev/pts/26
$GPGGA,180756,3756.2421,N,02346.8948,E,1,04,5.6,724.4,M,34.5,M,,*43
$GPRMC,180756,A,3756.2421,N,02346.8948,E,,,060521,5,E,A*01
$GPGGA,180758,3751.4936,N,02349.1123,E,1,04,5.6,384.3,M,34.5,M,,*4D
$GPRMC,180758,A,3751.4936,N,02349.1123,E,9109.2,159.8,060521,5,E,A*30
$GPGGA,180800,3747.8756,N,02353.0000,E,1,04,5.6,134.0,M,34.5,M,,*4C
$GPRMC,180800,A,3747.8756,N,02353.0000,E,8541.7,139.7,060521,5,E,A*3E
$GPGGA,180802,3745.9547,N,02358.0261,E,1,04,5.6,9.7,M,34.5,M,,*49
$GPRMC,180802,A,3745.9547,N,02358.0261,E,7942.4,115.8,060521,5,E,A*31

From this point, given I am running the mosquito container from docker implementation like with

docker run -it --rm --name mosquitto -p 1883:1883 mosquitto

How to pass the tty/pts26 feed to the docker mosquito? could you extend, please?

Thank you very much!

Reference:

1. Pseudo TTY. https://www.florian-wolters.de/blog/2015/12/11/pseudo-tty-devices-for-gps-emulation/
2. x86 gps emulator https://sourceforge.net/p/gpsfeed/
3. https://github.com/rdejana/w251-hw3/tree/main/mosquitto#mqtt-broker

[AndreV84]

with just python I could read, but I am still trying to figure out how to process it with mqtt
cat pts_v2.py

import sys                                           

with open("/dev/pts/26", "wb+", buffering=0) as term:
        while True:
        	print(term.read(26).decode(), end='')
        sys.stdout.flush()

based on https://stackoverflow.com/a/46181793

like

python3 pts_v2.py 
$GPGGA,191157,3756.8819,N,02404.8014,E,1,04,5.6,455.4,M,34.5,M,,*4D
$GPRMC,191157,A,3756.8819,N,02404.8014,E,3442.9,42.0,060521,5,E,A*04
$GPGGA,191159,3758.8726,N,02405.7998,E,1,04,5.6,809.2,M,34.5,M,,*4E
$GPRMC,191159,A,3758.8726,N,02405.7998,E,3853.2,21.6,060521,5,E,A*00
$GPGGA,191201,3801.1623,N,02405.9797,E,1,04,5.6,1190.8,M,34.5,M,,*73
$GPRMC,191201,A,3801.1623,N,02405.9797,E,4129.4,3.5,060521,5,E,A*39
$GPGGA,191203,3803.4176,N,02405.2627,E,1,04,5.6,1544.6,M,34.5,M,,*73
$GPRMC,191203,A,3803.4176,N,02405.2627,E,4184.8,345.9,060521,5,E,A*3C
$GPGGA,191205,3805.2904,N,02403.7043,E,1,04,5.6,1819.2,M,34.5,M,,*7E
$GPRMC,191205,A,3805.2904,N,02403.7043,E,4029.8,326.8,060521,5,E,A*32
$GPGGA,191207,3806.4714,N,02401.4940,E,1,04,5.6,1974.4,M,34.5,M,,*71

brainstormed here https://forums.developer.nvidia.com/t/windows-iot-core/177135/15

[rdejana]

look over the library https://pypi.org/project/paho-mqtt/. You would read the message in, then turn that into your payload.

[rdejana]

You can add a device when you run a container with the option --device=/dev/

[AndreV84]

@rdejana Thank you for following up!
Do you think this example might be a good reference or might work out at once? https://github.com/tspannhw/rpizw-nifi-mqtt-gps/blob/master/gps2.py
I believe to decomposite the entire NMEA pipeline to many messages then define each of them as is illustrated in the paho-mqtt reference might take infinite time. Probably some gps example needs to be studied to get ready libraries to process the import somehow

[rdejana]

what have tried? you can do things like read, send message, sleep, read again...

[AndreV84]

trying to adjust the code [2] of the gps example.
Otherwise the only way to decomposite the entire /dev/pts/26 output would seem to use "sed" to extract pieces of the text output somehow

1. running the mosquitto with

docker run -it --rm --name mosquitto -p 1883:1883 mosquitto

then trying to run modified code so it would conenct/ publish to the mosquitto

import os
from gps import *
from time import *
import time
import threading
import json
import paho.mqtt.client as mqtt
MQTT_BROKER = os.getenv('MQTT')

gpsd = None 

class GpsPoller(threading.Thread):
  def __init__(self):
    threading.Thread.__init__(self)
    global gpsd #bring it in scope
    gpsd = gps(mode=WATCH_ENABLE) #starting the stream of info
    self.current_value = None
    self.running = True #setting the thread running to true

  def run(self):
    global gpsd
    while gpsp.running:
      gpsd.next() #this will continue to loop and grab EACH set of gpsd info to clear the buffer

if __name__ == '__main__':
  gpsp = GpsPoller() # create the thread
  try:
    gpsp.start() # start it up
    while True:
      if gpsd.fix.latitude > 0:
        row = [ { 'latitude': str(gpsd.fix.latitude),
         'longitude': str(gpsd.fix.longitude),
         'utc': str(gpsd.utc),
         'time':   str(gpsd.fix.time),
         'altitude': str(gpsd.fix.altitude),
         'eps': str(gpsd.fix.eps),
         'epx': str(gpsd.fix.epx),
         'epv': str(gpsd.fix.epv),
         'ept': str(gpsd.fix.ept),
         'speed': str(gpsd.fix.speed),
         'climb': str(gpsd.fix.climb),
         'track': str(gpsd.fix.track),
         'mode': str(gpsd.fix.mode)} ]
json_string = json.dumps(row)
        client = mqtt.Client("P1")
        client.connect(MQTT_BROKER)
        client.publish("gps", payload=json_string, qos=0, retain=True)

        time.sleep(60)

  except (KeyboardInterrupt, SystemExit): #when you press ctrl+c
    gpsp.running = False
    gpsp.join() # wait for the thread to finish what it's doing

References:

1. http://www.danmandle.com/blog/getting-gpsd-to-work-with-python/
2. https://github.com/tspannhw/rpizw-nifi-mqtt-gps

[rdejana]

Here is a very simple GO app that does what you are asking. I'm using the TCP connection from the gps feed simulator vs dealing with setting up the pts device.

package main

/*
very simple demo app that reads from a socket and writes to mqtt
this is JUST an example
*/

import (
	"bufio"
	"fmt"
	MQTT "github.com/eclipse/paho.mqtt.golang"
	"net"
)

func main() {
	//setup mqtt, all hard coded
	topic := "demo"
	broker := "tcp://localhost:1883"

	id := "myclient"
	qos := 1

	opts := MQTT.NewClientOptions()
	opts.AddBroker(broker)
	opts.SetClientID(id)

	var connectLostHandler MQTT.ConnectionLostHandler = func(client MQTT.Client, err error) {
		fmt.Printf("Connect lost: %v", err)
		panic(err)
	}
	opts.OnConnectionLost = connectLostHandler
	//connect
	client := MQTT.NewClient(opts)
	if token := client.Connect(); token.Wait() && token.Error() != nil {
		panic(token.Error())
	}

	//using https://iweb.dl.sourceforge.net/project/gpsfeed/gpsfeed-latest.zip
	// and connecting via a tcp socket
	service := "localhost:2222"
	tcpAddr, err := net.ResolveTCPAddr("tcp4", service)
	if err != nil {
		panic(err)
	}

	conn, err := net.DialTCP("tcp", nil, tcpAddr)
	if err != nil {
		panic(err)
	}
	//buffered reader
	reader := bufio.NewReader(conn)
	for {
		//read a line in
		message,_, err := reader.ReadLine()
		if err != nil {
			fmt.Println(err)
			conn.Close()
			//c.Server.onClientConnectionClosed(c, err)
			break
		}
		//send a line
		token := client.Publish(topic, byte(qos), false, string(message))
		if token.Error() != nil {
			panic(token.Error())
		}

		token.Wait()
		fmt.Println(string(message))
	}


	//close out if we error
	client.Disconnect(250)

	fmt.Println("Sample Publisher Disconnected")



}

I'm reading a message, then writing it directly to MQTT. Note, I'm leaving my MQTT connection open to reduce the TCP overhead.

[AndreV84]

Thank you very much!
With the code below I can recieve the message from the mosquitto docker

import paho.mqtt.client as mqtt
import os
import time

MQTT_BROKER = os.getenv('MQTT')
MQTT_RECEIVE = "demo"



# The callback for when the client receives a CONNACK response from the server.
def on_connect(client, userdata, flags, rc):
    print("Connected with result code "+str(rc))

    # Subscribing in on_connect() means that if we lose the connection and
    # reconnect then subscriptions will be renewed.
    client.subscribe(MQTT_RECEIVE)


# The callback for when a PUBLISH message is received from the server.
def on_message(client, userdata, message):
    print("we have a message....")
    print("message received " ,str(message.payload.decode("utf-8")))
    print("message topic=",message.topic)
    print("message qos=",message.qos)
    print("message retain flag=",message.retain)


client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message

client.connect(MQTT_BROKER)

# Starting thread which will receive the frames
client.loop_start()


while True:
    time.sleep(60) # wait

It looks like

 python2 go1.py 
Connected with result code 0
we have a message....
('message received ', '$GPGGA,160606,3803.2431,N,02401.4439,E,1,04,5.6,1913.5,M,34.5,M,,*7E')
('message topic=', u'demo')
('message qos=', 0)
('message retain flag=', 0)
we have a message....
('message received ', '$GPRMC,160606,A,3803.2431,N,02401.4439,E,2138.1,319.6,070521,5,E,A*33')
('message topic=', u'demo')
('message qos=', 0)
('message retain flag=', 0)
we have a message....
('message received ', '$GPGGA,160608,3803.7311,N,02400.1303,E,1,04,5.6,1999.4,M,34.5,M,,*79')

the thing was to connect to a socket - that takes the pipeline without decomposition
Thank you very much!

[rdejana]

Cool. yea, I found socket easier to setup for the simulator.

[AndreV84]

@rdejana May I know if you have got any successes reducing the size of the docker container produced from nvidia l4t-ml base image? as my colleague points out to an effort to find a solution for reducing the size from 4+gb to at least 0.5 gb
It seems your Dickerfile is based on the same l4t-ml
https://github.com/MIDS-scaling-up/v2/blob/master/week06/labs/facenet-lab/Dockerfile
Thanks

[AndreV84]

Howdy -- @rdejana
you may like to check pipelines of mediapipe for jetson [ supports by default /dev/vide0 usb camera]
https://forums.developer.nvidia.com/t/mediapipe/121120?u=_av
https://github.com/AndreV84/mediapipe#Docker-hand-usb-cam-GPU-example