Reply
Emerging Member
Posts: 46
Registered: ‎01-08-2015
Kudos: 11
Solutions: 1

Anybody wants a MQTT client on the mPower?

I wonder, if there are people who would like to see MQTT enabled on the mPower (potentially more mFi) devices.

I have the basics running but need some scripting support now.

 

MF.v2.1.11# ./mosquitto_sub -h 192.168.1.20 -v -t home/testtopic
home/testtopic sendmesomething
Emerging Member
Posts: 49
Registered: ‎03-11-2016
Kudos: 17

Re: Anybody wants a MQTT client on the mPower?

Very much interested.

Did you end up compiling a binary for it?

Emerging Member
Posts: 46
Registered: ‎01-08-2015
Kudos: 11
Solutions: 1

Re: Anybody wants a MQTT client on the mPower?


@peturdainn wrote:

Very much interested.

Did you end up compiling a binary for it?


Indeed I compiled mosquito and put the binaries on the mPower.

Its working good/fast/reliable so far. Let me know if you are keen to BETA test.

Emerging Member
Posts: 77
Registered: ‎10-16-2012
Kudos: 14
Solutions: 1

Re: Anybody wants a MQTT client on the mPower?

Hi,

 

I have a Home Assistant install, and 1 (old) mFi mPower 3-port EU laying arround. I tried connecting it to Home Assistant (as it supports mFi), but I need a mFi controller up and running. I am not looking in spending more money into mFi, and setting up a controller (and maintaining it) for 1 outlet is just a little bit too much.

 

I decided to have a look at the mFi forums anyway, but I don't think UBNT is actively developing any mFi products?

 

I have no MQTT devices connected to my Home Assistant yet, but my Home Assistant install is only a couple of weeks old. I am planning to try out some MQTT devices.

 

How can I help you with this? 

Emerging Member
Posts: 46
Registered: ‎01-08-2015
Kudos: 11
Solutions: 1

Re: Anybody wants a MQTT client on the mPower?

Thanks for raising your hand!

How can you help?

Basically I need testing support and people who test/document the integration into various Home Automation Software (I can only cover openHAB)

Well, one possibility would be:

Make MQTT work with your Home Assistant with some MQTT device.
If that works fine, integrate your mPower device.
For that you need to install my (BETA) MQTT binaries/script on the mPower.
Give feedback about the installation process and about success/flaws/whatever.
Write a short guide how to connect Homeassistand and mPower/MQTT

Emerging Member
Posts: 49
Registered: ‎03-11-2016
Kudos: 17

Re: Anybody wants a MQTT client on the mPower?

I have two mPower-3 (EU) in active use, with an HTTP hack to integrate it in my node-red setup.

 

So testing your MQTT beta it would mean using it from node-red via a broker running on yet another device.

 

I admit I once tried to compile some stuff myself for the mPower but ran out of time figuring out dependency problems (the details are lost in time), so if you're willing to share your build setup, I'm all ears!

Emerging Member
Posts: 77
Registered: ‎10-16-2012
Kudos: 14
Solutions: 1

Re: Anybody wants a MQTT client on the mPower?

I will help you testing this, and write a short guide about what I did.

 

Make MQTT work with your Home Assistant with some MQTT device.

I will do this, but it will take a couple of weeks (just starting with Home Assistant, and first need to connect my existing devices). I will need to learn about MQTT (only read some basic stuff about it), and need to find some device.

 

If you explain to me how to install the binaries/script on the mPower, I will try it right after I got a MQTT working on Home Assistant.

Emerging Member
Posts: 46
Registered: ‎01-08-2015
Kudos: 11
Solutions: 1

Re: Anybody wants a MQTT client on the mPower?

Sent you both a private message

Emerging Member
Posts: 49
Registered: ‎03-11-2016
Kudos: 17

Re: Anybody wants a MQTT client on the mPower?

received! Thanks!

 

will try tonight.

Emerging Member
Posts: 49
Registered: ‎03-11-2016
Kudos: 17

Re: Anybody wants a MQTT client on the mPower?

So far I tested the publishing only, that works (didn't check all details)

 

I also tried to build it myself, which also worked - except my binaries are much bigger and buildroot options are set to strip and gcc flag -o3

 

Used buildroot 2017.08 and the lib comes in at about 70K (compared to your 49K).

Any idea?

 

I'm now coding some nodered to visualize the data Man Happy

 

Emerging Member
Posts: 49
Registered: ‎03-11-2016
Kudos: 17

Re: Anybody wants a MQTT client on the mPower?

I'm changing the way the MQTT payload is created, currently experimenting with something like this:

 

while sleep $refresh; 
do 
    echo "{ \
\"label\":[ \
\""`cat /etc/persistent/cfg/config_file | grep 'port.0.label' | cut -d '=' -f 2`"\", \
\""`cat /etc/persistent/cfg/config_file | grep 'port.1.label' | cut -d '=' -f 2`"\", \
\""`cat /etc/persistent/cfg/config_file | grep 'port.2.label' | cut -d '=' -f 2`"\"], \
\"output\":[ \
\""`cat /proc/power/output1`"\", \
\""`cat /proc/power/output2`"\", \
\""`cat /proc/power/output3`"\"], \
\"power\":[ \
\""`cat /proc/power/active_pwr1`"\", \
\""`cat /proc/power/active_pwr2`"\", \
\""`cat /proc/power/active_pwr3`"\"] \
}" | /var/etc/persistent/mqtt/mosquitto_pub -h $mqtthost -t $topic/sensors -s
done

which generates a JSON that looks like this:

 

{ "label":[ "port 1 label", "port 2 label", "port 3 label"], "output":[ "0", "0", "0"], "power":[ "0.0", "0.0", "0.0"] } 

which is processed in node-red in a functionblock, for example to split this into three messages containing respective power values:

 

var a = JSON.parse(msg.payload)
var msg1 = { topic:a.label[0] , payload:a.power[0] };
var msg2 = { topic:a.label[1] , payload:a.power[1] };
var msg3 = { topic:a.label[2] , payload:a.power[2] };
return [msg1,msg2,msg3];

The output of the above node-red functionblock can be connected directly to three gauges who will be happy to show the values

 

 

So the main difference is that I'm getting the data directly from /proc/power and /etc/persistent/cfg/config_file which feels a bit faster in execution.

 

 

thoughts welcome!

Emerging Member
Posts: 46
Registered: ‎01-08-2015
Kudos: 11
Solutions: 1

Re: Anybody wants a MQTT client on the mPower?

Hey!

 

glad that you have fun with it Icon Wink

 

regarding the payload ... yes at the beginning I also thought about building the JSON myself. Then I saw the out-of-the-box sensors.cgi.

 

I see a few advantages with sensors.cgi:

* it already implements the differences between 1/3/6/8 port mPower devices

* it reads the hardware config and provides all sensors. I guess that also works with the mPort devices (I don't own one)

* the JSON format is usable with JSONPATH (which is good for openhab)

 

On the downside:

* Performance .... yes you are right. I guess sensors.cgi perform some more IO operations

* JSONPATH might not fit to any needs (as I can see with your node-red experiments)

 

I'm a bit afaid we won't make the perfect universal payload. So we might end up in making the payload configureable (a few options such as default(JSONPATH), node-red, single values (no JSON at all but multiple topics)

Emerging Member
Posts: 49
Registered: ‎03-11-2016
Kudos: 17

Re: Anybody wants a MQTT client on the mPower?


glad that you have fun with it Icon Wink

Very! Icon Lol

 

To be honest, the output of sensors.cgi is also quite easy to parse in node-red.

 

What led me to try something else is that I didn't need all that data, my only interest is the label, the relay state and the current power.

 

But I'm not happy with my solution either Man Wink

Stuff I plan to do:

- store port count and labels in variables

- generate JSON based on port count, probably going to add the portcount in there too for generic parsing

- provide a way to trigger faster updates for a period of time (use case: when sending a command to switch a relay, output should be more responsive)

- make the refresh count configurable via MQTT sub

 

Will post the scripts when I have it running Man Very Happy

 

Emerging Member
Posts: 49
Registered: ‎03-11-2016
Kudos: 17

Re: Anybody wants a MQTT client on the mPower?

Is the number of outlets stored somewhere (other than detecting the port5 string doesn't exist if there are only 3)?

If not, I may just create the static labelname vars until I reach an empty string and use that as portcount

 

current progress:

(man my shell scripting could use more exercise)

#create the label variables only once (static content)
for i in 0 1 2
do
	TMPLABEL=$(cat /etc/persistent/cfg/config_file | grep 'port.'${i}'.label' | cut -d '=' -f 2)
	eval LABEL"$i"='${TMPLABEL}'
done

while sleep $refresh; 
do 
	for i in 0 1 2
	do
		eval OUTPUT"$i"=`cat /proc/power/output$((i+1))`
		eval POWER"$i"=`cat /proc/power/active_pwr$((i+1))`
	done

    echo "{\"label\":[\""${LABEL0}"\",\""${LABEL1}"\",\""${LABEL2}"\"],\"output\":[\""${OUTPUT0}"\",\""${OUTPUT1}"\",\""${OUTPUT2}"\"],\"power\":[\""${POWER0}"\",\""${POWER1}"\",\""${POWER2}"\"]}" | /var/etc/persistent/mqtt/mosquitto_pub -h $mqtthost -t $topic/sensors -s
done

Will probably change the payload again to be able to create it in the for loop. Oh well

 

Emerging Member
Posts: 49
Registered: ‎03-11-2016
Kudos: 17

Re: Anybody wants a MQTT client on the mPower?

[ Edited ]

another evening of tinkering, this is going to be it:

 

#!/bin/sh
export LD_LIBRARY_PATH=/var/etc/persistent/mqtt
refresh=60

while test $# -gt 0; do
        case "$1" in
                -h|--help)
                        echo ""
                        echo "Ubiquiti Networks mPower MQTT publisher"
                        echo " "
                        echo "options:"
                        echo "-h, --help                show brief help"
                        echo "-host=HOSTNAME            MQTT host"
                        echo "-t=TOPIC                  MQTT topic"
                        exit 0
                        ;;
                -host)
                        shift
                        if test $# -gt 0; then
                                mqtthost=$1
                        else
                                echo "no host specified"
                                exit 1
                        fi
                        shift
                        ;;
                -t)
                        shift
                        if test $# -gt 0; then
                                topic=$1
                        else
                                echo "no topic specified"
                                exit 1
                        fi
                        shift
                        ;;

                -r)
                        shift
                        if test $# -gt 0; then
                                refresh=$1
                        else
                                echo "no refresh time specified"
                                exit 1
                        fi
                        shift
                        ;;
                *)
                        break
                        ;;
        esac
done

if [ -z "$mqtthost" ]; then
    echo "no host specified"
    exit 0
fi

if [ -z "$topic" ]; then
    echo "no topic specified"
    exit 0
fi

# create the label variables (array) only once (static content) while counting ports
# note that port count detection will fail if one or more ports have empty name(s)

LASTPORT=0
for i in $(seq 0 1 5)
do
    TMPLABEL=$(cat /etc/persistent/cfg/config_file | grep 'port.'${i}'.label' | cut -d '=' -f 2)
    if [ -z "${TMPLABEL}" ]
    then
        break
    else
        # valid label found
        if [ $i -ne 0 ]
        then
            MQTTSTR_FIXEDSTART=$MQTTSTR_FIXEDSTART","
        fi
        MQTTSTR_FIXEDSTART=$MQTTSTR_FIXEDSTART"\""$TMPLABEL"\""
        LASTPORT=$i
    fi
done

#massage this message a bit: add portcount before and colon after
MQTTSTR_FIXEDSTART="{\"portcount\":"$((LASTPORT+1))",\"label\":["$MQTTSTR_FIXEDSTART"]"

echo Found $((LASTPORT+1)) ports, starting MQTT pub now...

while sleep $refresh; 
do 
    # for readability: start each payload with the fixed part
    MQTTSTR=$MQTTSTR_FIXEDSTART

    # variable part 1: output state (0/1)
    MQTTSTR=$MQTTSTR",\"output\":["
    for i in $(seq 0 1 $LASTPORT)
    do
        OUTPUT=`cat /proc/power/output$((i+1))`
        if [ $i -ne 0 ]
        then
            MQTTSTR=$MQTTSTR","
        fi
        MQTTSTR=$MQTTSTR$OUTPUT
    done
    MQTTSTR=$MQTTSTR"]"
    #end of variable part 1

    # variable part 2: power consumption 
    MQTTSTR=$MQTTSTR",\"power\":["
    for i in $(seq 0 1 $LASTPORT)
    do
        POWER=`cat /proc/power/active_pwr$((i+1))`
        POWER=`printf "%.3f" $POWER` if [ $i -ne 0 ] then MQTTSTR=$MQTTSTR"," fi MQTTSTR=$MQTTSTR$POWER done MQTTSTR=$MQTTSTR"]" #end of variable part 2 # expand here with more variable data if needed # end of the payload string MQTTSTR=$MQTTSTR"}" echo $MQTTSTR | /var/etc/persistent/mqtt/mosquitto_pub -h $mqtthost -t $topic/sensors -s done

The payload looks like this:

{"portcount":n, "label":["label1",...,"labeln"],"output":[output1,...,outputn],"power":[power1,...,powern]}

 

Improvements over previous versions:

1) tries to detect the number of ports by detecting empty labels

2) the portcount and label parts of the MQTT payload are created once at startup

3) power is truncated to mW (3 digits after the decimal point)

4) adding more variables is trivial

5) each variable is an array containing as much elements as there are ports detected, so node-red parsing is still as simpe as

var a = JSON.parse(msg.payload)
var msg1 = { topic:a.label[0] , payload:a.power[0] };
var msg2 = { topic:a.label[1] , payload:a.power[1] };
var msg3 = { topic:a.label[2] , payload:a.power[2] };
return [msg1,msg2,msg3];

So what is the advantage? Efficiency.

 

Where the showsensors output generated a payload of nearly 800 bytes, the above code produces only 121 bytes.

 

Remarks welcome!

Emerging Member
Posts: 46
Registered: ‎01-08-2015
Kudos: 11
Solutions: 1

Re: Anybody wants a MQTT client on the mPower?

Hey,

nice process man!

I think we have to learn if we find the perfect JSON that fits every need. Hoping that Taigar will test Home Assistant integration soon and can provide some feedback about JSON.

Looking forward with the goal of "releasing" this stuff...

 

1) How did the installation work for you? Did you try or did you just copy the files manually?

 

2)
>provide a way to trigger faster updates for a period of time
I thought about this as well. But we would need some shell inter-process communication because SUB and PUB are running in different processes and are independent.


Which brings me to another question:

We are currently using shell scripting. But we have proven that we can compile custom code for the mPower. What about coding a real programm (C/C++/?) for the MQTT adapter? That would provide quite some new possibilities. But I guess it is quite some effort. (And I'm not a C/C++ programmmer)

 

3) I'd suggest that we make the code flexible (parameter): either your mini-JSON or the out of the box full version. I'll do that in the next days.

Emerging Member
Posts: 49
Registered: ‎03-11-2016
Kudos: 17

Re: Anybody wants a MQTT client on the mPower?

There is no need to pick one or the other, my shell script can be the 'lite' solution or for tinkerers Man Wink

 

Regarding your questions:

1) I copied them manually. Then again, I also use my own compiled the binaries...

 

2) Let me think about this, I'll figure something out Man Happy

I do like the fact that the mPort specific stuff is shell scripts and not C/C++, it's way easier for people to adapt for their cause and if it works, it works. And that's me talking as C developer Man Wink

 

3) as my first remark. Could be two install scripts.

 

If you're doing work in the install, also provide an uninstall script in case people want it removed (selling device, not happy with MQTT or the MQTT turning out to misbehave in the long run)

 

 

Emerging Member
Posts: 49
Registered: ‎03-11-2016
Kudos: 17

Re: Anybody wants a MQTT client on the mPower?

[ Edited ]

while working on the faster updates I came up with the setup to use one parent script that parses the commandline, and then fires both pub and sub scripts, since I assume users will run both anyway

 

So my next iteration may touch a bit more code Man Very Happy

 

edit: got most of it running, including single argument parsing and IPC to do faster updates, need to test more before publishing. First get some sleep

Emerging Member
Posts: 49
Registered: ‎03-11-2016
Kudos: 17

Re: Anybody wants a MQTT client on the mPower?

so here's another iteration of my take on the scripts Man Wink

 

Changed:

- mqrun.sh parses the arguments, stores the data in environment variables, and starts mqpub.sh and mqsub.sh as subshells

- mqsub.sh requests faster updates by mqpub.sh when something was received

 

That last feature is done through a temp file in /tmp (so in RAM),

- mqsub.sh write into it how many times it wants faster updates

- mqpub.sh wakes up every second, checks the tempfile

  - if a value was inside it will count down the value and publish every second it wakes up

  - if the file was empty, it will count down the refresh setting every second until it is time to publish

 

 

comments welcome

 

As an afterthought, I could also just dump this on github, choice is a good thing Man Very Happy

Attachment
Emerging Member
Posts: 49
Registered: ‎03-11-2016
Kudos: 17

Re: Anybody wants a MQTT client on the mPower?

mqrun.sh didn't have the full path so that didn't go well when starting from rc.poststart...

 

I have now deployed this to two switches for testing over a longer time, with a node-red dashboard that shows power usage with gauges, and has switched that show output state, and respond to clicks for on/off switching

 

Let's run this for  1+ week(s) and see how it goes

 

attaching what I deployed (binaries are from my 2017.08 buildroot)

Attachment
Reply