@toggledbits @akbooer Sorry, I will try to be clearer.
" I don't know what "assigning the luup log function to a table" means"

In many of Patrick's plugins, he uses a global lua table to hold plugin data and functions. He assigns shorthand names that call common luup functions, then stores those truncated names in the table. Then the first class functions are called from the table. I believe it is this abstraction that is contributing to log entries that show nil values, as direct calls of "luup.log" do not show nil values. The technique should work in openluup, and most often it does, but sometimes it doesn't.

Here's an example from the log where it does not work:

2022-01-15 14:13:12.551   luup.variable_set:: 258.urn:micasaverde-com:serviceId:HaDevice1.CommFailure was: 0 now: 0 #hooks:0
2022-01-15 14:13:12.551   luup.variable_set:: 258.urn:micasaverde-com:serviceId:HaDevice1.CommFailureTime was: 0 now: 0 #hooks:0
2022-01-15 14:13:12.551   luup_log:149: Reactor: Starting nil (#nil)
2022-01-15 14:13:12.551   luup.variable_set:: 149.urn:toggledbits-com:serviceId:Reactor.Message was: Starting Unused1 now: Starting Basic Functions #hooks:0
2022-01-15 14:13:12.552   luup.variable_set:: 265.urn:toggledbits-com:serviceId:ReactorSensor.Invert was: nil now:  #hooks:0
2022-01-15 14:13:12.552   luup.variable_set:: 265.urn:toggledbits-com:serviceId:ReactorSensor.Invert was:  now: nil #hooks:0

In the log entry "luup_log:149: Reactor: Starting nil (#nil)" the text of the log entry is there "Reactor: Starting", but the variable following the text is nil. This is what I meant by "picking up on variables". The log entries are being displayed, but anything that is a variable is showing as nil. The variables themselves are not actually nil because I can see from program execution that they are doing the job that they're supposed to be doing. It is only in the log entries that they show as nil, at least insofar as what is visible to me.

As for using nil to delete a variable, that seems to be understood and is separate from the log problem.

I believe that this is a scope problem, thus the title of the post. But I don't believe this is a reactor scope problem in that the same issue does not occur on the Vera itself. So for whatever reason, in openLuup, the scope of the variable being displayed in the log is different from the scope of the log function--that the value of the variable is somehow hidden from the function.

This is a problem as it relates to the readability of the log, an annoyance more than anything, so I have no expectations to have it fixed. In my original screenshots, you can see multiple log entries that display as nil, so that's why I brought this to your attention as I felt that if there are scope issues at work with the logging function, then it could be happening elsewhere as well.

The only other thing that I would add is that in my startup lua, I call a module in the _G space so that I can access functions in the lua section of Reactor activities. My call is _G.myStartUpLua = require("myStartUpLua") I don't believe that this should have an effect on the log, but perhaps it does.

@therealdb Yes, that's where I'm headed too. Agreed that Hass is very stable. It's just that there are real structural changes being made all the time, and before I sink my time into learning the details of the software, I want to be sure that pretty much everything is final in the engine.

@toggledbits Yes, I think that's a large part of the problem as I'm seeing the same behavior with deleted groups in various sensors. The sensor groups flop between nil and empty in the attempt of trying to delete the variable.

There seems to be a separate issue of the openLuup log not picking up on variables for your various plugins log entries.

I'm guessing this has to do with assigning the luup log function to a table and then calling a truncated name ie L(...) when writing to the log. So there is some scope issue going on that is different on openLuup from that of a Vera, brought about by a level of abstraction.

@therealdb Well that's a shame. One more nail in the coffin. I'll probably move to HA as soon as the various versions of the software are consolidated. It's a great platform, but for my needs, it is still too subject to breaking structural changes.

@toggledbits OK. Does that mean that the log entries as shown above are expected ie. that the flopping (from nil to empty and back again) for reactor expression variables is routine?

@akbooer Yes, I tried browser refresh without any change. If you look at the log, you see the variables flopping from nil to empty and then back again to nil.

@therealdb are you seeing anything like this in your openLuup reactor installation? I'm only seeing this nil behavior with Reactor expression variables.

Hi Patrick/AK Booer

I am able to see Reactor expressions in the AltUI UI per the below and the expressions work as they should in my reactor sensors. When an expression changes, the reactor sensor responds accordingly.

However, I cannot see the expressions in luup state variables or the luup logs:

I tried reinstalling openLuup (latest development) and when that failed to change the noted behavior, I reinstalled lua5.1. There was also no change. Rebooting the machine also produced no change.
I suspect this is an openLuup issue as I also see nil values for some plugins:

And other plugins are fine:

2022-01-12 20:47:12.027   luup.variable_set:: 63.urn:upnp-micasaverde-com:serviceId:Weather1.CurrentDewPoint was: 32 now: 29.7 #hooks:0
2022-01-12 20:47:12.028   luup.variable_set:: 63.urn:upnp-micasaverde-com:serviceId:Weather1.WindSpeed was: 1.76 now: 2.8 #hooks:0
2022-01-12 20:47:12.028   luup.variable_set:: 63.urn:micasaverde-com:serviceId:HumiditySensor1.CurrentLevel was: 30 now: 27 #hooks:0
2022-01-12 20:47:12.028   luup.variable_set:: 65.urn:micasaverde-com:serviceId:HumiditySensor1.CurrentLevel was: 30 now: 27 #hooks:0
2022-01-12 20:47:12.028   luup.variable_set:: 63.urn:upnp-micasaverde-com:serviceId:Weather1.LastUpdate was: 1642047430 now: 1642049231 #hooks:0

Any ideas on how to troubleshoot this.....

@akbooer Ok, that's clear enough. Thanks

@buxton Yeah, running lua within the Reactor plugin/MSR is an incredibly powerful feature that, combined with expressions, pretty much allows you to do whatever you want to do. A wonderful piece of software.

Now I need to figure out how to construct the closure that will surround the watch callback function. Tricky, because there are so many variables that go into deciding when the routine should branch, and when branching should be avoided.

@akbooer OK thanks, that's what I needed to know.

One final question if you could. How does scope affect a watch called from start-up lua? I'm asking because I imagine the scope for a watch is plugin specific, and therefore is confined to the plugin.

The watch will be created in the initialization code of a global module, that is in turn referenced in start-up lua ie. "_G.myStartUpLua = require("myStartUpLua") "

This module contains functions that are called from Reactor activities similar to this:

Will the global watch callback function be visible to the watch if both are created within "myStartUpLua".

@akbooer Yes, I understand the callback and how it is triggered. The question is once a watch is established (in this case during startup lua), can it be removed. If so, how is that done. Secondly, what happens when you re-run the code that places a watch on a specific variable. Do the watches stack up in memory, or is the previous watch simply replaced by the new call. Or perhaps the new watch is ignored. I don't know as I can't find any documentation on how this works.

I only want the callback triggered under specific conditions ie. there are certain conditions where the watched variable should not trigger the callback routine-- such as when zones are opening and closing when there is no intention to arm the security panel. If I can't remove a watch and then re-add it later, I'm guessing that I have to account for those conditions within callback function. Is that correct?

Hi AK,

I can't find any info on how a variable watch actually ends.

I have the following variable watch on an alarm panel plugin UPnP variable. The code is part of some extended startup lua:

luup.variable_watch("waitForZoneToClose","urn:micasaverde-com:serviceId:EVL3VistaAlarmPanel1", "FaultedZoneNames", 7)

In the same startup lua, the global function "waitForZoneToClose" is called when the "FaultedZoneNames" variable changes, but this same function is also called at the start of a routine that determines what zones are actually faulted on the alarm panel.

openZoneCount= waitForZoneToClose(7, "urn:micasaverde-com:serviceId:EVL3VistaAlarmPanel1", "FaultedZoneNames" , "","")

So although the luup watch variables are included in the function params, they are not used within the function:

function waitForZoneToClose(dev_id, service, variable, old_val, new_val)
  openZoneCount = getOpenZones()	--retrieves open zone names on panel, and zones to bypass per mode setting
        
   if openZoneCount > 0 then  -- there are open zones
      buildReservedZonesTable()
            
     -- run routine to pop bypass and vent zones from the zonesToClose stack
      removeReservedZones()
      openZoneCount = tablelength(tZonesToClose)  -- run the count again to see if any zone was popped off the stack during removeReservedZones
   end
   return openZoneCount
end

Because I run the function recursively ie. when the variable that holds the open zones changes, I rerun the function that detects open zones, and if any relevant zones are still open, the code for the same watch is ran again, to detect if any zones have changed status, this before trying to auto arm the panel. So there might be a situation where two or more zones need to be closed before the panel will arm, and it takes real time to physically close each zone, one by one, and as each closes, the callback would trigger again.

Hope that makes sense.

I wasn't clear if a variable watch ends when the watched variable changes and the code then proceeds. Or do you have to end the watch via code somehow? I was concerned about a possible condition where watch upon watch would be added to memory until a luup reload would clear the watches, or do the watches end and self-cleanup when the callback function is called.

Allowing the MQTT service to be created as a container, or couple the MQTT service directly with the MSR container would be great. With the exception of openLuup, I don't install anything on my HA server that is not containerized. The reason being that dependencies get corrupted, or changed by another app, or services conflict, etc..... Containers eliminate all that chaos and allow me to take down any software that's not working, all the while not destabilizing the underlying server.

Also, I use MSR primarily for Vera/openLuup and Home Assistant. It appears that the connection to HA is a websocket, allowing HA to async updates to MSR, whereas the connection to Vera is REST based relying on polling. Due to underlying issues with the Lua socket library, I try to keep polling to an absolute minimum so I have MSR turned off at the moment. What would be really cool is a plugin that would reside on openLuup or a Vera that would catch any and all changed states, and then push those state changes to MSR, perhaps with a synchronous acknowledgement. This complicates MSR of course, as you would need to install two separate pieces of software, however, I think the benefits of a push model, outweigh the downside of maintaining two separate apps. On the other hand, Vera is not long for this world, so perhaps the effort is not worth it.

@fanan Per this topic https://smarthome.community/topic/744/sending-a-service-request-to-home-assistant?_=1638829172919 I couldn't get Alexa tts to work in MSR, but I do have it working in openLuup with the following code that sits inside a simple plugin:

myEchoDevice = "alexa_media_".. myEchoDevice
myEchoMessage = table.concat(myEchoMessage)
	
local request_body = json.encode {message = myEchoMessage, data = {["type"] = "announce",method = "all"}}

local response_body = {}
	
local theURL = 'http://'..HomeAssistantIP..':'..HomeAssistantPort..'/api/services/notify/'..myEchoDevice  
	
r, c, h = http.request {
   url = theURL,
   method = "POST",
   headers = {
   ["Content-Type"]   = "application/json",
   ["Authorization"] = "Bearer ".. HomeAssistantToken,
   ["Content-Length"] = request_body:len()
   },
source = ltn12.source.string(request_body),
sink   = ltn12.sink.table(response_body)
}

The variable "myEchoDevice" is the exact name of the specific echo device (or device group) as presented in the Amazon Alexa app. I typically broadcast to all my devices via a group, intercom style, so I catch the message wherever I am in my home. The variable "HomeAssistantToken" needs to be created in HA. "myEchoMessage "is a lua table in the above code, but you could just as easily capture the message as a string, and then trim the string prior to sending it to the HTTP post request.

@3rdstng Ha! if you're in the US, I'll take it. I still have a veralite running in my garage that could do with an upgrade.

@a-lurker Yeah, HA takes a couple of minutes to fully start, so 30 seconds is the bare minimum I could get away with on my system. Not to mention the host server's atom processor, while parsimonious on power use, is a dog when ten containers are starting up from boot. Luckily, power outages here are infrequent. When prices are right, I'll get a home battery/solar setup to eliminate that rare nuisance.

Place in "/etc/cmh-ludl"

openLuup_run.sh

#!/bin/bash
echo "Starting openLuup server"
echo "To see tail of logfile: tail -f ./out.log"
cd /etc/cmh-ludl
sudo rm /etc/cmh-ludl/logs/out.log
nohup ./openLuup_reload >> /etc/cmh-ludl/logs/out.log 2>&1 &

openLuup_stop.sh

#!/bin/bash
echo "Stopping openLuup server"
echo "To see tail of logfile: tail -f ./out.log"
cd /etc/cmh-ludl
curl http://localhost:3480/data_request?id=exit

Then start the "openluup" service with:

$ sudo systemctl enable openluup
$ sudo systemctl start openluup

@librasun Good to know. I'm only out shipping costs if they don't as there's no way I have the time to de-solder these boards.

@akbooer It's just easier to understand the code/ what's occurring in a plugin as opposed to openLuup, mainly because of the typical plugin structure and its place on top of the openLuup subsystem. That's probably a trivial distinction to you, but for me it's a huge difference in understanding as I'm not in any way fluent with the inside baseball of the OLuup object model.

Per your MQTT comment, though it can be done (https://github.com/jziolkowski/tdm), it would be difficult to have functional device bridges if you took away MQTT, as you'd have to interrogate a router in some fashion to find out if these devices exist on one's network, and then use http commands to control the devices. So I see MQTT handling as more an innate part of the OL subsystem, like http, than a device profile like tasmota or shelly, device profiles that are typically handled as stand alone plugins.

@librasun This is great to know. I have a box of these switches that failed from power outages. In each case, it seems a capacitor failed after power was turned back on, at least as far as I could tell from looking at the swollen cases. I didn't measure microfarads though as I did not want to start unsoldering components...

Will they take back switches without a receipt as some of mine are ancient?

@a-lurker There's a systemd example in the openLuup user guide. Here's my startup based on those directions

[Unit]
Description=openLuup and AltUI Server for Vera 
After=network-online.target
After=sockproxy.service
Wants=network-online.target

[Service]
Type=forking
ExecStartPre=/bin/sh -c 'until ping -c1 <VeraIP>; do sleep 1; done;'
ExecStartPre=/bin/sleep 30
WorkingDirectory=/etc/cmh-ludl
ExecStart=/bin/bash /etc/cmh-ludl/openLuup_run.sh
ExecStop=/bin/bash /etc/cmh-ludl/openLuup_stop.sh
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

"Wants=network-online.target" makes sure the network is up. Then the ping command loops until a response is sent back from the vera. Followed by a 30 second sleep to allow everything else to get up and running. However, no vera up-- no openLuup as the ping does not time out.

Your router must allow ICMP between the vera and the OL host. Per the OL help file, you then need to create the .sh files and then activate the systemd service. I use Patrick's sock proxy for an app, so that command does not need to be in there if you don't use the proxy.