openweather plugin ?

[therealdb]

@archers crazy days at work. push is in progress.

yes, this was from my own logs (where I attached owntracks as well - via a mqtt realy I have to test things around that's pushing mqtt messages from my prod system to my test ones).

[ArcherS]

@therealdb I tried to buid again from the new files, but I cannot get the Luup.MqttBridge to run the way it did before the update.
Unclear what is wrong, I think I did everything in the same was as before. I noticed that the icon on the Luup.MqttBridge file is different when looking at it in Ubuntu "Files".

//ArcherS

[therealdb]

@archers what's the output of dotnet publish -c Release -r linux-x64? You can do it under Windows and it will produce executables for Linux as well.

If you want just a single file, go with

dotnet publish -c Release -r linux-x64 -p:PublishSingleFile=true --self-contained true

this will produce a single file with everything inside. I'm updating the doc as well.

[therealdb]

@rogero @ArcherS @rafale77 I just published a compiled version for Linux-64. It's under releases. No SDK need to be installed, just chmod and you're ready to go.

Release Version 0.30 · dbochicchio/luup-mqtt

First compiled release. Just for Linux-x64. Build for Windows or Raspberry using .NET 5 SDK.

If you're running Raspberry and need ARM-64, let me know and I could add it compiled as well.

[ArcherS]

@therealdb thanks for the build!

When using your latest build the Luup.MqttBridge file for some reason is a "shared library (application/x-sharedlib)" and not as when building from the older version "executable (application/x-executable)" despite having set permissions.
Really strange, I managed to get around this by creating a .sh file containing "./Luup.MqttBridge" and running this.
Over first hurdle...

However when I launch the new built file I still only get this in the log file:

[19:17:23 INF] [MQTTServer] Broker is starting v 0.30.210109 on port 1883...

It is as if it hangs while trying to start/starting.

In the Tasmota console it says connected so the user/passwd seems ok:

19:17:21 MQT: Connected

So obviously not quite there yet.
Any ideas on what the pronlem could be?

//ArcherS

[therealdb]

@archers post your config and I'll try to replicate. Maybe there's some error not correctly surfacing.
Plus, post a couple of messages you are seeing on the console.

[ArcherS]

So I did some more testing and found out that I had still "DVES_%06X" as client in the Tasmota device...
I am sure I had tripple checked everything, stupid...

So I changed this to what it should be and now I get it to work , i.e. the device in OpenLuup gets the temp value. Success!

However the log does still not give more that the starting message for some reason.
Any ideas on this? Btw I still have this in the appsettings, not sure if that could be related:?

/* remove comment when you're OK with less logs
	"Serilog": {
		"MinimumLevel": {
			"Default": "Information",
			"Override": {
				"Microsoft": "Fatal",
				"System": "Fatal"
			}
		}
	}
	*/

I will do some more testing later on and see.
Thanks for the assistance!

//ArcherS

[therealdb]

@archers I'll triple check logs tomorrow. I have a new build ready, specifically supporting raw values to be pushed to devices (ie: owntracks).

[ArcherS]

@therealdb I noticed that the bridge had gone down over night. I restarted it and got the below error message after approx 10-15 minutes when it went down again. It ran for longer yesterday, maybe 1-2 hours before crashing.

I now started the old version (that works, it was of course my error on the Client in the Tasmota that was the problem), just to see if it also will go down.

//ArcherS

[08:55:53 INF] [MQTTServer] Broker is starting v 0.30.210109 on port 1883...
Unhandled exception. System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 10 seconds elapsing.
 ---> System.TimeoutException: The operation was canceled.
 ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
 ---> System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
 ---> System.Net.Sockets.SocketException (125): Operation canceled
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token)
   at System.Net.Http.HttpConnection.FillAsync(Boolean async)
   at System.Net.Http.HttpConnection.ReadNextResponseHeaderLineAsync(Boolean async, Boolean foldedHeadersAllowed)
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at Microsoft.Extensions.Http.Logging.LoggingHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at Polly.Retry.AsyncRetryEngine.ImplementationAsync[TResult](Func`3 action, Context context, CancellationToken cancellationToken, ExceptionPredicates shouldRetryExceptionPredicates, ResultPredicates`1 shouldRetryResultPredicates, Func`5 onRetryAsync, Int32 permittedRetryCount, IEnumerable`1 sleepDurationsEnumerable, Func`4 sleepDurationProvider, Boolean continueOnCapturedContext)
   at Polly.AsyncPolicy`1.ExecuteAsync(Func`3 action, Context context, CancellationToken cancellationToken, Boolean continueOnCapturedContext)
   at Microsoft.Extensions.Http.PolicyHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at Polly.Bulkhead.AsyncBulkheadEngine.ImplementationAsync[TResult](Func`3 action, Context context, Func`2 onBulkheadRejectedAsync, SemaphoreSlim maxParallelizationSemaphore, SemaphoreSlim maxQueuedActionsSemaphore, CancellationToken cancellationToken, Boolean continueOnCapturedContext)
   at Polly.AsyncPolicy`1.ExecuteAsync(Func`3 action, Context context, CancellationToken cancellationToken, Boolean continueOnCapturedContext)
   at Microsoft.Extensions.Http.PolicyHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at Microsoft.Extensions.Http.Logging.LoggingScopeHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.SendAsyncCore(HttpRequestMessage request, HttpCompletionOption completionOption, Boolean async, Boolean emitTelemetryStartStop, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpClient.SendAsyncCore(HttpRequestMessage request, HttpCompletionOption completionOption, Boolean async, Boolean emitTelemetryStartStop, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.GetStringAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
   at Luup.MqttBridge.Services.LuupWrapper.RunCommandAsync(Int32 deviceID, String action, String actionCommand, String commandParameter, String command, String serviceID, String value)
   at Luup.MqttBridge.Services.LuupWrapper.UpdateVariablesAsync(Int32 deviceID, String serviceID, String variableName, String value)
   at Luup.MqttBridge.Services.Mqtt.MQTTServer.ProcessMessageAsync(MqttDevice device, MqttApplicationMessage applicationMessage)
   at Luup.MqttBridge.Services.Mqtt.MQTTServer.<>c__DisplayClass9_0.<<StartAsync>b__3>d.MoveNext()
--- End of stack trace from previous location ---
   at System.Threading.Tasks.Task.<>c.<ThrowAsync>b__140_1(Object state)
   at System.Threading.QueueUserWorkItemCallbackDefaultContext.Execute()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
/home/user/mqttbridge/LuupMqttBridge.sh: line 1: 20380 Aborted                 (core dumped) ./Luup.MqttBridge

[therealdb]

Yep, I noticed on my own version as well. It's because the HTTP endpoint crashed (are you using openLuup, right?) or timedout after 10 secs. I'll probably make it configurable in next version and be more aggressive towards crashes.
It's still very beta, even if I'm running a modified version in production for my own.

New release coming up on GitHub in minutes

[ArcherS]

@therealdb yep, I am on OpenLuup.
I tested and the old version also crashes in the same way after 30-60 minutes.

//ArcherS

[therealdb]

@archers are you able to compile it yourself, for an easy fix? I have a new version ready, but I'll need more debug in a real world scenario. But I'm ready to push it, if you can try. I also added support for alternate topic (useful if you want to intercept two mqtt messages with different topics and payload, but execute the same action) and wildcard support for clienid (useful if you have messages coming from other brokers and the original clientid is lost).

[ArcherS]

@therealdb I was able to build the previous versions, so I can try the new version if you push it.

[therealdb]

@archers ok, it's on GitHub. Lemme know if it's working for you and I'll produce a compiled build (and I'll also automate it in the next days to automagically do it for me).

[ArcherS]

@therealdb new version built and started, I will report back how it goes.

[ArcherS]

@therealdb I have now had the updated Mqtt Bridge running continuously for little more than 25 hours. So far it is still alive and kicking.

The log file is now working I get entries in the log, which I did not get before. This makes it easier to see what is happening of course.
So when the bridge starts I get:

2021-01-30 11:24:16.055 +01:00 [INF] [MQTTServer] Broker is starting v 0.31.210130 on port 1883...
2021-01-30 11:28:40.877 +01:00 [INF] [MQTTServer] Updating #216 - urn:upnp-org:serviceId:TemperatureSensor1 / CurrentTemperature - 24.5
2021-01-30 11:33:40.770 +01:00 [INF] [MQTTServer] Updating #216 - urn:upnp-org:serviceId:TemperatureSensor1 / CurrentTemperature - 24.9
etc...

At times I get the following error, I assume it is related to the http issue:

2021-01-31 11:38:40.852 +01:00 [INF] [MQTTServer] Updating #216 - urn:upnp-org:serviceId:TemperatureSensor1 / CurrentTemperature - 25.5
2021-01-31 11:38:50.854 +01:00 [ERR] [LuupWrapper] RunCommandAsync: http://127.0.0.1:3480/data_request?id=variableset&output_format=json&DeviceNum=216&serviceId=urn:upnp-org:serviceId:TemperatureSensor1&Variable=CurrentTemperature&Value=25.5&RunAsync=1
System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 10 seconds elapsing.
 ---> System.TimeoutException: The operation was canceled.
 ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
 ---> System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
 ---> System.Net.Sockets.SocketException (125): Operation canceled
   --- End of inner exception stack trace ---
   at void System.Net.Sockets.Socket+AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at int System.Net.Sockets.Socket+AwaitableSocketAsyncEventArgs.GetResult(short token)
   at async ValueTask System.Net.Http.HttpConnection.FillAsync(bool async)
   at async ValueTask<ArraySegment<byte>> System.Net.Http.HttpConnection.ReadNextResponseHeaderLineAsync(bool async, bool foldedHeadersAllowed)
   at async Task<HttpResponseMessage> System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, bool async, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at async Task<HttpResponseMessage> System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, bool async, CancellationToken cancellationToken)
   at async ValueTask<HttpResponseMessage> System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, bool async, bool doRequestAuth, CancellationToken cancellationToken)
   at async ValueTask<HttpResponseMessage> System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, bool async, CancellationToken cancellationToken)
   at async Task<HttpResponseMessage> Microsoft.Extensions.Http.Logging.LoggingHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at async Task<TResult> Polly.Retry.AsyncRetryEngine.ImplementationAsync<TResult>(Func<Context, CancellationToken, Task<TResult>> action, Context context, CancellationToken cancellationToken, ExceptionPredicates shouldRetryExceptionPredicates, ResultPredicates<TResult> shouldRetryResultPredicates, Func<DelegateResult<TResult>, TimeSpan, int, Context, Task> onRetryAsync, int permittedRetryCount, IEnumerable<TimeSpan> sleepDurationsEnumerable, Func<int, DelegateResult<TResult>, Context, TimeSpan> sleepDurationProvider, bool continueOnCapturedContext)
   at async Task<TResult> Polly.AsyncPolicy<TResult>.ExecuteAsync(Func<Context, CancellationToken, Task<TResult>> action, Context context, CancellationToken cancellationToken, bool continueOnCapturedContext)
   at async Task<HttpResponseMessage> Microsoft.Extensions.Http.PolicyHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at async Task<TResult> Polly.Bulkhead.AsyncBulkheadEngine.ImplementationAsync<TResult>(Func<Context, CancellationToken, Task<TResult>> action, Context context, Func<Context, Task> onBulkheadRejectedAsync, SemaphoreSlim maxParallelizationSemaphore, SemaphoreSlim maxQueuedActionsSemaphore, CancellationToken cancellationToken, bool continueOnCapturedContext)
   at async Task<TResult> Polly.AsyncPolicy<TResult>.ExecuteAsync(Func<Context, CancellationToken, Task<TResult>> action, Context context, CancellationToken cancellationToken, bool continueOnCapturedContext)
   at async Task<HttpResponseMessage> Microsoft.Extensions.Http.PolicyHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at async Task<HttpResponseMessage> Microsoft.Extensions.Http.Logging.LoggingScopeHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at async ValueTask<HttpResponseMessage> System.Net.Http.HttpClient.SendAsyncCore(HttpRequestMessage request, HttpCompletionOption completionOption, bool async, bool emitTelemetryStartStop, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
   at async ValueTask<HttpResponseMessage> System.Net.Http.HttpClient.SendAsyncCore(HttpRequestMessage request, HttpCompletionOption completionOption, bool async, bool emitTelemetryStartStop, CancellationToken cancellationToken)
   at async Task<string> System.Net.Http.HttpClient.GetStringAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
   at async Task Luup.MqttBridge.Services.LuupWrapper.RunCommandAsync(int deviceID, string action, string actionCommand, string commandParameter, string command, string serviceID, string value)
2021-01-31 11:43:40.851 +01:00 [INF] [MQTTServer] Updating #216 - urn:upnp-org:serviceId:TemperatureSensor1 / CurrentTemperature - 25.5
2021-01-31 11:48:40.836 +01:00 [INF] [MQTTServer] Updating #216 - urn:upnp-org:serviceId:TemperatureSensor1 / CurrentTemperature - 25.6
etc...

After the above error it goes back to reporting updates.
The error comes quite random, at times once every approx 20-30 minutes, and at times it can be a few hours between them.

So far I have only one test device in OpenLuup getting values from the bridge, I do not know how adding some 50+ topics would affect the overall load and stability.

Edit: It could be worth mentioning that I today have some 50 devices in OpenLuup that are getting their values over http; some SiteSensors a bunch of virtual sensors getting data from Tasmota/Shelly devices etc. In other words there is a bit of activity already going on through the http interface today. Moving most of them to Mqtt would perhaps not increase the load at all, who knows.

Looking promising, thanks for the effort so far with the bridge!

[therealdb]

I have mine running with almost 20 virtual devices and it's rock solid. It's pushing to vera as well.

My forked version has the ability to update the value only if a difference threshold is met (useful to avoid too many requests to the http interface). I'll try to backport it here in the coming days.

As I said, I'm using it to push things from a lot of different things (tasmota, shelly, my windows app, miflora).

I'll add an http endpoint soon to publish messages as well.

[ArcherS]

@therealdb that sounds reassuring, thanks!

As a side note to anyone venturing into the world of mqtt I found Mqtt Explorer which seems to be a pretty nice tool for looking at messages, topics etc.
It can even create graphs for values to track them over time.

[therealdb]

@archers said in MQTT Plugin:

@therealdb that sounds reassuring, thanks!

is this still working for you?

[ArcherS]

@therealdb yep, it is still working, still only one test device receiving data.
I get the same error related to the http issue from time to time as shown in the log above, but it recovers and continue updating the device.