Scene number?

[parkerc]

Hi,

I’m sure this is not a unique question about Vera (& Lua 5.1) and I’ve seen this reference in place,, but I can’t seem to find a fix to make an https download request work..

To give you an example, the code below aims to download 2 different files, the http call works fine, returning a 200 code, but the https one doesn’t, it doesn’t return anything ?

Please could some highlight what I’m missing?

print("--------DOWNLOAD http---------")

local http = require("socket.http")
local body, code = http.request("http://pbs.twimg.com/media/CCROQ8vUEAEgFke.jpg")
print(code)
if not body then error(code) end
local f = assert(io.open('mnt/nas/webtest1.jpg', 'wb')) -- open in "binary" mode
f:write(body)
f:close()

print("--------DOWNLOAD https---------")

local https = require "ssl.https" --luasec
local body, code = https.request("https://u.cubeupload.com/jbcooper/16146313918060.jpg")
print(code)
if not body then error(code) end
local f = assert(io.open('mnt/nas/webtest2.jpg', 'wb')) -- open in "binary" mode
f:write(body)
f:close()

[akbooer]

Are you sure?

Using this version of your code:

print("--------DOWNLOAD http---------")

local http = require("socket.http")
local reply,code, headers, status = http.request("http://pbs.twimg.com/media/CCROQ8vUEAEgFke.jpg") 
print (pretty {length = #reply, code = code, headers = headers, status = status})

print("--------DOWNLOAD https---------")

local https = require "ssl.https" --luasec
local reply,code, headers, status = https.request("https://u.cubeupload.com/jbcooper/16146313918060.jpg") 
print (pretty {length = #reply, code = code, headers = headers, status = status})

gives me this output, showing that lots of bytes have been received...

 ––– 51 lines ––– 6.4 ms –––

--------DOWNLOAD http---------
{
  code = 200,
  headers = {
    ["accept-ranges"] = "bytes",
    ["access-control-allow-origin"] = "*",
    ["access-control-expose-headers"] = "Content-Length",
    age = "43902",
    ["cache-control"] = "max-age=604800, must-revalidate",
    connection = "close",
    ["content-length"] = "129373",
    ["content-type"] = "image/jpeg",
    date = "Tue, 20 Sep 2022 10:44:48 GMT",
    ["last-modified"] = "Fri, 10 Apr 2015 23:57:23 GMT",
    server = "ECS (lhb/63B8)",
    ["server-timing"] = "x-cache;desc= HIT,x-tw-cdn;desc=VZ,edge;dur=8",
    ["strict-transport-security"] = "max-age=631138519",
    ["surrogate-key"] = "media media/bucket/1 media/586679904554455041",
    ["timing-allow-origin"] = "https://twitter.com, https://mobile.twitter.com",
    ["x-cache"] = "HIT",
    ["x-connection-hash"] = "90f51e90a99324db0f571d118a350f3b3c8b49e130d1ebea21aeafd14f3cafe6",
    ["x-content-type-options"] = "nosniff",
    ["x-response-time"] = "353",
    ["x-tw-cdn"] = "VZ, VZ"
  },
  length = 129373,
  status = "HTTP/1.1 200 OK"
}
--------DOWNLOAD https---------
{
  code = 200,
  headers = {
    ["alt-svc"] = 'h3=":443"; ma=86400, h3-29=":443"; ma=86400',
    ["cache-control"] = "private, max-age=14400",
    ["cf-cache-status"] = "MISS",
    ["cf-ray"] = "74d9f7cbac867729-LHR",
    connection = "close",
    ["content-type"] = "image/jpeg",
    date = "Tue, 20 Sep 2022 10:44:48 GMT",
    ["last-modified"] = "Tue, 20 Sep 2022 10:44:48 GMT",
    nel = '{"success_fraction":0,"report_to":"cf-nel","max_age":604800}',
    ["report-to"] = '{"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=Phpy9q3l6o0X7pS%2BXX%2Bf6hJZhALJSPRGCXLZai7Sk%2BwfQ%2F2AGHCxizvvqGtmEb9%2FpFAW0TTS1i1O%2FOHlYIVkqH7bwwJsy%2F1CFyc5qTbvBjjnzDjvDNpdhXex%2Bjb5xBBi3gtQ"}],"group":"cf-nel","max_age":604800}',
    server = "cloudflare",
    ["strict-transport-security"] = "max-age=63072000; includeSubdomains",
    ["transfer-encoding"] = "chunked",
    ["x-content-type-options"] = "nosniff",
    ["x-frame-options"] = "DENY"
  },
  length = 108321,
  status = "HTTP/1.1 200 OK"
}

[toggledbits]

Perhaps this is a question of platform.... if the HTTPS request is done on an older Vera (Lite, 3), its firmware doesn't support TLS 1.2 and the server and client may not be able to negotiate the encryption. This is also starting to manifest on Vera Plus, Edge, and Secure systems running 7.32 to servers requiring TLS 1.3. This is one reason so many plugins to remote APIs are dying off. Interestingly, the curl on 7.32 is built with a more modern encryption library than Vera's Lua ssl library, so using curl (e.g. with popen()) is often the (temporary) antidote.

@akbooer, did you test on openLuup? If that's the case, your OS and Lua packages are almost certainly more modern and capable than that of current (and past) Vera firmware.

[akbooer]

Yes, this was under openLuup. My only remaining Vera is still running UI5, so I’m pretty sure that it would fail there. As the OP indicated, this wasn’t uniquely about Vera, so I hadn’t thought it worth trying.

[parkerc]

Sorry, yes this is being done on my VeraPlus, with latest ‘production’ firmware 7.31..

When I run this, i get nothing back not even an error for the https call.

Print output
--------DOWNLOAD http---------     
{ status="HTTP/1.1 200 OK", length=129373, headers={ x-content-type-options="nosniff", x-cache="MISS", server-timing="x-cache;desc=MISS, x-tw-cdn;desc=FT", x-served-by="cache-lhr6621-LHR, cache-tw-ZZZ1", content-length="129373", strict-transport-security="max-age=631138519", access-control-allow-origin="*", x-tw-cdn="FT", access-control-expose-headers="Content-Length", content-type="image/jpeg", timing-allow-origin="https://twitter.com, https://mobile.twitter.com", accept-ranges="bytes", connection="close", date="Tue, 20 Sep 2022 13:31:56 GMT", cache-control="max-age=604800, must-revalidate", last-modified="Fri, 10 Apr 2015 23:57:23 GMT" }, code=200 }     
--------DOWNLOAD https---------    

[parkerc]

Is the lack of TLS 1.2 with Vera addressed with something like this, not sure how this works, but I assume there is a default if something is not specified ?

local _, code = https.request{
      url = url,
      protocol = "tlsv1_2",
     }
     ResetData()
     Log(" sent data status: " .. code)
return code

[toggledbits]

Maybe. There are ciphers and hashes within TLS 1.2 range, some of which are supported by 7.31 firmware, and some of which are not. Some less secure ones have been deprecated and many servers now exclude them.

Try formulating your request with curl on the (Vera) command line. If you can get that to work, then (a) the Lua ssl library is the culprit, and (b) you can use curl (with popen() or by reading a temporary output file) for your request.

[parkerc]

Thanks all..

Looks like curl is the way to go in situations like this..

Quick question in regards to being able to do this all in Lua on a VeraPlus, I came across the following bit of code (not sure of the origin, but i can see it specifies a certificate file, is this an option ?

require("socket")
local ssl = require("ssl")
local url = "test"

local params = {
  mode = "client",
  protocol = "tlsv1_2",
  cafile = "/path/to/downloaded/cacert.pem", --<-- added cafile parameters
  verify = "peer", --<-- changed "none" to "peer"
  options = "all",
}

local conn = socket.tcp()
conn:connect(url, 443)

-- TLS/SSL initialization
conn = ssl.wrap(conn, params)
conn:dohandshake()

conn:send("GET / HTTP/1.1\n\n")
local line, err = conn:receive()
print(err or line)
conn:close()

[toggledbits]

Maybe, maybe not. Why don't you just try it? You're not playing with launch codes here. If it fails, you get disappointed, but that's about all. I would not use "peer", that's more strict and you have no client side certificate to negotiate with, I'd go back to "none". It's possible the root CA's are a problem, but more likely, it's one of several problems, and there's no workaround for the others (except curl, and that's only temporary, since it isn't being updated either).

The cafile, by the way, does not specify a certificate for the client (Vera), it specifies a number of root CAs, the ones at the top of the chain that issue known-good certificates below them. They are used to validate the server's certificate (i.e. make sure it's signed by a trusted entity -- one of those root CAs), they are not used as a client cert for the Vera.

[parkerc]

Ok, not had much luck with the variations I tried,

One thing I did notice was that the luasec version on VeraPlus is v0.4 and the openssl version is v1.0.2

# openssl version
OpenSSL 1.0.2l  25 May 2017

Checking the luasec guidance here, https://github.com/brunoos/luasec/wiki/LuaSec-1.0.x , that VeraPlus openSSL version supports luasec v 1.0.x ?

Without being able to install/use Luarocks, any idea how to upgrade from luasec 0.4 to 1.0.x?