Moving MSR from a QNAP container to RP 5 - some issues

[toggledbits]

Fun!? I'm Clonezilla-ing everyone's desktops, and retiring my old Linux server that has served as the main NAS, moving the Synology into that role. Lots of services to move (which has been done over time, but the last-minute cut-overs for some things that couldn't), etc. Busy busy!

The other example, "\s" is an unrecognized escape sequence so it's leaking it. I'll fix that. "\s" is what's required.

[toggledbits]

This post is deleted!

[LibraSun]

This one is only funny because it's true:

indexof=["indexOf(indexof,indexOf)!"],
indexOf(indexof,"indexOf")!=indexof

This one is just plain funny:

HA={ha:'HA'},
ha={HA:HA['ha']},
HA={ha:ha.HA}

Result: ["ha":"HA"]

And yet these are both uncharacteristically false?

[[[]]]===[[[]]]
[1]===[1]

despite both sides having length 1 and being of the same Type?
So is this (false); a bit surprising but that kinda makes sense with Objects...

{a:1}=={a:1}

Seriously though, I do have a slight concern about this evaluating to FALSE when it is axiomatically and mathematically TRUE:

sin(1)/cos(1)==tan(1)
// false, due to rounding in the 16th decimal place!

[toggledbits]

@librasun said in Expressions and LuaXP Functions:

And yet these are both uncharacteristically false?
[[[]]]===[[[]]]
[1]===[1]

Comparing objects (arrays are objects) is basically meaningless because you are actually trying to determine if they are the same object (i.e. identical types and sizes of data that occupy the same storage in RAM). They are not. JavaScript produces the same result (not surprisingly). This is a common error new programmers make, but it's language-dependent and can bite people making transitions between. Some languages will make deep comparisons of objects/arrays, but most that I've worked with do not. Python stands out as the exception for modern, and Lisp going back in my history. But it's not intrinsic in C, C++, Java, or JavaScript, for example (Java does it as methods of classes, so that's an explicit implementation, not a native language feature on intrinsic types).

@librasun said in Expressions and LuaXP Functions:

Seriously though, I do have a slight concern about this evaluating to FALSE when it is axiomatically and mathematically TRUE:
sin(1)/cos(1)==tan(1)
// false, due to rounding in the 16th decimal place!

Another common error programmers make: comparing floats directly for equality. Remember that all numbers in our computers are represented as integers. In the case of a floating point number, the typical representation is (simplifying a bit) some number of bits for a mantissa and some number of bits for an exponent. The number of bits for each limits the precision and range of what can be represented. So some rounding on the edges and beyond of the precision of the representation is normal and expected. It is always an error in programming to test if a floating point result is equal to another, therefore. One better/correct way to do it see if the absolute value of the difference between the two floats is very small, e.g. abs( ( sin(1)/cos(1) ) - tan(1 ) ) < 0.000001. Sometimes you will also need to consider the magnitude of the operands, as this effects the precision of the subtraction's result.

The other factor is that the implementation of sin(), cos() and tan() (and all similar) will have inherent precision limitations and errors (there are many methods, with tables for interpolation being a very common, fast approach, but not big on accuracy). Woe unto the structural or aerospace engineer who does not understand the implementation and limitations of the libraries on which his computation programs are based.

So as a software engineer, I would never expect that statement to be true on any computer, although as you say, it should be when read axionatically and mathemetically. Every result a computer produces is an approximation. The question how good the approximation is, and whether that's within the acceptable margin of error for the purpose.

[LibraSun]

Very good takeaways to bear in mind, thanks!

[toggledbits]

Expressions have a lot of power, as you've often pointed out, but they also remove some of the shielding that the rest of the interface has from system-specifics, and add a big learning curve. I'm grateful for all the hard poking and questions you are asking here, as these things are either bugs or facts that need to be memorialized in the documentation and these forums.

By the way, you had mentioned in the test script for lexpjs the Xxpect error, and I replied that it was intentional because there was no object/array comparison by which the test script could compare a complex object/array result with a complex array/object expectation. That has now been remedied in the test script; I don't know if you noticed. You motivated me to get that done, especially since we were accumulating too many results that required manual review, and manual review is in itself error-prone. I suppose I could port that into lexpjs for equality, at least on objects/arrays; I can't really do much about floating point comparison, because they can be much more complex. Maybe lexpjs needs a "~=" operator that means approximately equals? That would take some careful definition... not sure how much use it would get...

[LibraSun]

Tough call. I would say no to new features like that unless and until a recognizable minority of users requests them, which I doubt will ever happen. Diminishing returns on your time.

I also doubt most people will be writing complex expressions of the types I've been testing lately, but I was merely trying to flush out any incipient engine problems.

I'm glad these submissions have been equal parts entertaining and instructive, rather than an annoyance.

[toggledbits]

Agreed. I think that comparison of objects would be rarely used. Mostly, it seems, the expression language is used for access data in structures (like HTTP responses and entity attributes), and moving that around; simple calculations (like random delays) seem to be common; and finally, although I haven't seen it come up in MSR, is going to be time-series stuff. I've not seen a case in either MSR or RFV for object/array comparison.

Now, float equality, yes, I have seen that, but generally speaking, they have been involved in magnitude comparisons (< and > with or without =), and that makes the round-off errors less consequential. So agree no pressing need there, either.

To reiterate, everything you have thrown at me here is valuable, turning either into documentation changes, or a bug fix and an additional regression test.

[LibraSun]

PRO TIP (inspired by preceding reply):

WHEN PERFORMING EQUALITY TESTS among Float (high-precision) Expressions (variables) in MSR, use this format:

1. First, create a Global Expression such as epsilon with value 0.001 (or any smaller positive value sufficiently close to 0);
2. Rather than the test (A==B), where at least one of the variables is a Float, use (abs(A-B)<epsilon) instead.

This avoids the chance of a phony FALSE due to mantissa rounding by the calculation engine.

[LibraSun]

QUICK CONVERSION FROM DECIMAL (number) TO BINARY (string)

If your workflow demands a binary representation (e.g. you're using Switchboard plug-in's bit-masking features to control multi-switch status) and you really want to see 0's and 1's instead of MSR's always-decimal output, try this Expression on for size:

binConvert :=

b=13,
res=[1&b>>3,1&b>>2,1&b>>1,1&b],
join(res,"")

Result: 1101

NOTE: Here I've used only 4 "bits" but the concept generalizes to arbitrarily many bits, to accommodate larger inputs.

BONUS: if you want to round-trip a binary string into its decimal numeric equivalent, this Expression will get you there:

decConvert :=

B="001101",
binStr="0b"+B,
0b111111 & binStr

Result: 13 (decimal)

In this case, I've chosen 6 bits, but you could employ more or less to suit the length of your input string.

[LibraSun]

@toggledbits , I notice that the expression

A = [1,2,3], push(A,A)

does not generate an error (it should, since push(a,b) expects a non-object in b), nor does it generate a result. Just limbo.

[toggledbits]

This will fall into the categories of "don't do this" and "will not be fixed". The reason is objects (which arrays are) pass by reference so that (in more typical usage) push(A, 55) can fulfill its definition: modifying A in place by appending 55 to it. An attempt to fix it would result in having to pass copies of the arguments (pass by value), which means the first argument then is not A, but rather a copy of A, and then push() is modifying the copy in place and not the original array, so the original array never changes, defeating the purpose of the function.

[LibraSun]

Forging ahead with more newfangled expressions...
@toggledbits what would you expect from:

a=["a","b","c"], b=["d","e"],
each val in b: push(a,val)

My money was on:
["a","b","c","d","e"]
but instead I got:
[["a","b","c","d","e"],["a","b","c","d","e"]]

EDIT: Ah, here's the magic sauce:

a=["a","b","c"], b=["d","e"],
each val in b: c = push(a,val), c

Result: ["a","b","c","d","e"]

[LibraSun]

While you're here...

testArr = [1,2],
first m in testArr with m <= 2
Result: (number) 1 // correct results since 21089+

[LibraSun]

@toggledbits , I see the immediately-above situation has been rectified with 21090, so I'll edit accordingly.

I also see that you've instantiated some of the array- and text-handling functions you mentioned earlier, so THANK YOU!

For example:

concat( [1] , [2] ) now yields [1,2]
slice( [1,2,3] , 1 , 2 ) now yields [2,3]
pop( [1,2,3] ) now yields 3
push( [1,2,3] , 5 ) now yields [1,2,3,5]
shift( [1,2,3] ) now yields 1
unshift( [1,2,3] , 5 ) now yields [5,1,2,3] // NOTE: currently only permits a single addend; for instance, unshift( [1],2,3,4 ) results in [2,1] not [2,3,4,1]!

[LibraSun]

This one has me scratching my head a bit:

b = [ ] , each i in a = [ 1 , 2 ] : do push( b , i ) , push( b , i ) done

// yields [[1,1,2,2],[1,1,2,2]]
// expecting [1,1,2,2]

while this modified version:

b=[],each i in a=[1,2]: do push(b,i) done

// yields [[1,2],[1,2]]
// expecting [1,2]

[toggledbits]

The result of push is an array.

[LibraSun]

@toggledbits said in Expressions and LuaXP Functions:

The result of push is an array.

But if you

push ( [ ] , 1 )

shouldn't you get [ 1 ]? And not [ [ 1 ] , [ 1 ] ]?

[toggledbits]

It depends on what the complete expression is. Remember that the value of each is every non-null expression result during the iteration. So an array of arrays is a very possible response.

[LibraSun]

@toggledbits Unless I'm mis-reading the way each works, I had anticipated the underlying interpretation of:

b=[],each i in a=[1,2]: do push(b,i) done

to be as follows:
"Start with an empty array named b. For each item in the array named a, starting with 1, append that value into array b. First, you would get [ 1 ], and the next iteration, using the 2 from a, append that to b to yield [ 1 , 2 ]."

Any I missing an important piece of the puzzle? (I do recognize that do/done is not necessary here, but I left it in for direct comparison with the expression immediately preceding.)

Thanks. Sorry to give you grief over these operations.