Organizing/ structuring rule sets and rules

[LibraSun]

Already posted as a PR with much-simplified rendering of underlying formula.

[toggledbits]

OK. It may or may not make today's build. I'm still testing other parts of my parser changes.

[LibraSun]

Here's an odd response, to an expression defined (in 21085) as:

testStr = [  "cat", "  dog", "idle  ", "  jump  "  ],
trim(testStr)

// result = (string) "cat, dog,idle , jump"
// note (a) the lopsided leftover spaces, (b) concatenation into a single string, (c) type coercion into String
// unsure whether trim() ought even accept Array as input?

[LibraSun]

Meanwhile, this expression is returning the original string when I actually use a Regex as the "split" term:

testStr = "Th3 sw1ft br0wn c4t!",
split (testStr,"1",2)

// returns ["Th3 sw","ft br0wn c4t!"]
// expected

testStr = "Th3 sw1ft br0wn c4t!",
split (testStr,"\d",2)

// returns ["Th3 sw1ft br0wn c4t!"]
// unexpected

[LibraSun]

Another to ponder:

testObj = {animal:"cat",planet:"earth",size:"big"},
k = keys(testObj),
each key in k:
v = testObj.key

produces an empty array [] rather than (the expected) ["cat","earth","big"]. (Won't parse at all using .[key] btw.)

[toggledbits]

The correct syntax here is testObj[key]

Basically, [] and . are nearly equivalent member accessors. In JavaScript and lexpjs you can, for example, use array.0 to refer to the first element of an array, same as array[0]. The difference between them is that the interior of [] can be an expression, where . must only have an identifier on its right-hand side. So when you testObj[key], it will use the value of key for the member access. It would also be the case that testObj.animal and testObj['animal'] are equivalent.

Your earlier split (testStr,"\d",2) example doesn't work because "\d" is parsed as escape plus d, which is simply the letter d, which is not what you want. You want a literal backslash followed by a d, which would be the string "\\d" to get the RegExp character class for digits 0-9. This is also the way JavaScript would parse it and behave:

> t="The quick brown f0x"
'The quick brown f0x'
> t.match( "\d" )
null                      <---- No match!
> t.match( "\\d" )
[ '0', index: 17, input: 'The quick brown f0x', groups: undefined ]
>

[LibraSun]

Ah, typical of me not to have known the precise array['key'] syntax offhand.
Yay, that variant works as advertised. (You were supposed to be out having fun today!)
And thanks for the pro tip re: "slash"-escaping the right way.
I was thrown off by this one working as expected (even without the double slash construct):

testStr = "Does this work?",
match(testStr,"\st")

// yields " t"

[toggledbits]

Fun!? I'm Clonezilla-ing everyone's desktops, and retiring my old Linux server that has served as the main NAS, moving the Synology into that role. Lots of services to move (which has been done over time, but the last-minute cut-overs for some things that couldn't), etc. Busy busy!

The other example, "\s" is an unrecognized escape sequence so it's leaking it. I'll fix that. "\s" is what's required.

[toggledbits]

This post is deleted!

[LibraSun]

This one is only funny because it's true:

indexof=["indexOf(indexof,indexOf)!"],
indexOf(indexof,"indexOf")!=indexof

This one is just plain funny:

HA={ha:'HA'},
ha={HA:HA['ha']},
HA={ha:ha.HA}

Result: ["ha":"HA"]

And yet these are both uncharacteristically false?

[[[]]]===[[[]]]
[1]===[1]

despite both sides having length 1 and being of the same Type?
So is this (false); a bit surprising but that kinda makes sense with Objects...

{a:1}=={a:1}

Seriously though, I do have a slight concern about this evaluating to FALSE when it is axiomatically and mathematically TRUE:

sin(1)/cos(1)==tan(1)
// false, due to rounding in the 16th decimal place!

[toggledbits]

@librasun said in Expressions and LuaXP Functions:

And yet these are both uncharacteristically false?
[[[]]]===[[[]]]
[1]===[1]

Comparing objects (arrays are objects) is basically meaningless because you are actually trying to determine if they are the same object (i.e. identical types and sizes of data that occupy the same storage in RAM). They are not. JavaScript produces the same result (not surprisingly). This is a common error new programmers make, but it's language-dependent and can bite people making transitions between. Some languages will make deep comparisons of objects/arrays, but most that I've worked with do not. Python stands out as the exception for modern, and Lisp going back in my history. But it's not intrinsic in C, C++, Java, or JavaScript, for example (Java does it as methods of classes, so that's an explicit implementation, not a native language feature on intrinsic types).

@librasun said in Expressions and LuaXP Functions:

Seriously though, I do have a slight concern about this evaluating to FALSE when it is axiomatically and mathematically TRUE:
sin(1)/cos(1)==tan(1)
// false, due to rounding in the 16th decimal place!

Another common error programmers make: comparing floats directly for equality. Remember that all numbers in our computers are represented as integers. In the case of a floating point number, the typical representation is (simplifying a bit) some number of bits for a mantissa and some number of bits for an exponent. The number of bits for each limits the precision and range of what can be represented. So some rounding on the edges and beyond of the precision of the representation is normal and expected. It is always an error in programming to test if a floating point result is equal to another, therefore. One better/correct way to do it see if the absolute value of the difference between the two floats is very small, e.g. abs( ( sin(1)/cos(1) ) - tan(1 ) ) < 0.000001. Sometimes you will also need to consider the magnitude of the operands, as this effects the precision of the subtraction's result.

The other factor is that the implementation of sin(), cos() and tan() (and all similar) will have inherent precision limitations and errors (there are many methods, with tables for interpolation being a very common, fast approach, but not big on accuracy). Woe unto the structural or aerospace engineer who does not understand the implementation and limitations of the libraries on which his computation programs are based.

So as a software engineer, I would never expect that statement to be true on any computer, although as you say, it should be when read axionatically and mathemetically. Every result a computer produces is an approximation. The question how good the approximation is, and whether that's within the acceptable margin of error for the purpose.

[LibraSun]

Very good takeaways to bear in mind, thanks!

[toggledbits]

Expressions have a lot of power, as you've often pointed out, but they also remove some of the shielding that the rest of the interface has from system-specifics, and add a big learning curve. I'm grateful for all the hard poking and questions you are asking here, as these things are either bugs or facts that need to be memorialized in the documentation and these forums.

By the way, you had mentioned in the test script for lexpjs the Xxpect error, and I replied that it was intentional because there was no object/array comparison by which the test script could compare a complex object/array result with a complex array/object expectation. That has now been remedied in the test script; I don't know if you noticed. You motivated me to get that done, especially since we were accumulating too many results that required manual review, and manual review is in itself error-prone. I suppose I could port that into lexpjs for equality, at least on objects/arrays; I can't really do much about floating point comparison, because they can be much more complex. Maybe lexpjs needs a "~=" operator that means approximately equals? That would take some careful definition... not sure how much use it would get...

[LibraSun]

Tough call. I would say no to new features like that unless and until a recognizable minority of users requests them, which I doubt will ever happen. Diminishing returns on your time.

I also doubt most people will be writing complex expressions of the types I've been testing lately, but I was merely trying to flush out any incipient engine problems.

I'm glad these submissions have been equal parts entertaining and instructive, rather than an annoyance.

[toggledbits]

Agreed. I think that comparison of objects would be rarely used. Mostly, it seems, the expression language is used for access data in structures (like HTTP responses and entity attributes), and moving that around; simple calculations (like random delays) seem to be common; and finally, although I haven't seen it come up in MSR, is going to be time-series stuff. I've not seen a case in either MSR or RFV for object/array comparison.

Now, float equality, yes, I have seen that, but generally speaking, they have been involved in magnitude comparisons (< and > with or without =), and that makes the round-off errors less consequential. So agree no pressing need there, either.

To reiterate, everything you have thrown at me here is valuable, turning either into documentation changes, or a bug fix and an additional regression test.

[LibraSun]

PRO TIP (inspired by preceding reply):

WHEN PERFORMING EQUALITY TESTS among Float (high-precision) Expressions (variables) in MSR, use this format:

1. First, create a Global Expression such as epsilon with value 0.001 (or any smaller positive value sufficiently close to 0);
2. Rather than the test (A==B), where at least one of the variables is a Float, use (abs(A-B)<epsilon) instead.

This avoids the chance of a phony FALSE due to mantissa rounding by the calculation engine.

[LibraSun]

QUICK CONVERSION FROM DECIMAL (number) TO BINARY (string)

If your workflow demands a binary representation (e.g. you're using Switchboard plug-in's bit-masking features to control multi-switch status) and you really want to see 0's and 1's instead of MSR's always-decimal output, try this Expression on for size:

binConvert :=

b=13,
res=[1&b>>3,1&b>>2,1&b>>1,1&b],
join(res,"")

Result: 1101

NOTE: Here I've used only 4 "bits" but the concept generalizes to arbitrarily many bits, to accommodate larger inputs.

BONUS: if you want to round-trip a binary string into its decimal numeric equivalent, this Expression will get you there:

decConvert :=

B="001101",
binStr="0b"+B,
0b111111 & binStr

Result: 13 (decimal)

In this case, I've chosen 6 bits, but you could employ more or less to suit the length of your input string.

[LibraSun]

@toggledbits , I notice that the expression

A = [1,2,3], push(A,A)

does not generate an error (it should, since push(a,b) expects a non-object in b), nor does it generate a result. Just limbo.

[toggledbits]

This will fall into the categories of "don't do this" and "will not be fixed". The reason is objects (which arrays are) pass by reference so that (in more typical usage) push(A, 55) can fulfill its definition: modifying A in place by appending 55 to it. An attempt to fix it would result in having to pass copies of the arguments (pass by value), which means the first argument then is not A, but rather a copy of A, and then push() is modifying the copy in place and not the original array, so the original array never changes, defeating the purpose of the function.

[LibraSun]

Forging ahead with more newfangled expressions...
@toggledbits what would you expect from:

a=["a","b","c"], b=["d","e"],
each val in b: push(a,val)

My money was on:
["a","b","c","d","e"]
but instead I got:
[["a","b","c","d","e"],["a","b","c","d","e"]]

EDIT: Ah, here's the magic sauce:

a=["a","b","c"], b=["d","e"],
each val in b: c = push(a,val), c

Result: ["a","b","c","d","e"]