IQ4sh - Calculator for CLI or scripts -Testers wanted

[amigo]

I have a new project called IQ4sh or simply 'iq'. It's a calculator written to replace calls to 'bc', 'awk', etc., from within scripts. It also can be used from the CLI as a one-shot calculator or sourced into you shell session for interactive use anytime.

I need testers and folks eager to try something light and easy on your system. This calculator is not only ~for~ the shell -it's written in shell. It has been tested under dash, zsh, posh and bash, where it all works fine.

You can get iq from github at:
https://github.com/math-for-shell/iq4sh

I know that puppians first question will be: "Will it work with Puppy?" That's where I need the testers, as I can't answer that question. I don't know how posix-compatible the busybox shell is theses days(I know BB has several shells).

I started this project about 18 months ago because I wanted to try programming some 'real AI' from within the shell. I found only one example on github(https://github.com/justinstaines/neuralnetwork) -and it didn't really work 100%. It created a very simple neural-network with 2 inputs, 3 hidden nodes/neurons and a single output. But, I could see that it was learning. The trouble was that, on nearly every line of the script it was calling 'bc' or 'awk'. In 69 lines of code, it called bc 42 times and awk 4 times. Worse, since most of the script was a loop, so that running 10 iterations of 'learning', it called bc 420 times, awk 40 times (and 'cut' 9 times).

I thought to myself, surely there's a way to reduce or eliminate those external calls. Looking on the web for calculators written in the shell-language, I found fp.sh by Vidar Holen(github-koalaman), but it was not a usable calculator at all -but an impressive demo of what might be possible. I set out to write my own calculator -and it's not quite as easy as it might sound. In the end, IQ4sh is the result.
iq of course includes the basic operations of add, subtract, multiply and divide. But that doesn't really get us very far -when we want to calculate the 'logistic' or sigmoid function. So, iq also has functions for exp(), pow(), log(2/n/10), nthroot and many more. In a companion file called 'iq+' there are many functions which can be tried in conjunction with 'iq'.

A quick way to test if iq is working on your system is to run this command:
./iq pow -s5 3.1416 ^ -2.2
which should give this answer: 0.08058
Yes, it will take a second to return the answer, since the above problem involves quite a lot calculations. 'pow' is the slowest function of all and used as above it calls nearly every other function in iq.
Let's try another example:
./iq nthrt -s7 5 2.7182 ( 5th root of 2.7182 to precision=7)
returns: 1.2213954
And, to illustrate a bit more speed:
./iq mul -s20 1.61803398874989484820458683436 3.141592653589793238462643383279
which (quickly) returns: 5.08320369231525981580

After downloading or cloning, make the files iq and iq+ executable and run the program like this:
./iq liqhelp
to see the opening help page. There you'll see a listing of the available functions. Running './iq iqhelp funcname' will show the help for the name of the function you give(funcname).
If you want to try the more advanced functions from iq+, it can be run the same as iq:
./iq+ tanh_pade -s6 3.75 returns: 0.998762
There is no help for the functions from iq+, so you'll want to open the file and look over the functions to see what is there.
I hope that a few of you will try the program and let me know what you think of it, what you miss about it or what could be improved.
I have succesfully used iq to replace all the calls to bc and awk in the above-mentioned 'neural-network' project and started another couple of projects which put iq to the test.

[chris_r]

This is very impressive! I only had about half an hour to play with it, but the additioin, multiplication and division passed the common sense test. Using the pow() function with integer values seemed to work, but when I tried to take a square root using a pow x ^ 1/2, the values seemed to be about 10% off. For example ...

Code: Select all

 ./iq pow 2 0.5 

returned a value of 1.55563 instead of something close to 1.41 that I expected.

I tested with a 32 bit Bionic Pup.

Again, this is very impressive. Hopefully next week I can play with it some more.

[MochiMoppel]

@amigo Nice project and interesting code. I like the idea to use only shell functions, however I'm not sure I understand what you are aiming for.

The trouble was that, on nearly every line of the script it was calling 'bc' or 'awk'. In 69 lines of code, it called bc 42 times and awk 4 times. Worse, since most of the script was a loop, so that running 10 iterations of 'learning', it called bc 420 times, awk 40 times (and 'cut' 9 times).

Wouldn't this mean that instead of calling bc 420 times you now have to call iq 420 times? Or is it that 1 call to one of iq's special functions can replace multiple calls to bc?

If the latter is true then my small test with floating point division makes not much sense, but that's what I miss the most in bash and what sometimes forces my to use bc although I hate this tool.

Here some observations:
1) iq is much slower than bc. Of course that can't be a surprise but has to be taken into account when using iq.
2) iq becomes more than twice as fast when replacing its shebang from #!/bin/sh to #!/bin/ash. In this case it would make little sense to source the code into bash script because it might become as slow as the receiving script.
3) dc seems faster and probably is more powerful than bc but I have only the crippled busybox version which can't even set a precision level.
4) for my test I tried a for i in {00..10} loop to force zero padding for the first 10 digits. Strangely this didn't work in iq only for i=08 and i=09 .
5) iq and bc cut to a precision. Would be great if iq could round

Following test conducted with unmodified iq, i.e. with a iq sporting a #!/bin/sh shebang :

Code: Select all

#!/bin/sh
arg1=1234.56
arg2=75.1

echo -e "Code: iq div -sX $arg1 / $arg2 \n----------------"
time {
for i in {0..10}; do
{ echo -n "-s$i " ; iq div -s$i $arg1 / $arg2 ;}
done
}

echo -e "\n\nCode: echo \"scale=X; $arg1 / $arg2\" | bc \n----------------"
time {
for i in {0..10}; do
echo -n "scale=$i " ; echo "scale=$i; $arg1 / $arg2" | bc
done
}

echo -e "\n\nCode: busybox dc $arg1 $arg2 div p \n----------------"
time {
for i in {0..10}; do
busybox dc $arg1 $arg2 div p
done
}

echo -e "\n\nError when using iq in {00..10} loop \n----------------"
for i in {00..10}; do
{ echo -n "-s$i " ; iq  div -s$i $arg1 / $arg2 ;}
done

echo -e "\nNo problems with bc in {00..10} loop \n----------------"
for i in {00..10}; do
echo -n "-s$i " ; echo "scale=$i; $arg1 / $arg2" | bc
done

echo -e "\nCompare rounding (or lack of it) at 4 digits precision\n----------------"
echo -n '    iq:' ;iq div -s4 $arg1 / $arg2
echo -n '    bc:' ;echo "scale=4; $arg1 / $arg2" | bc
echo -n '    dc:' ;dc $arg1 $arg2 div p
echo -n 'printf:' ;printf '%.4f\n'  $(iq div -s10 $arg1 / $arg2)

Output:
----------------------

Code: iq div -sX 1234.56 / 75.1
----------------
-s0 16
-s1 16.4
-s2 16.43
-s3 16.438
-s4 16.4388
-s5 16.43888
-s6 16.438881
-s7 16.4388814
-s8 16.43888149
-s9 16.438881491
-s10 16.4388814913

real 0m0.787s
user 0m0.557s
sys 0m0.060s

Code: echo "scale=X; 1234.56 / 75.1" | bc
----------------
scale=0 16
scale=1 16.4
scale=2 16.43
scale=3 16.438
scale=4 16.4388
scale=5 16.43888
scale=6 16.438881
scale=7 16.4388814
scale=8 16.43888149
scale=9 16.438881491
scale=10 16.4388814913

real 0m0.087s
user 0m0.007s
sys 0m0.020s

Code: busybox dc 1234.56 75.1 div p
----------------
16.4389
16.4389
16.4389
16.4389
16.4389
16.4389
16.4389
16.4389
16.4389
16.4389
16.4389

real 0m0.041s
user 0m0.000s
sys 0m0.017s

Error when using iq in {00..10} loop
----------------
-s00 16.0
-s01 16.4
-s02 16.43
-s03 16.438
-s04 16.4388
-s05 16.43888
-s06 16.438881
-s07 16.4388814
-s08 /bin/iq: line 272: 4 + 08: value too great for base (error token is "08")
-s09 /bin/iq: line 272: 4 + 09: value too great for base (error token is "09")
-s10 16.4388814913

No problems with bc in {00..10} loop
----------------
-s00 16
-s01 16.4
-s02 16.43
-s03 16.438
-s04 16.4388
-s05 16.43888
-s06 16.438881
-s07 16.4388814
-s08 16.43888149
-s09 16.438881491
-s10 16.4388814913

Compare rounding (or lack of it) at 4 digits precision
----------------
iq:16.4388
bc:16.4388
dc:16.4389
printf:16.4389

[jamesbond]

@amigo, glad to have you back in the forum!

And with a nice project to boot! Looking forward to see the ANN code. When I first heard you say that (in another thread), I thought ... is this yet another project to connect to google/microsoft AI bots? But apparently it's not, you're planning to do it from scratch, with shell script no less

I'll do a quick test on Fatdog, we have bash by default here (as "sh"), but also busybox ash and dash. I will report back.

Wouldn't this mean that instead of calling bc 420 times you now have to call iq 420 times? Or is it that 1 call to one of iq's special functions can replace multiple calls to bc?

I would guess amigo plans to "source" the "iq" directly into the ANN and access the functions (exp(), pow(), etc) directly.

EDIT: some early tests. Wow it supports arbitrary precision also

Code: Select all

# time dash ./iq factorial 80
71569457046263802294811533723186532165584657342365752577109445058227039255480148842668944867280814080000000000000000000

real	0m1.015s
user	0m0.869s
sys	0m0.207s

# time sh ./iq factorial 80 # "sh" is "bash"
71569457046263802294811533723186532165584657342365752577109445058227039255480148842668944867280814080000000000000000000

real	0m3.536s
user	0m2.374s
sys	0m1.252s

Running it with busybox ash doesn't work. My version of ash is abit old, it's 1.27
# ash -x ./iq factorial 80
+ iqversion=1.61803398
+ EE=2.718281828459045235360287471352
+ ee=2.71828182845
+ PI=3.141592653589793238462643383279
+ pi=3.14159265358
+ PHI=1.61803398874989484820458683436
+ phi=1.61803398874
+ defprec=5
+ [ 1 !=  ]
+ cmd=factorial
+ shift
+ factorial 80
+ fact_out=80 fact=-1
+ [ -1 -gt 1 ]
+ echo 80
80
+ exit 0

[MochiMoppel]

I would guess amigo plans to "source" the "iq" directly into the ANN and access the functions (exp(), pow(), etc) directly.

Yes, that's very fast. However I get strange results starting from precision 6

Code: Select all

source=1
. /bin/iq

arg1=1234.56
arg2=75.1
time {
for i in {0..10}; do
{ echo -n "-s$i " ; div -s$i $arg1 / $arg2 ;}
done
}

-s0 16
-s1 16.4
-s2 16.43
-s3 16.438
-s4 16.4388
-s5 16.43888
-s6 16.-11838
-s7 16.2734029
-s8 16.10518367
-s9 16.899970325
-s10 16.-767590881

real 0m0.047s
user 0m0.037s
sys 0m0.000s

BTW: I assumed that [ANN] is a tag and stands for announcement.

[jamesbond]

BTW: I assumed that [ANN] is a tag and stands for announcement.

ANN is either Artificial Neural Network, or Annealed Neural Network. Not sure which one amigo has in mind.

[williams2]

I have a script named bc2

Code: Select all

#!/bin/ash
# usage: bc2 8/3*7+2
echo "scale=15;$@" | bc

It could be a shell function, of course. I prefer it as a shell script.

for example, showing that bc does not round:

Code: Select all

# bc2 2/3
.666666666666666
#

You could use python or perl.

Code: Select all

# python -c 'print(2.0/3)'
0.666666666667
#
# perl -e 'print 2/3'
0.666666666666667
#

I installed a cli rpn calculator from https://github.com/louisrubet/rpn (about 344k)

Code: Select all

# rpn 2 3 /
0.66666666666666666666666666666666666667
#

[MochiMoppel]

I'm still playing with the basic div function.
Strange things happen when using large integers:

Code: Select all

: Fast, result OK;   iq div 123456000000000         / 751   # 164388814913.44873
: Fast, result OK;   iq div 1234560000000000        / 751   # 1643888149134.48735
: Fast, result OK;   iq div 12345600000000000       / 751   # 16438881491344.8735
: Fast, result OK;   iq div 123456000000000000      / 751   # 164388814913448.73501
: >2sec, odd result; iq div 1234560000000000000     / 751   # 1604447403462050.599
: Infinite loop???;  iq div 12345600000000000000    / 751   # no result
: Fast, odd result;  iq div 123456000000000000000   / 751   # 100000000000000064.38881
: Fast, odd result;  iq div 1234560000000000000000  / 751   # 1000000000000000643.88814

Seems to be directly related to the integer limitations of the shell.
My bash shell (version 4.3.25) can only handle integers with a maximum value of 9223372036854775807 :

Code: Select all

echo $((9223372036854775806 /2 )) #4611686018427387903
echo $((9223372036854775807 /2 )) #4611686018427387903
echo $((9223372036854775808 /2 )) #-4611686018427387904
echo $((9223372036854775809 /2 )) #-4611686018427387903
echo $((9223372036854775810 /2 )) #-4611686018427387903

[amigo]

Thanks to everyone for chipping in. 'ANN' was meant as ANNOUNCE, but NN= neural-network. Made a bunch of pregress yesterday and will push changes today.
I'm not calling iq over and over for the NN. The whole thing is sourced into the NN script. Since it is all functions they stay loaded and available for use. This is how it gets some speed advantage over repeatedly calling externals.
Thanks very much for testing and showing concrete examples (of failure), so I can chase those down.

[amigo]

@MochiMopel, which 'ash' are you talking about. I'm guessing you mean busybox ash. I have been wondering if iq would run under any ash-variant besides dash.

[MochiMoppel]

@MochiMopel, which 'ash' are you talking about. I'm guessing you mean busybox ash.

Yes.

[amigo]

Please get updated iq and iq+. All the issues mentioned yesterday should be fixed -please try your tests again.
Also, to not keep you waiting anymore, there's a new repo at:
https://github.com/math-for-shell/ai4sh
with neural-network demo ( predict-able folder ) I mentioned.
I'll be gone a couple of hours and then try to address the questions, etc. from yesterday.

[amigo]

@chris_r, nice find -fixed.
@jamesbond -howdy fella -sniffing around the factorials already?
There used to be some math-guys around on murga -I don't remember the handles. Is anyone 'like that' around here? I'm still shopping for efficient algorithms to solve more or faster. For instance a better approximation for sin or cos... or loss-less range re/deduction for e^x and elsewhere...

[MochiMoppel]

Please get updated iq and iq+. All the issues mentioned yesterday should be fixed

Not all. 2 of the 3 issues I mentioned are not fixed.

1) Large numbers: Seem to be OK now.

2) value too great for base (error token is "08") not fixed. Apparently value is treated as octal. Stripping leading zeros should fix it.

3) Div function produces garbage results after being called a couple of times from within a script. I initially suspected that this was related to the scale value but is seems to be related to the number of times the function is called.
This script:

Code: Select all

#!/bin/sh
source=1
source iq

div  2 / 3
div  2 / 3
div  2 / 3
div  2 / 3
div  2 / 4
div  2 / 4
div  2 / 4
div  2 / 4

produced

Code: Select all

0.66666
0.66666
0.66666
0.-9713
0.79544
0.-2073
0.15683
0.20881

BTW: I suggest to rename the source variable to avoid confusion with the shell builtin of the same name.

[JakeSFR]

Hey, @amigo, welcome back!
I like (CLI) calculators, but I suck at math. Hmm, maybe that's why I like them...

Anyway:

3) Div function produces garbage results after being called a couple of times from within a script. I initially suspected that this was related to the scale value but is seems to be related to the number of times the function is called.

I think it's because the internal mod_pad variable is not initialized and it's growing in length after each iteration, when iq is sourced:

Code: Select all

# for i in {1..10}; do echo -n "$mod_pad ... "; div  2 / 3; done
 ... 0.66666
00000 ... 0.66666
0000000000 ... 0.66666
000000000000000 ... 0.-9713
00000000000000000000 ... 0.10605
0000000000000000000000000 ... 0.-2764
000000000000000000000000000000 ... 0.20910
00000000000000000000000000000000000 ... 0.27842
0000000000000000000000000000000000000000 ... 0.53491
000000000000000000000000000000000000000000000 ... 0.25647

Code: Select all

# for i in {1..10}; do mod_pad=; div  2 / 3; done
0.66666
0.66666
0.66666
0.66666
0.66666
0.66666
0.66666
0.66666
0.66666
0.66666

Greetings!

[amigo]

@MochiMoppel @JakeSFR thanks for hanging in there. The issues should be fixed now.

It was very quiet here for a couple of days. I thought maybe I had lost you all. I really appreciate the help you all provide. Most of the problems have been regressions because of major re-writes of functions just before starting to put stuff on github.

You guys might be interested to see a thread I started at LQ -particularly starting here:
https://www.linuxquestions.org/question ... ost6299752

@MM -regarding rounding, the predecessor to iq (called fp4sh) did rounding, but it's expensive as we must watch and compare current result with the last to see where repetition is happening -and how long of a repeating pattern should be rounded? Especially in AI calculations, we may have values very close to 1 -like 0.999999999902. fp4sh also had a mode where results could be padded when shorter -think of Currency formats. But including any of those things will greatly slow down every major function of iq.

The really big thing missing is being able to evaluate nested expressions like: ( 1 + (2/3) - .3). Don't hold your breath waiting for that to happen, though, as that would be very expensive. iq was conceived first for scripting -the interactive and CLI modes are an afterthought as a way to quickly test functionality without having to alter a driving script. If I know Puppians, someone is probably already working on a gtkdialog front-end for iq, which could add other functionality. Suggestions for inclusion of other stuff is appreciated, but we are watching our calories.

There is still a broader question about what to include in iq itself, and what should be relegated to iq+ -if at all. Your thoughts are welcome.

[JakeSFR]

@MochiMoppel @JakeSFR thanks for hanging in there. The issues should be fixed now.

Thanks! A similar problem with R_int:

Code: Select all

# src=1 . ./iq
# for i in {1..5}; do echo -n "$R_int ... "; div 5 r 2; done
 ... 2r1.0
1 ... 2r11.0
11 ... 2r111.0
111 ... 2r1111.0
1111 ... 2r11111.0
#
# for i in {1..5}; do R_int=; div 5 r 2; done
2r1.0
2r1.0
2r1.0
2r1.0
2r1.0
#

Greetings!

[amigo]

Also now fixed, thanks again. I'll be out-of-house for next few hours, but keep it up. If we can just get the first 300 lines right...

[JakeSFR]

Corner cases, but anyway...

Code: Select all

# ./iq pow 1.0 3.0
1.0
# ./iq pow 1.0 3.00
./iq: line 774: . % 25 : syntax error: operand expected (error token is ". % 25 ")
Only integer roots allowed
1.0

This is also interesting:

Code: Select all

# ./iq pow 3.0 3.0
27.0
#
# ./iq pow 3.00 3.0
./iq: line 290: 3.00000000000 / 271828182845 : syntax error: invalid arithmetic operator (error token is ".00000000000 / 271828182845 ")
./iq: line 70: [: gt0: integer expression expected
./iq: line 71: [: -1gt0: integer expression expected
./iq: line 72: [: -1gt0: integer expression expected
div: 3 inputs required: number1 operator(/,%,r) number2
./iq: line 70: [: gt0: integer expression expected
./iq: line 71: [: -1gt0: integer expression expected
./iq: line 72: [: -1gt0: integer expression expected
div: 3 inputs required: number1 operator(/,%,r) number2

...and so on...

I've got both fixed with a separate case for '0', but not sure if that's the best approach:

Code: Select all

# remove extra trailing zeros from fractions
[ ${#pbasefrac} -gt 1 ] && while : ;do case $pbasefrac in '0') break ;; *'0') pbasefrac=${pbasefrac%*?} ;; *) break ;; esac ;done
[ ${#pexpfrac} -gt 1 ] && while : ;do case $pexpfrac in '0') break ;; *'0') pexpfrac=${pexpfrac%*?} ;; *) break ;; esac ;done

I was thinking about *'0') -> *'00') instead, but it leaves one trailing 0 for non-zero fractions.

Speaking of fractions, simplfrac also doesn't like 0 as an arg.

Oh, and these:

Code: Select all

# ./iq pow 0 0
.0
# ./iq pow 1 0
.0
#
# ^ both should be 1
#
# ./iq pow 00 00
how did we get here?
1.1
# ^ good question :)

Greetings!

[amigo]

You're a great help -I've gotten these all and probably the next ones you'll find, but I'll not push till tomorrow to make sure. It's bed time here anyway.

[amigo]

Okay, I've pushed some updates again, including the stuff you found and a couple of speed-up. And more comments.

I'm interested to know what you think, generally, about which functions are included in main 'iq'. I added the 100 lines of "e^x" at the very last after finally getting it up to speed. And getting a complete, still limited, pow is ~450 lines of code and comments(and all that goes with it). I put in "e^x" because it may bring a better solution to pow. And 'gcf' or similar might replace simplfrac.

Anyway, you probably have some thoughts on modularity and feature/mission creep. Or?

[MochiMoppel]

@MM -regarding rounding, the predecessor to iq (called fp4sh) did rounding, but it's expensive as we must watch and compare current result with the last to see where repetition is happening -and how long of a repeating pattern should be rounded? Especially in AI calculations, we may have values very close to 1 -like 0.999999999902. fp4sh also had a mode where results could be padded when shorter -think of Currency formats. But including any of those things will greatly slow down every major function of iq.

Make friends with printf
I know that you have already dismissed the idea but IMHO much too soon. May not be suitable when the user needs a 20digit precision but for common tasks I find it more than sufficient (your 0.999999999902 wouldn't be a problem). For currency, tax or other calculations proper rounding to 2 or max 4 digits is essential and and it's irrelevant that iq *could* display 1000 digits.

I tested printf in your div function. Works great and fast.
Basically I increased scale_div internally by 1 (we obviously need 1 digit more to calculate rounding ) , adjusted some calculations and used printf "%.$((scale_div-1))f\n" $Q_sign$Q_int.$Q_frac
as additional output command (printf by default eliminates '+', keeps '-' , prints empty variables as '0', so variables can be written simpler than in your echo command).

The only thing left is to add a test condition to determine when to use printf rounding and when echo cutting. I found that (busybox) ash printf is a bit stricter than bash printf but still allows a total of 16 digits in integer+fraction, i.e values like
9999999999999.123
0.123456789012345
can safely be rounded. If this is true in other shells then a test will be simple.

For huge numbers digit cutting can and must still be used. Probably doesn't really matter if the 1000th digit is cut or rounded

[JakeSFR]

Okay, I've pushed some updates again, including the stuff you found and a couple of speed-up. And more comments.

Thanks!

I'm interested to know what you think, generally, about which functions are included in main 'iq'. I added the 100 lines of "e^x" at the very last after finally getting it up to speed. And getting a complete, still limited, pow is ~450 lines of code and comments(and all that goes with it). I put in "e^x" because it may bring a better solution to pow. And 'gcf' or similar might replace simplfrac.

Anyway, you probably have some thoughts on modularity and feature/mission creep. Or?

As I said I suck at math big time, so I'm not even sure what I'd like to see included...
But when I casually play with things, I tend to break them, so I've got a few new issues instead.

Some commands without an arg:

Code: Select all

# ./iq pow
logx: how did we get here?
0.1
#

Code: Select all

# ./iq epow
./iq: line 69: [: lt0: integer expression expected
./iq: line 70: [: -1lt0: integer expression expected
./iq: line 71: [: -1lt0: integer expression expected
0
#

Code: Select all

# ./iq simplfrac
./iq: line 765: % 25 : syntax error: operand expected (error token is "% 25 ")
#

Code: Select all

# ./iq nthrt
./iq: line 636: [: -gt: unary operator expected
./iq: line 640: [: -gt: unary operator expected
#

Code: Select all

# ./iq exp
./iq: line 69: [: gt0: integer expression expected
./iq: line 70: [: -1gt0: integer expression expected
./iq: line 71: [: -1gt0: integer expression expected
./iq: line 69: [: gt0: integer expression expected
...

___________

Should be -1:

Code: Select all

# ./iq div 1 / -1
1.0
# ./iq div -1 / 1
1.0
#

Looks like just an oversight here:

Code: Select all

--- iq_old	2021-11-15 21:35:38.000000000 +0100
+++ iq	2021-11-15 22:52:58.818683432 +0100
@@ -275,7 +275,7 @@
     Q_fracsize=${#M_frac} R_int=''
     # test early for division by zero or easy answers
     case $dvsr in 0|00|'') echo "div: Division by zero" >&2 ; return 1 ;; 
-        "$mod") [ $scale_div -eq 0 ] && echo 1 ||  echo 1.0 ; return ;;
+        "$mod") [ $scale_div -eq 0 ] && echo ${Q_sign#+*}1 ||  echo ${Q_sign#+*}1.0 ; return ;;
     esac
     case $mod in 0|00|'') [ $scale_div -eq 0 ] && echo 0 || echo '0.0' ; return ;;esac
     if [ ${#mod} -lt 19 ] && [ ${#dvsr} -lt 19 ] ;then

___________

I just noticed fstpow and it's cool and fast. Spits an error for exp with a fraction, though.

Code: Select all

# ./iq fstpow 2 24.1
./iq: line 611: 24.1: syntax error: invalid arithmetic operator (error token is ".1")

Digression about 2^24: once upon a time I had a simple scientific calculator, can't recall the brand, but something known, I think (EDIT: found it, it was "RFT MR 609"), which was producing 2^24 = 16777200!
Multiplying 2*2*2*..*2 manually was ok. 2^23 and less as well as 2^25 and more were also ok, only 2^24 was affected.
That moment my utter trust in calculators was destroyed...

Also, there's inconsistency between fstpow and pow:

Code: Select all

# ./iq fstpow -2 3
-8.0
# ./iq fstpow -2 4
16.0
# ./iq fstpow -2 5
-32.0
# ./iq fstpow -2 6
64.0

Code: Select all

# ./iq pow -2 3
-8.0
# ./iq pow -2 4
-16.0
# ./iq pow -2 5
-32.0
# ./iq pow -2 6
-64.0

Both are correct, apparently: https://en.wikipedia.org/wiki/Order_of_ ... minus_sign, but opinions are very divided:

- bc, dc, Bash, Ash, Galculator, Calcoo, Genius, LO Calc, C (math.h's pow):

Code: Select all

-2^3 = -8
-2^4 = 16
-2^5 = -32
-2^6 = 64

- Wcalc, BB-awk/gawk/mawk, Python, Perl, Mathomatic, SpeedCrunch, Extcalc, Google:

Code: Select all

-2^3 = -8
-2^4 = -16
-2^5 = -32
-2^6 = -64

___________

Should be 0 here:

Code: Select all

# ./iq nthrt 0 2
0.0005
#

And shouldn't it be NaN?

Code: Select all

# ./iq nthrt -1 2
.00000
#

Or at least .00000+1.0i.

Greetings!

[amigo]

MochiMoppel, I know about printf and did use it before. However it will not work properly with large numbers and figuring out when to use it and when to bypass it -slower than just doing it. I used it to generate pads ad truncate fractions. It's also not implemented uniformly among shells and you have to do a $(printf ....) too often.

I'm still working today on getting all the weird input edge-cases for pow -since it calls -or sometimes shouldn't call -all those other pow and nthrt functions, simplfrac, etc. NULL inputs, yuukk. Another test to do...

Functions like fstpow are not really meant to be public. fstpow gets called a lot for doing an nthrt -a lot. Therefore it does no input verification at all. It only handles integer exponents and does no scale adjustments needed for accuracy when you do something like:

Code: Select all

fstpow -s3 2.1782  30
fstpow -s7 2.1782  30

You see, you need to know how much precision to use for the intermediate multiplications. You can do this though:
[codest]fstpow -s15 -t5 2.1782 30][/code]
If you want accurate answers for integer exponents, use 'intpow'. It finds the scale neededand runs its' own inline fstpow. If you send it a base<1 ( 0.xxx ^ int ) it will send the work to epow -which does everything you send it as a fraction anyway. But again, epow is for integer exponents only. Exponents with a fraction are more complex. pow is basically a wrapper to all those others. If you do a calcualtion like:

Code: Select all

pow -s7 3.1415926 -2.72

that will call nearly every function in 'iq'
More tomorrow.

[MochiMoppel]

It's also not implemented uniformly among shells and you have to do a $(printf ....) too often.

???
Not once, but never mind. Can always be done by the user as a post process on iq's output if needed.

[JakeSFR]

Functions like fstpow are not really meant to be public.

It's shown in 'Utility functions', so I thought it is.

If you want accurate answers for integer exponents, use 'intpow'.

Oh yes, intpow is the one I actually wanted for fast integer calculations.

Greetings!

[amigo]

I've spent two days working on all the special cases for nthrt. I've narrowed it down ...to under 20! Everything else gets handled normally. I'm still gonna hang on to that till tomorrow and check more in the mo'nin'.

About the various pows. For fstpow you need to KNOW how much scale is needed to gice the accuracy you want. And that is the whole problem. The algorithm is blistering fast and will do a 100th power in 7 iterations. If your inputs are integers, you can use this without problems.

intpow is about the 30th pow function I've written raising decimal numbers to an integer power -using the same algorithm as fstpow. Trying to find a reliable scale is a nightmare. With C or other languages where you run on hardware, you can blow up the scale and get away with. Of course, you get floating-point results, so there is loss of precision. Anyway, we only have the basic integer math of the shell, so we need to find the smallest scale we can use and still get the requested accuracy. Some fairly innocent-looking problem can lead to using scales of more than 300 digits -but not always. intpow is the first time that I've gotten it right. It uses log10 to find out exactly how much the number of digits is gonna grow. It still overestimates for many problems, but I haven't caught it giving any wrong answers. The call to logx costs a lot, but it replaces a huge mess of estimations which were quicker, but gave wrong answers.

epow is the final answer, as it can absolutely get the scale right -down to the digit. Using what I learned with log in intpow, I built epow to handle the small problems with base<1: 0.2864 ^ 17. But, at the same time, the method works also for large numbers. If you use the '-e' scaling you can do nearly any problem with very large or very small answers. The normal scaling '-s' is fastest, but if you use it for very small numbers, you'll get answers like: 0.000000 -which means that somewhere to the right of those 6 digits, there is a non-zero value. If you use the SigFig scaling '-S' like:

Code: Select all

epow -S7 0.234 10  =  0.0000004922192  
compare to:
epow -s7 0.234 10  = 0.0000004

epow answers really big problems, pretty fast, but only in e-notation.

Sorry, off to bed, but I'll push changes tomorrow. Just for fun try:

Code: Select all

epow -e7 0.027 ^ 999999999999999999 
and then:
epow -e7 0.027 ^ 1000000000000000000

[amigo]

I've done a lot of fixing, input handling, errors -primarily in nthrt, which should handle negatives bases correctly for Real-Cartesian.
Started same as above for intpow and epow . next time, pow will get nearly the same error routine as nthrt
Added tests for NULL input to most functions.
Thanks for testing a boring calculator.
Enjoy the comments, or not, but they are fleeting anyway.

[MochiMoppel]

Using busybox shebang #!/bin/ash:

Code: Select all

# iq  div 8 / 0.20000000000000
40.0

# iq  div 8 / 0.2000000000000000000
add: At least 2 inputs required: num1 num2 ; or: num1 [+,-] num2
0.0

add???
 
 
Using shebang #!/bin/sh. All these while loops for removing leading/trailing zeros can become very expensive:

Code: Select all

# time iq  div 8 / 0.20000000000000     
40.0

real	0m0.071s
user	0m0.060s
sys	0m0.000s

# time iq  div 8 / 0.2000000000000000000
40.0

real	0m0.348s
user	0m0.190s
sys	0m0.087s

 
 
Without a loop the execution speed would remain the same, no matter how many zeros need to be removed:

Code: Select all

# time iq  div 8 / 0.20000000000000     
40.0

real	0m0.074s
user	0m0.060s
sys	0m0.007s

# time iq  div 8 / 0.2000000000000000000
40.0

real	0m0.060s
user	0m0.050s
sys	0m0.007s

For the latter test I made this change in the div() function:

Code: Select all

#~     while : ;do case $M_frac in ?'0') M_frac=${M_frac%*?} ;; *) break ;;esac ;done
#~     while : ;do case $D_frac in ?'0') D_frac=${D_frac%*?} ;; *) break ;;esac ;done
M_frac=${M_frac%${M_frac##*[^0]}}
D_frac=${D_frac%${D_frac##*[^0]}}

Leading zeros in integers could be removed in the same way, and if the sign for the quotient is determined earlier then the signs and leading zeros could be removed in only one instruction, e.g. like this:

Code: Select all

M_int=${M_int#${M_int%%[^0+-]*}}

[amigo]

You're very sharp MM! and thorough. Just spent a little time fixing div for small divisor problems -before I saw your post.
It was already covered for shorter numbers.

Code: Select all

M_int=${M_int#${M_int%%[^0+-]*}}

That's very nice indeed -something I hadn't really looked into enough. Just tried that under bash, zsh, dash and zsh, where it works. Strangely, it does not under posh, which is not important -but may point to potential problems under shells like ash. About ash variants, if they don't support '$()' command substitution, they will not work for iq and their math units might not be up-to-speed or have no function support.
Is their some bracket-fu to directly read a single leading or trailing digit?

I will use the nice -is that character class or glob?- pattern where possible. There are places like epow, where we need to not only remove, but also count the zeros. I will gladly lose the unknown posh for the neater and faster code.

div is the oldest function still left in iq and had few of the possible speedups implemented. I had been retired from coding for about five years, and my last math class was ~50 years ago, so just getting the methods and order of operations was a steep climb.

I've been working on getting input/error handling in place for all public stuff. And I came to a decision about how to re-arrange the code. iq will only have +, -, *, /, and %, plus tsst of course. And an integer-only pow function mimicking bash/zsh/ksh '**' operator. The new subtraction-by chunking seems to be working fine, so I've already been working on revving-up div. I'm pretty sure I've gotten all the special cases/errors for nthrt covered, but I still have to do the same for pow itself -mostly same conditions.