Learning some Python via pyspread (playing)

[wiak]

Having come across nomacs graphics viewer, which uses Qt5

and then an animated webp to git convertor written in Python,

I've become interested in both Qt apps more generally and also Python (using or not using qt.

Like I said elsewhere, I don't know Python coding at all, but came across a spreadsheet (called pyspread - version 1.99.7) written in python(ver 3); also relies on PyQt (python-pyqt5 in Arch Linux repos) which is auto installed as dependency as far as I remember. It's a three dimensional spreadsheet - i.e. 3D matrix (X, Y, Z axes - so lots of 2D tables sitting on top of each other; in attached image, Z is three - like 3 excel tabs).

"pyspread" is no ordinary spreadsheet (though looks like one on first sight) - none of the usual spreadsheet functions/formulas work(!), but... instead, all the cells understand Python, so can use very powerful Python functions/formulas/macros and learn a bit Python as you play with it! I've quickly worked through the pyspread tutorial this evening and now see how to basically use it (now I just need to learn some more basic Python to write formulas and macros for entering into the cells. I rather think this pyspread could be used to write some really powerful utility/apps since Python has so many useful libraries to do so many things - the way the optional matplotlib works makes me think this could be a very powerful application. Okay, so anyone thinking it will be easy to convert an Excel-type spreadsheet to this is mistaken - it's very different indeed... forget that idea, but instead think of it as a great resource to practice learning basic Python on...

https://pyspread.gitlab.io/
https://gitlab.com/pyspread/pyspread/-/tree/master

pyspread is a non-traditional spreadsheet application that is based on and written in the programming language Python.
...
pyspread expects Python expressions in its grid cells, which makes a spreadsheet specific language obsolete. Each cell returns a Python object that can be accessed from other cells. These objects can represent anything including lists or matrices.
...
Cells may display text, markup, images or charts
...
Directly using Python code in a grid is a core feature of pyspread. The target user group has experience with or wants to learn the programming language Python

Was trivial to install on my WDL_Arch64 system (which is pretty much the case for anything I've stumbled across):

Code: Select all

pacman -S qt5-base qt5-svg

then I installed pyspread itself (as normal user) from AUR. Just did the usual git clone method, followed by makepkg -si to install, for that. I pretty much never use any kind of AUR helper.

Optional (for charts/graphs):

Code: Select all

pacman -S python-matplotlib

(No, I wasn't using Masterpdfeditor... I just was experimenting with embedding an image file into one of pyspread cells - happened to be image of masterpdfeditor info page).

[wiak]

I see another reason to learn Python @rockedge:

https://medium.com/zmninja/pyzm-the-pyt ... b1afe00bae
https://github.com/pliablepixels/pyzm

[wiak]

@rockedge

Off topic, but talking about Zoneminder. I seem to have successfully installed latest Arch AUR version on my WDL_Arch64, but I have a problem with Mariadb:

https://wiki.archlinux.org/title/ZoneMinder

Tells me to run command below, but error as shown. I have no idea about this...

Code: Select all

# mysql_install_db --user=mysql --basedir=/usr --datadir=/var/lib/mysql 2> /dev/null
Installing MariaDB/MySQL system tables in '/var/lib/mysql' ...

Installation of system tables failed!  Examine the logs in
/var/lib/mysql for more information.

The problem could be conflicting information in an external
my.cnf files. You can ignore these by doing:

    shell> /usr/bin/mysql_install_db --defaults-file=~/.my.cnf

You can also try to start the mysqld daemon with:

    shell> /usr/bin/mariadbd --skip-grant-tables --general-log &

and use the command line tool /usr/bin/mariadb
to connect to the mysql database and look at the grant tables:

    shell> /usr/bin/mysql -u root mysql
    mysql> show tables;

Try 'mysqld --help' if you have problems with paths.  Using
--general-log gives you a log in /var/lib/mysql that may be helpful.

Perhaps I have some needed user or group to create. There is a user and group "mysql" in /etc/passwd and /etc/group

[rockedge]

@wiak exactly! I use both zmNinja (since it's very beginnings) and the entire zmeventnotification system that performs the object and face recognition for Zoneminder. I have worked with "asker" also known as "pliablepixels" for about 6 years now, testing and giving him SSH access to my Puppy Linux driven ZM versions to hammer out some stuff.

asker has put zmNinja in a sort of hibernation! Just happened....here is the notice:
https://forums.zoneminder.com/viewtopic.php?t=30996

I have been making my X10 contol switches and system in Python.....simple stuff but a start!!

[rockedge]

@wiak I ran into this at times as well. I think I deleted the /etc/mysql/my.cnf file completely or I just copied or renamed a generic sample copy my.cnf and now tried to start the server OR I re-ran the mysql tables installer. It has been awhile since I ran into it BUT you are very very close to a good install of ZM
Arch makes a good ZM platform once it's installed and running!

[wiak]

@wiak I ran into this at times as well. I think I deleted the /etc/mysql/my.cnf file completely or I just copied or renamed a generic sample copy my.cnf and now tried to start the server OR I re-ran the mysql tables installer. It has been awhile since I ran into it BUT you are very very close to a good install of ZM
Arch makes a good ZM platform once it's installed and running!

Tried simply installing mariadb on my other WDL_Arch64 install; never had trouble installing anything before so think must be a users/groups and/or permissions matter. Do you know of any special users/groups I have to create @rockedge? I do have a user mysql. Am I supposed to be logged in as that user to configure mariadb with?:

Code: Select all

mariadb-install-db --user=mysql --basedir=/usr --datadir=/var/lib/mysql

I also note that /var/lib/mysql/ is indeed owned by user:group mysql:mysql, so I don't know what is wrong (didn't make any different removing /etc/my.cnf and Arch Wiki doesn't say any other config required.

EDIT: Ah, just noticed error on running above mariadb-install-db command:
# mariadb-install-db --user=mysql --basedir=/usr --datadir=/var/lib/mysql
Installing MariaDB/MySQL system tables in '/var/lib/mysql' ...
2021-07-26 18:22:24 0 [ERROR] InnoDB: The Auto-extending data file './ibdata1' is of a different size 0 pages than specified by innodb_data_file_path
2021-07-26 18:22:24 0 [ERROR] InnoDB: Plugin initialization aborted with error Generic error
2021-07-26 18:22:25 0 [ERROR] Plugin 'InnoDB' init function returned error.
2021-07-26 18:22:25 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
2021-07-26 18:22:25 0 [ERROR] Unknown/unsupported storage engine: InnoDB
2021-07-26 18:22:25 0 [ERROR] Aborting

Installation of system tables failed! Examine the logs in
/var/lib/mysql for more information.InnoDB: Write to file ./ibdata1 failed at offset 0, 1048576 bytes should have been written, only 0 were written. Operating system error number 22.

Installing MariaDB/MySQL system tables in '/var/lib/mysql' ...
/usr/bin/mariadbd: /usr/lib/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by /usr/bin/mariadbd)

Yeah, well, not much I can do about that since Arch repo only providing libstdc++5-3.3.6-7 (assuming it needs libstdc++5-3.4.29?? but I'm not sure...).

I'm not sure what is going on...

EDIT2: I upgraded gcc and that error vanished, but now have other errors I've seen. Goodness knows what is going on:

Code: Select all

# mariadb-install-db --user=mysql --basedir=/usr --datadir=/var/lib/mysql
Installing MariaDB/MySQL system tables in '/var/lib/mysql' ...
2021-07-26 18:22:24 0 [ERROR] InnoDB: The Auto-extending data file './ibdata1' is of a different size 0 pages than specified by innodb_data_file_path
2021-07-26 18:22:24 0 [ERROR] InnoDB: Plugin initialization aborted with error Generic error
2021-07-26 18:22:25 0 [ERROR] Plugin 'InnoDB' init function returned error.
2021-07-26 18:22:25 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
2021-07-26 18:22:25 0 [ERROR] Unknown/unsupported storage engine: InnoDB
2021-07-26 18:22:25 0 [ERROR] Aborting

Installation of system tables failed!  Examine the logs in
/var/lib/mysql for more information.

It is failing to create/write file /var/lib/mysql/ibdata1 (ends with size 0). I think this may be some permission problem. I don't know anything about mysql (mariadb) such as what its 'root' user is and where passwds/permissions set. Not sure what to do.

[rockedge]

Strange stuff. The only other user needed is www-data:www-data for the Apache server and Zoneminder. This is changeable. Reading some information about similar errors seemed to indicate to perform this:

Add the following line in the /etc/my.cnf file inside the mysqld block. Then try again

Code: Select all

innodb_data_file_path = ibdata1:10M:autoexten

On Fossapup64 I could not successfully install mariaDB so I switched to the MySQL server which initialized correctly, which mariaDB did not do.

Also I am considering a permissions problem, but need to try some tests to track that down.

[wiak]

So the good news is mariadb installed okay on WDL_Arch64 and WDL_manjaXFCE with w_changes="" (or with no w_changes parameter on the kernel line).

The bad news is I was using w_changes=RAM1, which remains in development, where upper_changes is in RAM and mariadb didn't work then. It's odd because that RAM1 mode seems to work fine with anything else I've installed but somehow causes a mariadb database build to freak out and refuse to write the database at all. I haven't a clue about the reason at this moment - actually my method of putting upper_changes in RAM was a bit of guess work invention, so it is a wonder that ever seemed to work maybe. Yet it did and does work for anything except this mariadb build/initialise database step:

Code: Select all

sudo mariadb-install-db --user=mysql --basedir=/usr --datadir=/var/lib/mysql

Issue is I am short of time or desire to work on this at present so don't know when or if I'll ever work out what wrong - but at least I can now get it working albeit not whilst upper_changes loaded into RAM mode... I will think and check further about this though, to see if I can find some clue as to why above code line not working in RAM mode. Really weird, but I suppose I must be missing some detail I hadn't thought of when trying to implement this in-RAM mode. Way past my bedtime, I can hardly think straight like alone write clearly just now...

EDIT: Been mucking around with this all day. I simply can't get mariadb (mysql) to work when using WDL mode w_changes=RAMx. Works fine with w_changes="" (or if w_changes omitted from grub kernel line) - i.e. when not putting changes into RAM but instead writing directly to boot media.

Oh well, maybe the issue will be understood/fixed someday, by me or someone else. No big deal to me really since thus far I haven't needed mysql working on my family's main systems, and for special needs (such as Zoneminder) I prefer a dedicated machine and probably don't want captures stored to RAM memory in that case only anyway. Still, it is a limitation I'm annoyed I haven't solved - it definitely can be solved - I just don't myself know how. I continue to use w_changes in RAM for more general use - works fine (hence my finding the mysql case so odd...).

[wiak]

So I made a quick WDL_Ubuntu and tried using apt install mariadb-server, whilst using w_changes=RAM1 mode, and that in fact worked fine - the /var/lib/mysql database stuff was all created successfully and the mariadb server started fine. Seems therefore to be no issue with w_changes mode of WDL initrd (i.e. I seem to have been wrong to suspect that) - rather, seems to be an issue with Arch and Manjaro uses same pacman -Sy mariadb... very odd, but I can find nothing wrong with WDL_Arch64 itself and WDL_manjaXFCE using uses the official Manjaro aside from running it via WDL initrd. I'm therefore considering this issue closes now (I must remember to move these mariadb posts into WeeDog 'Bugs' section...