curl/wget help search and download by filename
I would like to search and online directory ie. ibiblio/puppylinux and download only file with 'wine' in the filename
much like -A for extensions, had a little trawl but could not find an answer
ta
Discussion, talk and tips
https://forum.puppylinux.com/
I would like to search and online directory ie. ibiblio/puppylinux and download only file with 'wine' in the filename
much like -A for extensions, had a little trawl but could not find an answer
ta
I can try to put something together to do this. I have some functions written in PHP that perform this but I assume you are looking for a shell script solution?
thanks 'edge
don't mind how it's done, just don't feel like trawling ibiblio manually
Hi @ally this should do it, required are elinks and wget.
Should be possible with wget only, I guess, but don't know how atm, with elinks it's simple using -dump to get a full list of all subdirectories.
The wine packages will end up in dir wineget/pet_packages-....-wine
Will contain some empty folders, which means there's no wine package available on ibiblio for that puppy version.
Code: Select all
#!/bin/bash
# download only wine pet packages from https://distro.ibiblio.org/puppylinux/pet_packages....
# depends on elinks and wget
PET_PACKAGES=$(elinks -dump https://distro.ibiblio.org/puppylinux/ | grep 'https:' | grep 'pet_packages-' | awk '{print $2}')
for i in $PET_PACKAGES; do
DIR="wineget/$(basename "$i")-wine"
mkdir -p "$DIR"
wget --no-check-certificate -P "$DIR" -r -e robots=off -nd -l1 -A "wine*" "$i"
done
fantastic, sincerest thanks
did have to grab elinks (fossapup64) but it's working now
I stupidly closed the lid last night on the laptop and interrupted the wine files upload but only 15gigs to go
will throw these up after
again sincerest thanks!
You're most welcome ally !
Couldn't let it to figure out how to do it with only wget , so for who is interested:
EDIT: Later changed the code a little, so that on top of script the variable can be set for what's the package name is starting with, can change to e.g. "abiword"
(and then all abiword packages will be downloaded, although probably nobody will, except ally perhaps ).
Code: Select all
#!/bin/bash
# download only wine pet packages from https://distro.ibiblio.org/puppylinux/pet_packages-.....
# depends on wget
# variable for what's the package name is starting with, can change to e.g. "abiword" (or whatever)
PKG="wine"
wget --no-cache --no-check-certificate -T 60 -t 2 "https://distro.ibiblio.org/puppylinux/" -O /tmp/ibiblio.html
cat /tmp/ibiblio.html | grep -vi "Parent Directory" | grep -Po "href=.*" | grep 'pet_packages-' | while read I; do
PET_PKGS="$(echo -n $I | cut -d ">" -f 1 | cut -d \" -f 2)"
DIR="${PKG}get/${PET_PKGS}"
mkdir -p "$DIR"
wget --no-check-certificate -P "$DIR" -r -e robots=off -nd -l1 -A "${PKG}*" "https://distro.ibiblio.org/puppylinux/$PET_PKGS"
done
rm /tmp/ibiblio.html