curl/wget help search and download by filename

[ally]

I would like to search and online directory ie. ibiblio/puppylinux and download only file with 'wine' in the filename

much like -A for extensions, had a little trawl but could not find an answer

ta

[rockedge]

I can try to put something together to do this. I have some functions written in PHP that perform this but I assume you are looking for a shell script solution?

[ally]

thanks 'edge

don't mind how it's done, just don't feel like trawling ibiblio manually

[fredx181]

Hi @ally this should do it, required are elinks and wget.
Should be possible with wget only, I guess, but don't know how atm, with elinks it's simple using -dump to get a full list of all subdirectories.
The wine packages will end up in dir wineget/pet_packages-....-wine
Will contain some empty folders, which means there's no wine package available on ibiblio for that puppy version.

Code: Select all

#!/bin/bash
# download only wine pet packages from https://distro.ibiblio.org/puppylinux/pet_packages....
# depends on elinks and wget
PET_PACKAGES=$(elinks -dump https://distro.ibiblio.org/puppylinux/ | grep 'https:' | grep 'pet_packages-' | awk '{print $2}')
for i in $PET_PACKAGES; do
DIR="wineget/$(basename "$i")-wine"
mkdir -p "$DIR"
wget --no-check-certificate -P "$DIR" -r -e robots=off -nd -l1 -A "wine*" "$i"
done

[ally]

fantastic, sincerest thanks

did have to grab elinks (fossapup64) but it's working now

I stupidly closed the lid last night on the laptop and interrupted the wine files upload but only 15gigs to go

will throw these up after

again sincerest thanks!

[fredx181]

You're most welcome ally !
Couldn't let it to figure out how to do it with only wget , so for who is interested:
EDIT: Later changed the code a little, so that on top of script the variable can be set for what's the package name is starting with, can change to e.g. "abiword"
(and then all abiword packages will be downloaded, although probably nobody will, except ally perhaps ).

Code: Select all

#!/bin/bash
# download only wine pet packages from https://distro.ibiblio.org/puppylinux/pet_packages-.....
# depends on wget

# variable for what's the package name is starting with, can change to e.g. "abiword" (or whatever)
PKG="wine" 

wget --no-cache --no-check-certificate -T 60 -t 2 "https://distro.ibiblio.org/puppylinux/" -O /tmp/ibiblio.html

cat /tmp/ibiblio.html | grep -vi "Parent Directory" | grep -Po "href=.*"  | grep 'pet_packages-' | while read I; do
PET_PKGS="$(echo -n $I | cut -d ">" -f 1 | cut -d \" -f 2)"
DIR="${PKG}get/${PET_PKGS}"
mkdir -p "$DIR"
wget --no-check-certificate -P "$DIR" -r -e robots=off -nd -l1 -A "${PKG}*" "https://distro.ibiblio.org/puppylinux/$PET_PKGS"
done
rm /tmp/ibiblio.html