Puppy Linux Discussion Forum

So, the King of your Kingdom has decided to regain his former vassals, who have recently fallen under the influence of a distant Queen. More and more websites are being shut down or unavailable. Do your friends, the same peons and pawns as yourself, advise you to make copies of the sites so that they are available offline?

There is no need to hurry. But the tools can be prepared. Let's list the tools for creating offline copies of websites and for downloading videos.

Offline browser utilities:
PMirrorget. Built-in (in Puppy) utility. There are few functions. I haven't checked the performance for a long time.
HTTrack, or rather in our case WebHTTrack. It hasn't been updated for several years, but it works well. Can be installed directly from PPM.

Video downloaders:
ClipGrab. It does its job quite well.
FireDM. Python+curl+youtube-dl. Not bad.
VDL. Java+youtube-dl and yt-dlp. Good.
XDM. Java+many promises of a miracle. You can take a look.
Youtube-Get. Python+yt-dlp. The invention of our local craftsman from the forum. It is highly recommended for familiarization.

The people demand bread and circuses!
Online services (as an exception):
9xbuddy
YT1s
Y2Mate

If there is something interesting, we will add it.

For UTube I use https://yt1s.com/en205 and https://y2mate.is/en74/ online

I download and save file.

Is this what you mean?

9xbuddy - https://9xbuddy.com/ the best in my view. y2mate is also very good for just youtube

Online services... They themselves can be blocked But so be it. I will add online services to the first post.

I refuse to use software of that sizes. I only checked the first one (ClipGrab) and that was a 90MB+ download. I'd much rather use the online alternatives. The other issue with the software is that the download "configurations/requirements" of these download sites (like youtube for example) change so often that you will regularly have to update your software whereas with the online services that's not your problem.

Not sure if i understand you right.
Browser extensions for downloading/archiving websites
Firefox: webscrapbook
Chrome: WebScrapBook
Palemoon: ScrapBook X

HerrBert wrote: ↑Tue Mar 15, 2022 4:22 pm

Browser extensions

I haven't used browser plugins for similar purposes yet. I will try

I remembered about two online services for downloading websites to a computer disk. The first allows you to download 50 pages of the first level for free, the second - content up to 20 10 MB (when did they manage to change the conditions?!). Not much, but it can be useful for copying small and structurally uncomplicated sites.

A small example from life
I didn't have time to make a copy of one of my favorite sites, FantasyAnime. Meanwhile, recently its author received a notification (Cease & Desist) from Square Enix, in which they demanded to remove a lot of ROM files with games from the site. Here is the author's own story: https://fantasyanime.com/news#121421

Grey wrote: ↑Tue Mar 15, 2022 5:05 pm

A small example from life
I didn't have time to make a copy of one of my favorite sites, FantasyAnime. Meanwhile, recently its author received a notification (Cease & Desist) from Square Enix, in which they demanded to remove a lot of ROM files with games from the site. Here is the author's own story: https://fantasyanime.com/news#121421

At the moment (April 3, 22:03 MSK) I am downloading the fantasyanime website using HTTrack. 78 gigabytes have already been downloaded (it's already the second day) There are a lot of iso images with games for some platforms, as well as music and soundtracks in the images. I don't download big videos (anime), I made filters by type -*.avi (as well as mkv, mp4, mpg), but I left wmv (they are small and are introductory videos for RPG games).

I wanted to filter the music and choose between downloading mp3 and wav. But then I decided to download both options and if the size of the site archive seems large, delete one of the types (most likely wav) manually.

Grey wrote: ↑Sun Apr 03, 2022 7:20 pm

At the moment (April 3, 22:03 MSK) I am downloading the fantasyanime website using HTTrack.

So, a local mirror of the site has been created. Almost two days at a speed of 100 megabits. It turned out 93 gigabytes and this is without the anime cartoons that I filtered out. Only the RPG+Museum part. I thought it would be 45-50

I hope that PackIt will pack it all and reduce it by at least a third

Grey wrote: ↑Tue Apr 05, 2022 9:16 pm

Almost two days at a speed of 100 megabits.

Why did it take so long to download? Because it takes a lot of time (and CPU heating) to parse pages, which HTTrack periodically deals with.

Ah, I forgot the final thoughts

1. HTTrack, though old, is still in shape

2. Let the mayor of the town, the president of the country or the "evil Americans (NATO, as an option )" turn off the Internet now. There will be something to browser offline

wget is already there in any Puppy I think. Just open a terminal where you want the site files to go and:

wget -r -p https://website.com

or the "evil Americans (NATO, as an option )" turn off the Internet now.

my cat is sitting on the button right now..............

wish she would stop messing with it......uploads are really slow tonight

rockedge wrote: ↑Wed Aug 30, 2023 1:19 am

or the "evil Americans (NATO, as an option )" turn off the Internet now.

my cat is sitting on the button right now..............
burning_seeing-eye.jpg

wish she would stop messing with it......uploads are really slow tonight

rockedge, is Verizon you service provider? It's mine. You may recall my several posts about 'weak wifi' and that I use pwidgets. pwidgets has a module which will display 'wifi strength'. The highest reading is 70/70. In my basement, the best reading it ever displays is 53/70. Anything below 46/70 and large downloads/uploads become impossible. Occasionally it will drop to 36/70.

I've been tracking it over the last year. About 10 P.M. EST there is almost always a drop which continues 'til the morning. My suspicion is that about that time Verizon reduces the power it applies to generate signals by reducing the power it uses/consumes. That, of course, would lower the cost of operation resulting in increased profits. Doing so would also benefit Verizon another way. If a consumer complains he/she is told to obtain 5G, at an additional $15 p/mo. The instruction not only doesn't solve the problem, it enhances it. 5G, carrying more data, takes longer to fully transmit.

If the above hypothesis is correct it is likely conduct violating Verizon's license and its contracts with consumers.

If I were some aggressive, consumer-oriented State Attorney General, or FCC, I'd subpoena Verizon's electricity consumption records. But I'm not, and have no direct connections. Best I could do is write letters that most likely would be consigned to waste-bins.

@mikeslr I have Frontier fiber optic service. Weird was downloads were fast and uploads really slow, which is unusual. Think perhaps the router and optic coupler need a reboot......I haven't uploaded yet today so can't be certain that will improve anything.

I agree with your theory and it would be exactly something Verizon would do.

@mikeslr You won't ever get a 70/70 signal unless your wifi reception is about 1 meter from the router without anything in between. 53/70 is relatively good. It may be worth to try and place your wifi reception unit higher if you can. I use a wifi dongle connected to an extended cable from the computer. You then have more opportunity to move your dongle around to get a better reception spot. It's amazing what a few feet to the left/right/ higher/ lower can achieve.