Corrupting the Puppy

[cobaka]

Dear fellow Puppians! Woof

Recently I had a discussion with a long-time friend.
It began when I explained my ageing P4 desktop 'fell on it's sword' and how I replaced it with a more modern desktop.
I got the replacement very cheap, because the machine was password protected and no-one could crack the password.
I just reformatted the HDD (and erased Windows). After that, it took about 15 minutes to install uPupBB32 and the world was a better place. (Certainly a much faster place!)

My friend explained the vulnerabilities of the Puppy Linux system.
I explained that the OS itself was (or could be) read-only. Personally I rarely boot from a CD - but it's possible.
I did this often in the past, but booting from a CD is slower than booting from a HDD or flash-drive.
My friend countered by explaining how the OS files ON THE SERVER could be 'infiltrated' and replaced.
(The problem isn't just your machine. The server is also vulnerable. Hmmm ....

That (of course) would involve also replacing the corresponding checksum file(s) (MD5 ... SHAx and so on).
Additionally, I think the actual size of the key files is known to the byte, so if both the master files and the MD5 files were exchanged the byte-count is still available as an additional point to check. A 'corrupter' could, (I suppose) 'pack' the original with null bytes and make the byte count for the 'original' and the 'fake' file the same.

At any rate, the point of this posting is a feeble attempt to understand the likely-hood of corrupting Puppy Linux.
For those who doubt, I must explain I have fairies in my back-yard. Anything is possible.

cobaka

PS My friend/colleague is obsessed with the idea of viruses, worms, foreign agents (and so on) taking over his desktop.

[Jafadmin]

Tell your "friend" that MD5 & ShaX are not security keys. They are checksums. Their only purpose is to verify that a download was successful. If a download is corrupted the checksum will fail. That said, most devs do keep a private sha256 checksum of their distros/apps so that they can personally verify that the version available for download is clean.

In anticipation of your "friend's" mental hyperventalizations, the tech community of planet earth has provided a handy technical explanation for the truly inquisitive: https://en.wikipedia.org/wiki/Checksum

[m-cuda]

Since the forger of an iso file can create a forgery that matches the original MD5 what is needed is a digital signature. However, Puppy iso files are not digitally signed, (I really don't understand why they are not distributed that way. AFAIK all Microsoft distributed software is digitally signed and some other Linux distros are digitally signed, e.g., Ubuntu) The best you can do is to download a distribution from multiple independent sources and do a binary compare. It is unlikely that a malevolent entity would be able to penetrate multiple independent web sites.

[Flash]

While it's been shown that a MD5 hash can be duplicated, it's not an easy thing to do. I'd like to see someone change a Puppy iso with malicious intent and still have the MD5 checksum come out the same. Of course, if that someone has that kind of access to a trusted server, they can change the checksum too.

[taersh]

PS My friend/colleague is obsessed with the idea of viruses, worms, foreign agents (and so on) taking over his desktop.

Which would make all discussions completely useless and also a huge waste of time.
At least, I have better things to do than to have discussions with that sort of a "friend"!

[cobaka]

The discussion began with the proposition that a 'rogue entity' might post a 'nobbled' version of the Puppy together with a 'nobbled' checksum. Generally my knowledge of virus, worms, trojan horses (etc) is about average. I know that there are a small number of people here who know a great deal more on this topic than I do. So I posted the question.

The consensus (thus far)is that corruption through malice is possible but not likely.
I did hope for answers that went to 'possible but vanishingly small'.
More likely is corruption of the file during transmission. Quite frankly, I don't understand how this can happen, but people here say it is possible. In the past I wrote device drivers for FDD & HDD controllers. I want to say that the process of checking the integrity of data transfer (sectors by sector) is thorough. I have never seen a single invalid sector of data transferred from disk to memory without being trapped by the inbuilt CRC check. Is the CRC - fault flag set? Yes? Then abandon the transfer. I trust the logic in the DMA controller!

I have never seen the protocol for checking files that are down-loaded, but I would be most surprised if this process was not similarly thorough.

Thanks to all for comment.

cobaka.

[Jafadmin]

I have never seen the protocol for checking files that are down-loaded, but I would be most surprised if this process was not similarly thorough.

Thanks to all for comment.

cobaka.

https://www.internetdownloadmanager.com ... lems9.html

[cobaka]

Thanks @Jafadmin

I understand the problem. The problem arises when various down-load streams become mixed (plus ...)
This brings me back to the need for using the checksum to check the integrity of the file on my disk.

cobaka

[Jafadmin]

Again .. The only use for the md5 checksum when downloading is to verify that the file downloaded without error. Once you have verified the download, the checksum is no longer useful.

You would be surprised how often file downloads become corrupted. Since you are familiar with CRC & ACK/NAK, read up about the difference between TCP & UDP, sometime.

[wiak]

CRC is a very simple check check really. Not as 'reliable' afterall...

https://stackoverflow.com/questions/383 ... dealt-with

Over uniformly distributed data, it is expected to detect other types of errors at a rate proportional to 1 in 2¹⁶. The checksum also has a major limitation: the sum of a set of 16-bit values is the same, regardless of the order in which the values appear.

And yes, many download apps pull down files in parallel parts and then join them together using an application layer protocol, so that can introduce further errors.

[garnet]

The first rule of the security club is that we don't talk about the security club ^_^

[Flash]

It's my understanding that the TCP protocol specifies that each packet is sent with a checksum and that each packet's checksum is checked against one that is calculated at the receiving end using the same algorithm. If the checksums match, the packet is put in the queue. If the calculated sum does not match the one the packet was received with, the packet is discarded and a new one is requested. When all the required packets have been received and their checksums matched, they are assembled into the final file.

Given all that, it seems to me that you can be pretty close to certain that the file you received is the exact same as the one at the other end.

The UDP protocol also includes checksums but the packets are simply dropped if the checksums do not match. UDP packets are sent in the hope that they will be received intact and in the correct order. Replacement packets are not requested and will not be sent. UDP is used for real time apps that aren't sensitive to having a few packets lost or corrupted, mainly VOIP.

[wiak]

It's my understanding that the TCP protocol specifies that each packet is sent with a checksum and that each packet's checksum is checked against one that is calculated at the receiving end using the same algorithm. If the checksums match, the packet is put in the queue. If the calculated sum does not match the one the packet was received with, the packet is discarded and a new one is requested. When all the required packets have been received and their checksums matched, they are assembled into the final file.

Given all that, it seems to me that you can be pretty close to certain that the file you received is the exact same as the one at the other end.

Yes, TCP protocol gives end to end 'reliability' whereas UDP just throws out packets and they either make it or they don't...
However, as I posted about TCP protocol checksum is very simple check.

The checksum also has a major limitation: the sum of a set of 16-bit values is the same, regardless of the order in which the values appear.

In other words, it is relatively common for a TCP segment to get corrupted in a couple of places but end up having the same CRC checksum and hence overall corruption results. You can not be certain a received file is the exact same if relying on TCP alone (though received file, program, distro may generally appear to work okay... until corrupted area used...).

Having said that, when the underlying physical network is relatively reliable anyway (e.g. ethernet or fibre) then the error rate can be very low anyway and TCP checksum will find most of the few errors. If, however, there is something like a satellite or radio link involved the underlying error rate can be huge and TCP will certainly not catch everything.

[wiak]

Note that even noisy satellite links work 'quite' reliably with TCP (which is called a layer 3 protocol) because the physical layer 1 (below TCP layer) also uses a protocol (e.g. HDLC) that itself uses a checksum (generally a larger CRC than the one used for TCP) so by the time TCP gets its data passed to it most errors have already been detected and physical packets containing errors have been dropped (so TCP will resort to retransmission request). Still, it cannot be said that TCP is 100% reliable though I suspect most of the time everything gets through fine...

The layer in between say HDLC and TCP, which is IP (layer 2), also includes a checksum in its datagram packet but that is only used by the IP software layer code for checking the IP header information (not the enclosed data) has not been corrupted (i.e. IP address information). So the overall 'reliability' comes from a combination of the physical link layer protocol CRC and the TCP layer CRC (plus TCP retransmission mechanisms and TCP ensuring data packets remain in correct original order), which is pretty good but not 100% reliable actually. Hence md5 or similar application layer check is much safer/more-reliable and especially since the algorithm used by application layer or user-software stage for checking can afford to be much more complex and safe since doesn't need to be carried out in real time.

When UDP is used at layer 3 rather than TCP, there is no check done on the data integrity for what that layer receives, nor are there any retransmissions asked for, nor are received datagrams re-ordered if they happen to be received in non-original order. No checking or retransmission means it is efficient and fast and for apps like video or audio data humans can't usually see the missing bits or out of order packets overall... so doesn't matter (and the majority of datagrams probably still arrive in order anyway, and bad ones get dropped by the underlying physical layer checks) - results can be poor on noisy satellite links though and especially if it is raining or cloudy... But even UDP apps can be made reliable via some application software itself putting some checksome onto the original data (at the server end) and the client application software checking received parts okay (and asking for retransmissions when necessary), but if such reliability is required it is better to use TCP in the first case since the mechanisms are standardised and efficient at that earlier stage. The problem is that application layer software checking cannot keep up with real-time video/audio error checking or retransmissions and even TCP retransmission delays are problematic for such data.

I am familiar with all this because prior to the Internet becoming publicly available I was in a university research group contracted to develop TCP/IP protocols over noisy very small aperture satellite links (VSATs) so every day I would book time slots between the UK and ESA in the Netherlands and carry out experiments via small C programs transmitting either UDP or TCP (using Sun Solaris UNIX) whilst monitoring MIB statistics and directly for error rates, modifying system software TCP/IP stack to fine tune buffering, retransmission timeouts and so on, trying to get the data through uncorrupted as best and efficiently as possible - had huge fixed delay because of time to bounce off the satellite and major rain/cloud fade effects that required buffering and corresponding delay lags. That's why I learned how to program in C actually. Best time came at the end since we were sent to Seville to report: https://www.esa.int/Newsroom/Press_Rele ... xperiments

[bigpup]

I have seen corruption, in installed Puppy versions, caused by installing them to a not clean partition or format.
Especially doing a frugal install to fat32 format, that has been used for some time, and never been de-fragmented.

Just deleting does not actually remove the old code from the drive.
It just deletes the information in the file allocation table about what was deleted and marks the area as writable for new code.
you're only removing the map to the data—not the data itself—while also giving the operating system permission to overwrite that area of the hard drive.

So you delete a frugal install of a Puppy version and install a different frugal install of a Puppy version.
Good chance the new install is being written to the same location the deleted one was using.
Should be OK and cause no problem.
Key words should be OK!

I have seen a lot of strange problems solved by doing a completely new partition table, partition(s), and format to a USB drive.
This makes it about as clean as can be for installing on.
Unless you want to go all out and write all ones or zeros to the whole thing, to totally remove everything and start with a clean drive.
Yes, I have seen that to be needed to get a USB drive back to fully working.