How Can a ZIP File Get Smaller Without Losing Anything?
From Explain One Thing · 8:29
Show by The Podcaster's Warehouse
A smaller file can be a more economical description of the same data. Follow a fictional packing desk to understand repeat references, lossless reconstruction, and why some files barely shrink.Sources:https://www.rfc-editor.org/rfc/rfc1951https://www.zlib.net/zlib_faq.htmlTranscript:You have a folder that's too large to send. You compress it into a ZIP file, and the size drops. Later, someone opens the ZIP and gets the original files back.Where did the missing information go? If nothing was lost, how could there be less of it?This is Explain One Thing. Our question: how can a ZIP file get smaller without losing anything?The key is to separate information from the particular way it's written down. Sometimes a long piece of data can be described more economically. Lossless compression stores a description that lets the original data be reconstructed exactly.Let's start at an imaginary packing desk. You are writing instructions for someone arranging colored cards. The instruction says red, red, red, red, red, red. Instead, you could write red six times.That's a shorter description of the same sequence, provided both people understand the rule. Nobody removed a red card from the intended arrangement. You changed how the arrangement was represented.Our card shorthand is invented for this explanation. It isn't the actual syntax inside a ZIP file. But it gives us the first idea: repetition can sometimes be described instead of written out again and again.Now make the packing task a little more interesting. Your instruction sheet contains the same long phrase in several places. You write it once, and later use a reference meaning copy the phrase from earlier on this sheet.That reference has a cost. You have to explain where to look and how much to copy. If the repeated phrase is long enough, the reference can be shorter than another complete copy. If it's only one short symbol, the reference might be a poor bargain.Compression methods make versions of these tradeoffs. One common method used with ZIP is called DEFLATE. It combines ways of referring to repeated material with a way of coding symbols so that common ones can use shorter codes.You don't need to memorize the method's name. Follow the economy. Spend fewer bits describing predictable or repeated structure, while retaining enough instructions to rebuild the data.Think of a theater program. If every mention of a character used the actor's full biography, the booklet would be enormous. Printing the biography once and using the character's name afterward saves space, because readers know how the references connect.A computer doesn't use the same reasoning as a theater audience. It follows precise format rules. But both examples depend on a shared interpretation. Shorthand is useful only when the reader can expand it correctly.That shared-rule requirement explains why a compressed file needs a decoder. Looking at the compressed bytes as if they were ordinary prose is like reading the packing desk's references without its instructions. The data isn't necessarily damaged. You're using the wrong interpretation.Now consider the word lossless. It means the original data can be recovered exactly when the intact compressed data is decoded correctly. Every byte matters. A spreadsheet formula, a program instruction, and a punctuation mark can't simply be guessed because they seem unimportant.Imagine two people exchanging a numbered list of locker combinations. A message that is almost the same might fail completely. Lossless compression aims to preserve the precise original sequence, rather than a result that merely looks or sounds similar.That differs from lossy compression. Some image, audio, and video methods reduce size partly by discarding information according to a model of what the result can tolerate. A listener or viewer may find the result acceptable, but the discarded data generally can't be recovered just by reversing the process.The distinction isn't smaller versus larger. Both approaches can produce smaller files. The distinction is what kind of promise the method makes about reconstructing the original.Return to our packing desk. Suppose the instruction sheet is already written in very efficient shorthand. We ask another person to shorten it again without changing anything. There may be little obvious redundancy left to remove.That helps explain why an already compressed photograph or video may not shrink much when placed in a ZIP archive. Its existing representation has already done a lot of the space-saving work. A second method doesn't automatically discover another large saving.Some data may even become slightly larger after compression, because storing the necessary instructions and overhead costs space. Compress is the name of an operation, not a guarantee that every possible input becomes smaller.Here's an imaginary travel analogy. A suitcase is already packed tightly. Putting it inside another suitcase doesn't guarantee a smaller package. The…
More episodes of Explain One Thing
More episodes
- Why Are Soap Bubbles Round?
- Why Do Some Airplane Trails Last So Long?
- Why Does a Microwave Leave Cold Spots?
- Why Does Rain Have a Smell?
- What Makes Water Hard?
- How Does GPS Know Where You Are?
- How Does a Barcode Know the Price?
- Why Does Metal Feel Colder Than Wood?
- How Can Adding Sound Make Noise Quieter?
- Why Do Leaves Change Color in Autumn?
PodSearch is a free, independent podcast search engine. Read about PodSearch, the privacy policy or contact us.