What is the best way to set up the information to be transmitted to give the maximum information transfer across a given channel?

This is of interest in many applications; an example is sending a large document by email. The document may be several hundred kilobytes in size and to send it by email over a telephone line using a modem operating at 28.8 kbps may take a considerable time. If the data are compressed, the file can in certain cases be reduced in size by a factor of over 10, which would reduce the time taken to transmit the file, thus improving the convenience and minimizing cost. If the source information contains redundant information, data compression can shorten the message by removing redundancy. An example would be sending a text file over a communications link where the source device removes redundant characters before transmission. If the file contains sequences that are repeated, such as several space characters in a row, then instead of transmitting each of them the number of spaces could be transmitted. At the receiver the reverse process would allow the original document to be recovered. This process can be done with any character and can be used on simple binary data. This is termed run length encoding and is widely used in a variety of applications including fax machines. Try sending a fax of plain paper and compare the time taken with that taken for a page of typed notes.

 More powerful algorithms are available but they all work by identifying redundancy and removing it. Note that the information content of the message is not altered and these techniques are termed compaction. Data compression may lose some information, which is acceptable in certain situations such as digital TV. The term data compression is used both for those processes that may lose information and for those that do not (i.e. compaction). Another example of coding source information is the Morse code.