INTRODUCTION AND BACKGROUND

1. INTRODUCTION

2. NEED/MOTIVATION FOR COMPRESSION

3. BASIC DEFINITIONS

4. STRATEGIES FOR COMPRESSION

5. INFORMATION THEORY PRELIMINARIES

1. INTRODUCTION

• What is Compression?
  • It is a process of deriving more compact (i.e., smaller) representations of data
  
  
• Goal of Compression
  • Significant reduction in the data size to reduce the storage/bandwidth requirements
  
  
• Constraints on Compression
  • Perfect or near-perfect reconstruction (lossless/lossy)
  
  
• Strategies for Compression
  • Reducing redundancies
  • Exploiting the characteristics of human vision

• Massive Amounts of Data Involved in Storage/Transmission of Text, Sound, Images, and Videos in Many Applications
  
• Applications
  • Medical imaging
  • Teleradiology
  • Space/Satellite imaging
  • Multimedia
  • Digital Video: entertainment, home use
  • Digital photography
  
  
• Concrete Figures
  • A typical hospital generates close to 1 terabits per year
  • NASA's EOS will generate 1 terabytes per day
    
  • One 2-hour video = 1.3 terabits
  • Video transmission speed = 180Mb/sec
  • With MPEG1 (1.5Mb/s), need compression ratio of 120
  • With MPEG2 (4-10Mb/s), need comp. ratio of 18-45

• Lossless Compression: 100% accurate reconstruction of the original data

• Lossy Compression: The reconstruction involves errors which may or may not be tolerable

• Bit Rate: Average number of bits per original data element after compression
  

• 

• Signal-to-Noise Ratio (SNR) in the case of lossy compression.
  Let I be an original signal (e.g., an image), and be its lossily reconstructed counterpart. SNR is defined to be:
  

• Coding: Compression

• Codeword: A binary string representing either the whole coded data or one coded data symbol

• Symbol-Level Representation Redundancy
  • Different symbols occur with different frequencies
  • Variable-length codes vs. fixed-length codes
  • Frequent symbols are better coded with short codes
  • Infrequent symbols are coded with long codes
  • Example Techniques: Huffman Coding
  
  
• Block-Level Representation Redundancy
  • Different blocks of data occur with varying frequencies
  • Better then to code blocks than individual symbols
  • The block size can be fixed or variable
  • The block-code size can be fixed or variable
  • Frequent blocks are better coded with short codes
  • Example techniques: Block-oriented Huffman, Run-Length Encoding (RLE), Arithmetic Coding, Lempil-Ziv (LZ)
  
  
• Inter-Pixel Spatial Redundancy
  • Neighboring pixels tend to have similar values
  • Neighboring pixels tend to exhibit high correlations
  • Techniques: Decorrelation and/or processing in the frequency domain
  • Spatial decorrelation converts correlations into symbol- or block-redundancy
  • Frequency domain processing addresses visual redundancy (see the next slide)
  
  
• Inter-Pixel Temporal Redundancy (in Video)
  • Often, the majority of corresponding pixels in successive video-frames are identical over long spans of frames
  • Due to motion, blocks of pixels change in position but not in values between successive frames
  • Thus, block-oriented motion-compensated redundancy reduction techniques are used for video compression
  
  
• Visual Redundancy
  • The human visual system (HVS) has certain limitations that make many image contents invisible. Those contents, termed visually redundant, are the target of removal in lossy compression.
    
  • In fact, the HVS can see within a small range of spatial frequencies: 1-60 cycles/arc-degree

    (Plot by hand the contrast sensitivity function) 
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    	

  • Approach for reducing visual redundancy in lossy compression
    1. Transform: Convert the data to the frequency domain
    2. Quantize: Under-represent the high frequencies
    3. Losslessly compress the quantized data

• Discrete Memoryless Source S: A data generator where the alphabet is finite and the symbols generated are independent of one another. Assume the alphabet is {a₁,a₂,...,a_n}
• Let p_k = Probability that symbol a_k is generated (transmitted) by the source
  
• Entropy: H(S)=-(p₁log p₁ + p₂log p₂ + ... + p_nlog p_n)
  
• H(S) is the minimum average number of bits/symbol possible
  
• Sources with Memory: Presence of inter-symbol correlation
  
• Their entropy is still the min average number of bit/symbol
  
• Adjoint Source of Order N
  • Treat each possible block A of N symbols as a macrosymbol, and compute the probability P_A
  • Treat the source as a memoryless source consisting of the macrosymbols As and their probabilities P_A
  • The entropy
  
  
• Theorem (Shannon): For any source S with memory, as N