Module 13: Probability and Algorithms

General approach:

• Generate many random data sets.

• Run algorithm on each data set.

• Evaluate (performance or quality) on each data set.

• Output average performance or quality.

Simple example: sorting

• We will evaluate the running time of Java's sorting algorithm using several randomly created arrays (of integers).

• Comparison with theoretical running time:
  • Java uses MergeSort
    => theoretical running time is O(n log(n)).
  • We will compute the ratio of the actual running time to theoretical time.
    => should be approximately constant.

• Sample code: (source file)

  
    static double performTrials (int numTrials, int arraySize)
    {
      // Initialize array A. 
      int[] A = new int [arraySize];
      for (int i=0; i < arraySize; i++)
        A[i] = i;
  
      // We will measure the total time taken over all trials. 
      double totalTimeTaken = 0;
  
      // Repeat "numTrials" times: 
  
      for (int n=0; n < numTrials; n++) {
  
        // Randomly permute elements. 
        RandomPermutation.permute (A);
  
        // Start timing. 
        double startTime = System.currentTimeMillis();
  
        // Sort. 
        Arrays.sort (A);
  
        // Measure time taken. 
        double timeTaken = System.currentTimeMillis() - startTime;
  
        // Accumulate, for average. 
        totalTimeTaken += timeTaken;
        
      }
      
      // Return average time taken. 
      return totalTimeTaken / numTrials;
    }
     

• To compare with theoretical time:

  
        // Perform comparison with each array size. 
        // Here, arraySizes[i] = size of i-th array. 
        for (int i=0; i < arraySizes.length; i++) {
  
          // Get estimate using randomly generated data. 
          double avgTimeTaken = performTrials (numTrials, arraySizes[i]);
  
          // Get theoretical running time. 
          double theoretical = theoreticalTime (arraySizes[i]);
  
          // Print ratio. 
          System.out.println ("Actual=" + avgTimeTaken + " theoretical=" + theoretical 
                              + " ratio=" + (avgTimeTaken/theoretical));
        }
      

More interesting example: the Marriage Problem

• The Marriage Problem:
  • There are n men and n women.
  • Each man ranks the women in order of preference.
  • Each woman ranks the men in order of preference.
  • A (group) marriage is a 1-1 pairing of men and women.
  • A marriage is unstable if there exists two couples (M_a, W_b) and (M_c, W_d) such that
    1. M_a prefers W_d over W_b
    2. W_d prefers M_a over M_c
    If this were the case, M_a and W_d would elope.
    => unstable solution.
  • Goal: write an algorithm to produce a stable marriage.

• The Proposal algorithm: "Man proposes, woman disposes"

  
  Algorithm: proposalAlgorithm (menPrefs, womenPrefs)
  Input: preferences of the men and women
  
       // Initially, nobody is married. 
  1.   UnmarriedList = place all men in list sorted by ID;
  
       // As long as there are unmarried guys (and therefore, unmarried gals) 
       // we'll find pairings. 
  
  2.   while UnmarriedList not empty
  
  3.     Mi = lowest-numbered guy in UnmarriedList
  
         // Mi tries his luck. 
  4.     for j=0 to n-1
  
  5.       Wk = woman who is Mi's j-th preference.
  
  6.       // Mi proposes to Wk. 
  7.       if Wk is not married
             // She accepts. 
  8.         Mi and Wk are married
  9.         exit for-loop
  10.      else if Wk prefers Mi to current husband
             // She elopes. 
  11.        Wk's current husband is dumped back on the UnmarriedList;
  12.        Mi and Wk are married
  13.        exit for-loop
  14.      endif
  
  15.    endfor
  
  16.  endwhile
  
  Output: stable marriage
      

• Analysis:
  • At every inner most step (the if-statement), a pairing occurs only if W_k has not rejected M_i before.
  • But every woman rejects each man only once.
    => maximum O(n²) rejections.
  • Or, O(n²) executions in the if-statement
    => O(n²) time overall.
  • We have assumed operations in the if-statement take O(1) time
    => can be done if ranks are computed ahead of time (one time sorting cost of O(n log(n)).

• What about random data?
  • O(n²) is worst-case.
  • Best-case: O(n) (first try).
  • What is the average running time?
    => use randomly generated data.
  • What is the average "quality"?
    => find the average ranking of each mate.

• Estimating the time taken:
  • Create several random data sets: random preferences.
  • Run the algorithm each time.
  • Accumulate total running time, and find the average.
  • Sample code: (source file)

    
        double totalTimeTaken = 0;
    
        // Repeat "numTrials" times: 
    
        for (int n=0;  n< numTrials; n++) {
    
          // Create set of random preferences for men and women. 
          for (int i=0; i < numCouples; i++) {
            // Create random prefs. 
            RandomPermutation.permute (menPrefs[i]);
            RandomPermutation.permute (womenPrefs[i]);
          }
    
          // Start timing. 
          double startTime = System.currentTimeMillis();
    
          // Perform matching. 
          marry (menPrefs, womenPrefs);
    
          // Measure time taken. 
          double timeTaken = System.currentTimeMillis() - startTime;
    
          // Accumulate, for average. 
          totalTimeTaken += timeTaken;
    
        }
      
        // Return average time taken. 
        return totalTimeTaken / numTrials;
          

• It is possible to show that the average running time is no worse than O(n log(n)).

In-Class Exercise 13.11: Download this template and estimate the "quality" of solution for both men and women. To do this, use the following:

• Once a solution (marriage) is produced, examine the spouse for each person, and find where in the ranking of that person the spouse occured. (Thus, the rank is equivalent to "how desirable in my list is my assignent spouse").
• Call this the "spouse-rank" for a married person.
• Then, compute the average "spouse-rank" over all the men.
• Compute the average "spouse-rank" over all the women.
• For each particular marriage, you will compute: (1) the average "spouse-rank" for the men, and (2) the average "spouse-rank" for the women. Now, over multiple data sets (and therefore, multiple marriages), compute the averages of each of these averages.
• You will also need the classes UniformRandom and RandomPermutation. Both can be downloaded together as a jar file or as a gzipped tar file.