Parallel sorting Algorithms

29. Jun 2016
1 von 31

Parallel sorting Algorithms

• 2. Sorting in Parallel ● Why? it is a frequent operation in many applications. ● Goal? sorting a sequence of values in increasing order using n processors ● Potential speedup? best sequential algorithm has complexity= O(n log n) ● the best we can aim with a parallel algorithm, using n processors is: optimal complexity of a parallel sorting algorithm: O(n log n)/n = O(log n)
• 3. Parallel Sorting Algorithms ● Parallel Bubble Sort ● Parallel Merge Sort ● Bitonic Sort ● Shear sort
• 4. Parallel Sorting Algorithms ● Parallel Bubble Sort ● Parallel Merge Sort ● Bitonic Sort ● Shear sort
• 5. Bubble Sort ● One of the straight-forward sorting methods. – Cycles through the list. – Compares consecutive elements and swaps them if necessary. – Stops when no more out of order pair. ● Slow & inefficient. ● Average performance is O(n^2 ).
• 7. Parallel Bubble Sort: ● Also known as Odd-Even Bubble sort. ● operates in two alternate phases: Phase-even: even processes exchange values with right neighbors. Phase-odd: odd processes exchange values with right neighbors.
• 9. Parallel Bubble Sort: Algorithm: 1. For k = 0 to n-1 2. If k is even then 3. for i = 0 to (n/2) do in parallel 4. If A[2i] > A[2i+1] then 5. Exchange A[2i] ↔ A[2i+1] 6. Else 7. for i = 0 to (n/2)-1 do in parallel 8. If A[2i+1] > A[2i+2] then 9. Exchange A[2i+1] ↔ A[2i+2] 10. Next k
• 10. Parallel bubble sort : Analysis ● Steps 1-10 is a one big loop that is represented n times. ● Therefore, the parallel time complexity is O(n). ● The algorithm, odd-numbered steps need (n/2) - 1 processors and even-numbered steps require (n/2) processors. ● Therefore, this needs O(n) processors.
• 11. Parallel Sorting Algorithms ● Parallel Bubble Sort ● Parallel Merge Sort ● Bitonic Sort ● Shear sort
• 12. Mergesort: ● Example of a divide-and-conquer algorithm. ● Sorting method to sort a vector; first subdivides it in two parts, ● applies again the same method to each part and when they are both sorted (2 sorted vectors/lists) with m and n elements, ● They are merged to produce a sorted vector that contains m + n elements of the initial vector. ● The average complexity is O(n log n). ● T (n) = b n=1 =2T(n/2)+b(n) n>1 ● Solve the recurrence relation, T(n) = O(nlogn)
• 13. Parallel Merge Sort: ● Collects sorted list onto one processor. ● Merges elements as they come together. ● Simple tree structure. ● Parallelism is limited when near the root. Divide:
• 14. Parallel Merge Sort: ● Collects sorted list onto one processor. ● Merges elements as they come together. ● Simple tree structure. ● Parallelism is limited when near the root. Conquer :
• 15. Parallel Merge sort: Using a strategy to assign work to processors organized in a tree.
• 16. Merge sort - Time complexity ● Sequential Merge sort, O(nlogn) ● In Parallel,We have n processors ● logn time is required to divide sequence ● logn is for merging it. ● logn+logn= 2 logn ● O(logn)
• 17. Parallel Sorting Algorithms ● Parallel Bubble Sort ● Parallel Merge Sort ● Bitonic Sort ● Shear sort
• 18. Bitonic Sort : ● A Bitonic List is defined as a list with no more than one LOCAL MAXIMUM and no more than one LOCAL MINIMUM. ● (Endpoints must be considered - wraparound ) A sequence is bitonic if it contains two sequences, one increasing and one decreasing, i.e. ● a1 < a2 < . . . < ai−1 < ai > ai+1 > ai+2 > . . . > an for some i such that (0 ≤ i ≤ n) ● a sequence is bitonic if the property described is attained by a circular rotation to the right of its elements.
• 19. Bitonic sort: How many parallel steps does it take to sort ? --> logN How would you sort a bitonic list WHEN N is NOT A POWER OF 2 ? -->Use as many processes as N' where N' is a power of 2 and the smallest number larger than N. Assign a special value such as "infinity" to all the extra processes (some kind of a padding).
• 20. Bitonic Sort: How to divide a bitonic sequence into two bitonic subsequences ?? ● The compare-and-exchange operation moves smaller values to the left and greater values to the right. ● Given a bitonic sequence, if we apply recursively these operations we get a sorted sequence.
• 21. Bitonic Sort: How to divide a bitonic sequence into two bitonic subsequences ?? ● The compare-and-exchange operation moves smaller values to the left and greater values to the right. ● Given a bitonic sequence, if we apply recursively these operations we get a sorted sequence.
• 22. Number of steps (P=n) ● In order to form a sorted sequence of length n from two sorted sequences of length n/2, there are log(n) comparator stages required. ● T(n) = log(n) + T(n/2) ● The solution of this recurrence equation is, ● T(n) = log(n) + log(n)-1 + log(n)-2 + ... + 1 = log(n) · (log(n)+1) / 2 ● Each stage of the sorting network consists of n/2 comparators. On the whole, these are Θ(n·log2 (n)) comparators. ● For P=n, T(n)= Θ(n·log2 (n)) / n = Θ(log2 (n))
• 23. Bitonic Sort: How processors will be connected at different stages?? (when N=P).
• 24. Parallel Sorting Algorithms ● Parallel Bubble Sort ● Parallel Merge Sort ● Bitonic Sort ● Shear sort
• 25. Two-Dimensional Sorting on a Mesh ● Two network structures have received special attention: mesh and hypercube ● The layout of a sorted sequence on a mesh could be row by row or snake-like:
• 26. Shear Sort: Input: unsorted nn-array. Output: the array sorted in snake-like order Method: repeat log(n) times i. sort the rows (in alternating direction); ii. sort the columns; sort the rows; Sorting the rows in alternating direction means that even rows are sorted from left to right and odd rows from right to left.
• 27. Shear sort: Alternate row and column sorting until list is fully sorted. Alternate row directions to get snake-like sorting:
• 28. Shear sort – Time complexity ODD-PHASE: Even Rows: sort ascending order (use O-E Transposition sort) O(n) Odd Rows: sort descending order (use O-E Transposition sort) O(n) EVEN-PHASE: Sort each column in ascending order. It takes O(logn) phases to sort (n x n) numbers: T = O(nlogn) Tpar = O(logn)
• 29. Parallel sorting - summary Computational time complexity using P=n processors Parallel bubble sort - O(n) Parallel merge sort - O(log n) Bitonic sort - O(log2 n) Parallel Shear sort - O(logn)
• 30. References: ● “Parallel Programming Techniques & Applications Using Networked Workstations & Parallel Computers, 2nd ed.” B.Wilkinson) ● http://web.mst.edu/~ercal/387/Suppl/class.9.txt ● http://www.gel.usherbrooke.ca/mailhot/gei431/labos/labo2/oets_ss2.html ● http://www.iti.fh-flensburg.de/lang/algorithmen/sortieren/twodim/shear/shearsorten. htm ● https://en.wikipedia.org/wiki/Bitonic_sorter