CS 528: Lecture 4 - 2/9/99

2/9/99


Click here to start


Table of Contents

CS 528: Lecture 4 - 2/9/99

Assignments and Readings

Shared-Memory Synchronization Primitives

Terminology (Again)

Process Address Space

Multi-Threaded Process Address Space

Why Synchronization?

Synchronization Primitives

Mutual Exclusion

Implementation of Mutual Exclusion

What To Do While Waiting?

Lock Contention

Deadlocking Revisited

Condition Variables

Condition Variables

Producer/Consumer Example

Condition Variables

Semaphore

Barrier

Implementation of Semaphores and Barriers

Implementing Collective Communication

Motivation

Collective Communication Primitives

Broadcast and Multicast

All-to-All

Reduction

Fallacy In Kumar Chapter 3

Our Approach

Broadcast

Huang & McKinley, 94: Recursive Doubling

Why Do We Care About Network Topology?

Ring Broadcast

Ring Broadcast - Being A Little Smarter

Mesh Broadcast

Hypercube Broadcast

Reduction

Ring All-To-All

Mesh All-To-All

Hypercube All-To-All

All-To-All: Copying?

Barrier

Collective Communication on Clusters

Topology Still Matters

How to Address Topology Issues in Clusters?

Possible Cluster Broadcast

Hardware-Assisted Multicast

Advantage of Hardware-Assisted Multicast

Back To Programming Models

ZPL: Array Operations

ZPL: Complexity of Operations

ZPL: Operation Performance

ZPL: Operation Cost - Putting In the Constants

ZPL: Estimating Relative Performance

ZPL: Estimating Relative Performance

Chamberlain et al., 1998

Recap of Lectures 1-4

Layers of Abstractions

Parallel Computing Inherently More Complex than Sequential (and Less Mature)

Modest Potential for Performance Gain

Layers of Abstractions

Parallel Architecture

Programming Model

Effect of Primitive Performance on Application Performance

Programming Model, Again

Programming Model

Next Week

Author: Thu D. Nguyen

Email: tdnguyen@cs.rutgers.edu