CSCI 3366 (Introduction to Parallel and Distributed Processing):
Sample Programs

MPI

• Sample makefile Makefile for compiling C and C++ programs. (See the Note About Makefiles below.)
• Sample list-of-machines file MPIhosts.
• Helpful hints: Using MPI on the Linux Lab Machines.
• "Hello, world" program: hello.c.
• "Hello, world" program with command-line arguments and hostname: fancy-hello.c.
• Trivial example of send/receive: send-recv.c.
• Simple send/receive with timing: time-msg.c.
• Programs to compute pi by numerical integration:
  • Sequential version (using MPI timing function): num-int-seq.c.
  • Parallel version: num-int-par.c.
  • Parallel versions using OpenMP constructs with SPMD approach (see comments in code for differences among versions): num-int-par-spmd-1.c, num-int-par-spmd-2.c.
• Generic "exchange messages with neighbors" programs, contrasting different types of communication:
  • With regular sends/receives: exchange.c.
  • With regular sends/receives, improved (to not hang on large messages): exchange-alt.c.
  • With synchronous sends/receives: exchange.c.
  • With asynchronous communication: exchange-asynch.c.
  • With asynchronous communication, another way: exchange-persistent.c.
• Generic master/worker programs:
  • Sequential version (using MPI timing function): master-worker-seq.c.
  • Parallel version (static assignment of tasks to workers): master-worker-par-static.c.
  • Parallel version (dynamic assignment of tasks to workers): master-worker-par-dynamic.c.
• Programs for heat diffusion example:
  • Sequential version (using MPI timing function): heat-eqn-seq.c.
  • Parallel version: heat-eqn-par.c.
  • Parallel version with overlapping computation and communication: heat-eqn-par-overlap.c.
• Programs for Mandelbrot example:
  • Sequential version (using MPI timing function): mandelbrot-mpi-seq.c.
  • Parallel version assigning work by blocks: mandelbrot-mpi-block.c.
  • Parallel version assigning work dynamically: mandelbrot-mpi-dynamic.c.
  • Parallel version assigning work round-robin: mandelbrot-mpi-cyclic.c.
  • Common code for GUI: mandelbrot-gui.h.
  • Makefile: Makefile.mandelbrot.
• Programs for matrix multiplication example:
  • Simple sequential version (using MPI timing function): matrix-mult-seq-simple.c.
  • Block-based sequential version (using MPI timing function): matrix-mult-seq-block.c.
  • Block-based parallel version: matrix-mult-par-block.c.
  • Additional code for above programs (you will need these to compile): matrix-mult-seq-initfortest.h, matrix-mult-seq-print.h, matrix-mult-seq-utility.h, matrix-mult-par-initfortest.h, matrix-mult-par-print.h, matrix-mult-par-utility.h.

OpenMP

• Sample makefile Makefile for compiling C and C++ programs. (See the Note About Makefiles below.) (A different Makefile is needed for the Fedora Core 4 machines.)
• Helpful hints: OpenMP on the Linux Lab Machines. (These notes are for machines running Fedora Core 5. See last year's version of my notes for instructions applicable to the remaining Fedora Core 4 machines -- Dione00 (4 processors), Dione01 and Dione02 (2 each).)
• "Hello, world" program: hello.c.
• Programs to compute pi by numerical integration:
  • Sequential version (using OpenMP timing function): num-int-seq.c.
  • Parallel version using OpenMP constructs: num-int-par.c.
• Simple example of synchronization among threads:
  • Without synchronization: synch-1.c.
  • Using critical section for synchronization: synch-2.c.
  • Using lock for synchronization: synch-3.c.
• Generic master/worker programs:
  • Sequential version (using OpenMP timing function): master-worker-seq.c.
  • Parallel version (static assignment of tasks to workers): master-worker-par-static.c.
  • Parallel version (dynamic assignment of tasks to workers): master-worker-par-dynamic.c.
• Programs for heat diffusion example:
  • Sequential version (using OpenMP timing function): heat-eqn-seq.c.
  • Parallel version: heat-eqn-par.c.
  • SPMD-style parallel version: heat-eqn-par-spmd.c.
• Programs for Mandelbrot example:
  • Sequential version (using OpenMP timing function): mandelbrot-omp-seq.c.
  • Parallel version with one-point-at-a-time critical section: mandelbrot-omp-by-points.c.
  • Parallel version with one-row-at-a-time critical section: mandelbrot-omp-by-rows.c.
  • Common code for GUI: mandelbrot-gui.h.
  • Makefiles: Makefile.mandelbrot (for FC5 machines) and Makefile.mandelbrot.old (for FC4 machines).
• Programs for matrix multiplication example:
  • Simple sequential version (using OpenMP timing function): matrix-mult-seq-simple.c.
  • Block-based sequential version (using OpenMP timing function): matrix-mult-seq-block.c.
  • Block-based parallel version: matrix-mult-par-block.c.
  • Additional code for above programs (you will need these to compile): matrix-mult-initfortest.h, matrix-mult-print.h, matrix-mult-utility.h.

Java

• "Hello, world" programs:
  • Version using anonymous inner class: Hello1.java.
  • Version using static inner class: Hello2.java.
  • Versions using Java 5.0 (a.k.a. 1.5) features: Hello3.java, Hello4.java, Hello5.java.
• Simple example of synchronization among threads:
  • Without synchronization: HelloSynch1.java.
  • With incorrect synchronization: HelloSynch2.java.
  • With correct synchronization, two ways: HelloSynch3.java, HelloSynch4.java.
• Programs to compute pi by numerical integration:
  • Sequential version: NumIntSeq.java.
  • Parallel version: NumIntPar.java.
• "Bounded buffer" class and test program, showing use of wait/notify synchronization and interrupts: BoundedBuffer.java, TestBoundedBuffer.java.
• Generic master/worker programs:
  • Version using sockets: SocketsMaster.java, SocketsWorker.java.
  • Version using RMI: RMIMaster.java, RMIMasterInterface.java, RMIWorker.java, RMIWorkerInterface.java.
  • Common code: Task.java, TaskResult.java.

POSIX threads (pthreads)

• Sample makefile Makefile for compiling C programs.
• "Hello, world" program: hello.c.
• Simple example of synchronization among threads:
  • Without synchronization: synch-1.c.
  • Using mutex lock for synchronization: synch-2.c.
• Programs to compute pi by numerical integration:
  • Sequential version: num-int-seq.c.
  • Parallel version: num-int-par.c.
  • Function to time programs (you will need this to compile above programs): timer.h.

Note About Makefiles