CSCI 3366 (Introduction to Parallel and Distributed Processing):
Sample Programs

• Sequential version (using OpenMP timing function): mandelbrot-omp-seq.c.
• Parallel version with one-point-at-a-time critical section: mandelbrot-omp-by-points.c.
• Parallel version with one-row-at-a-time critical section: mandelbrot-omp-by-rows.c.
• Common code for GUI: mandelbrot-gui.h.
• Makefile: Makefile.mandelbrot

• Simple sequential version (using OpenMP timing function): matrix-mult-seq-simple.c.
• Block-based sequential version (using OpenMP timing function): matrix-mult-seq-block.c.
• Block-based parallel version: matrix-mult-par-block.c.
• Additional code for above programs (you will need these to compile): matrix-mult-initfortest.h, matrix-mult-print.h, matrix-mult-utility.h.

• Sequential version (using OpenMP timing function): master-worker-seq.c.
• Parallel version (static assignment of tasks to workers): master-worker-par-static.c.
• Parallel version (dynamic assignment of tasks to workers): master-worker-par-dynamic.c.

MPI

• Sample makefile Makefile for compiling C and C++ programs. (See the Note About Makefiles below.)
• Sample list-of-machines file MPIhosts.
• Script mpirun-on-upmachines to run MPI program on selected machines. To use, copy into a directory in your search path (current directory will probably work) and make executable with chmod u+x mpirun-on-upmachines. Executing with no arguments gives a usage message.
• Helpful hints: Using MPI on the Linux Lab Machines.
• "Hello, world" program: hello.c.
• "Hello, world" program with command-line arguments and hostname: fancy-hello.c.
• Trivial example of send/receive: send-recv.c.
• Simple send/receive with timing: time-msg.c.
• Programs to compute pi by numerical integration:
  • Sequential version (using MPI timing function): num-int-seq.c.
  • Parallel version: num-int-par.c.
• Generic "exchange messages with neighbors" programs, contrasting different types of communication:
  • With regular sends/receives: exchange.c.
  • With regular sends/receives, improved (to not hang on large messages): exchange-alt.c.
  • With synchronous sends/receives: exchange.c.
  • With asynchronous communication: exchange-asynch.c.
  • With asynchronous communication, another way: exchange-persistent.c.
• Programs for heat diffusion example:
  • Sequential version (using MPI timing function): heat-eqn-seq.c.
  • Parallel version: heat-eqn-par.c.
  • Parallel version with overlapping computation and communication: heat-eqn-par-overlap.c.
• Programs for Mandelbrot example:
  • Sequential version (using MPI timing function): mandelbrot-mpi-seq.c.
  • Parallel version assigning work by blocks: mandelbrot-mpi-block.c.
  • Parallel version assigning work dynamically: mandelbrot-mpi-dynamic.c.
  • Parallel version assigning work round-robin: mandelbrot-mpi-cyclic.c.
  • Common code for GUI: mandelbrot-gui.h.
  • Makefile: Makefile.mandelbrot.
• Programs for matrix multiplication example:
  • Simple sequential version (using MPI timing function): matrix-mult-seq-simple.c.
  • Block-based sequential version (using MPI timing function): matrix-mult-seq-block.c.
  • Block-based parallel version: matrix-mult-par-block.c.
  • Additional code for above programs (you will need these to compile): matrix-mult-seq-initfortest.h, matrix-mult-seq-print.h, matrix-mult-seq-utility.h, matrix-mult-par-initfortest.h, matrix-mult-par-print.h, matrix-mult-par-utility.h.
• Generic master/worker programs:
  • Sequential version (using MPI timing function): master-worker-seq.c.
  • Parallel version (static assignment of tasks to workers): master-worker-par-static.c.
  • Parallel version (dynamic assignment of tasks to workers): master-worker-par-dynamic.c.

Java

• Helpful hints: Java Without an IDE.
• "Hello, world" programs:
  • Version using anonymous inner class: Hello1.java.
  • Version using static inner class: Hello2.java.
  • Version using static inner class and environment variable to set number of threads: Hello2a.java.
  • Versions using Java 5.0 (a.k.a. 1.5) features: Hello3.java, Hello4.java, Hello5.java.
• Simple example of synchronization among threads:
  • Without synchronization: HelloSynch1.java.
  • With incorrect synchronization: HelloSynch2.java.
  • With correct synchronization, two ways: HelloSynch3.java, HelloSynch4.java.
• Programs to compute pi by numerical integration:
  • Sequential version: NumIntSeq.java.
  • Parallel version: NumIntPar.java.
• "Bounded buffer" class and test program, showing use of wait/notify synchronization and interrupts: BoundedBuffer.java, TestBoundedBuffer.java.
• Programs to test/time mergesort:
  • Sequential version: MergeSortSeq.java.
  • Parallel version: MergeSortPar.java.
• Generic master/worker programs:
  • Supporting code: FakeTasks.java.
  • Sequential version: MasterWorkerSeq.java.
  • Multi-threaded versions:
    • MasterWorkerParStatic.java.
    • MasterWorkerParDynamic.java.
  • Client-server versions using sockets:
    • Supporting code: MasterWorkerSocketsMaster.java.
    • Worker: MasterWorkerSocketsWorker.java.
    • Master using "parallel" approach (it knows how many workers): MasterWorkerSocketsParMaster.java.
    • Master using "distributed" approach (it continues to accept workers until all tasks are done): MasterWorkerSocketsDistribMaster.java
    To execute: Compile everything, then run either MasterWorkerSocketsParMaster or MasterWorkerSocketsDistribMaster as the master, plus one or more workers. The start-workers script below may be useful.
  • Client-server version using RMI:
    • Supporting code: MasterWorkerRMIMasterInterface.java, MasterWorkerRMIWorkerInterface.java.
    • Master: MasterWorkerRMIMaster.java.
    • Worker: MasterWorkerRMIWorker.java.
    To execute: Compile everything, start an instance of the registry with the command "rmiregistry", then run one master and one or more workers. The start-workers script below may be useful.
• Script start-workers to start "worker" processes on selected machines. (Somewhat preliminary version.) To use, copy into a directory in your search path (current directory will probably work) and make executable with chmod u+x start-workers. Executing with no arguments gives a usage message.

POSIX threads (pthreads)

• Sample makefile Makefile for compiling C programs.
• "Hello, world" program: hello.c.
• Simple example of synchronization among threads:
  • Without synchronization: synch-1.c.
  • Using mutex lock for synchronization: synch-2.c.
• Programs to compute pi by numerical integration:
  • Sequential version: num-int-seq.c.
  • Parallel version: num-int-par.c.
  • Function to time programs (you will need this to compile above programs): timer.h.

Note About Makefiles