CSCI 3366 (Parallel and Distributed Processing):
Sample Programs

Disclaimer: Code linked from this page is intended as a cleaned-up and commented version of examples from class. Many programs could be further improved with regard to functionality or design, but then they would no longer be as close to what we did in class.

OpenMP

• My notes Using OpenMP on the Linux Lab Machines.
• Sample makefile Makefile. (See the Note About Makefiles below.)
• ZIP archive containing all examples OpenMP.zip (including some not listed here yet).
• "Hello, world" program: hello.c.
• Simple example of synchronization among threads:
  • Without synchronization: synch-1.c.
  • Using critical section for synchronization: synch-2.c.
  • Using lock for synchronization: synch-3.c.
• Utility functions for use with remaining examples: timer.h,
• Programs to compute pi by numerical integration:
  • Sequential version: num-int-seq.c.
  • Parallel version using OpenMP constructs: num-int-par.c.
  • Parallel versions using OpenMP with SPMD approach (rather than OpenMP parallel for): num-int-par-spmd-1b.c, num-int-par-spmd-1a.c, num-int-par-spmd-2.c.
• Generic master/worker programs:
  • Sequential version: master-worker-seq.c.
  • Parallel version (static assignment of tasks to workers): master-worker-par-static.c.
  • Parallel version (dynamic assignment of tasks to workers): master-worker-par-dynamic.c.
  • Additional code for above programs (you will need these to compile): master-worker-functions.h.
• Programs for heat diffusion example:
  • Sequential version: heat-eqn-seq.c.
  • Parallel version: heat-eqn-par.c.
  • SPMD-style parallel version: heat-eqn-par-spmd.c.
  • Additional code for above programs (you will need these to compile): heat-eqn-functions.h.
• Programs for Mandelbrot example:
  • Sequential version: mandelbrot-seq.c.
  • Parallel version with one-point-at-a-time critical section: mandelbrot-omp-by-points.c.
  • Parallel version with one-row-at-a-time critical section: mandelbrot-omp-by-rows.c.
  • Additional code for above programs (you will need these to compile): mandelbrot-gui.h.
  • Makefile: Makefile
• Programs for matrix multiplication example:
  • Simple sequential version: matrix-mult-seq-simple.c.
  • Block-based sequential version: matrix-mult-seq-block.c.
  • Simple parallel version: matrix-mult-par-simple.c.
  • Block-based parallel version: matrix-mult-par-block.c.
  • Additional code for above programs (you will need these to compile): matrix-mult-initfortest.h, matrix-mult-print.h, matrix-mult-utility.h. matrix-mult-par-utility.h.

MPI

• My notes Using MPI on the Linux Lab Machines.
• Sample makefile Makefile. (See the Note About Makefiles below.)
• ZIP archive containing all examples MPI.zip (including some not listed here yet).
• Sample script to run program run-pgm. Sample list of machine names Hosts-pandora.txt.
• "Hello, world" programs: hello.c. hello++.cpp.
• "Hello, world" program with command-line arguments and hostname: fancy-hello.c.
• Utility functions for use with remaining examples: utility.h, timer.h, cmdline.h
• Trivial example of send/receive: send-recv.c.
• Simple send/receive with timing: time-msg.c.
• Programs to compute pi by numerical integration:
  • Sequential version: num-int-seq.c.
  • Parallel version: num-int-par.c.
• Generic master/worker programs:
  • Sequential version: master-worker-seq.c.
  • Parallel version (static assignment of tasks to workers): master-worker-par-static.c.
  • Parallel version (dynamic assignment of tasks to workers): master-worker-par-dynamic.c.
  • Additional code for above programs (you will need these to compile): master-worker-functions.h.
• Programs for heat diffusion example:
  • Sequential version: heat-eqn-seq.c.
  • Parallel version: heat-eqn-par.c.
  • Parallel version with overlapping computation and communication: heat-eqn-par-overlap.c.
  • Additional code for above programs (you will need these to compile): heat-eqn-functions.h, heat-eqn-mpi-functions.h.
• Programs for Mandelbrot example:
  • Sequential version: mandelbrot-seq.c.
  • Parallel version assigning work by blocks: mandelbrot-mpi-block.c.
  • Parallel version assigning work dynamically: mandelbrot-mpi-dynamic.c.
  • Parallel version assigning work round-robin: mandelbrot-mpi-cyclic.c.
  • Additional code for above programs (you will need these to compile): mandelbrot-gui.h.
  • Makefile: Makefile.
• Programs for matrix multiplication example:
  • Simple sequential version: matrix-mult-seq-simple.c.
  • Block-based sequential version: matrix-mult-seq-block.c.
  • Block-based parallel version: matrix-mult-par-block-1.c.
  • Block-based parallel version, revised to use additionall commmunicators matrix-mult-par-block-2.c.
  • Additional code for above programs (you will need these to compile): matrix-mult-initfortest.h, matrix-mult-print.h, matrix-mult-utility.h, matrix-mult-par-initfortest.h, matrix-mult-par-print.h, matrix-mult-par-utility.h.
• Programs demonstrating various ways to communicate (not shown in class):
  • Simple send/receive (can block if messages are large): exchange.c.
  • Simple send/receive, but with half the processes sending then receiving and the other half receiving then sending: exchange-alt.c.
  • Synchronous send/receive exchange-synch.c.
  • Non-blocking send/receive: exchange-asynch.c.
  • Send/receive with "persistent communication" (not shown in class): exchange-persistent.c.

Java

• My notes Java from the Command Line.
• ZIP archive containing all examples Java.zip (including some not listed here yet).
• Script to compile all Java files in directory: compile-all.
• Utility classes: Utility.java.
• "Hello, world" programs:
  • Version using anonymous inner class: Hello1.java.
  • Version using static inner class: Hello2.java.
  • Versions using java.util.concurrent: Hello3.java, Hello4.java, Hello5.java.
• Simple example of synchronization among threads:
  • Without synchronization: HelloSynch1.java.
  • With incorrect synchronization: HelloSynch2.java.
  • With correct synchronization, two ways: HelloSynch3.java, HelloSynch4.java.
• Programs to compute pi by numerical integration:
  • Sequential version: NumIntSeq.java.
  • Parallel version using explicit threads: NumIntPar1.java.
  • Parallel version using java.util.concurrent: NumIntPar2.java.
• Programs to test/time mergesort:
  • Sequential version: MergeSortSeq.java.
  • Parallel version: MergeSortPar.java.
• Generic master/worker programs:
  • Supporting code: FakeTasks.java.
  • Sequential version: seq/MasterWorkerSeq.java.
  • Multi-threaded versions:
    • threads/MasterWorkerParStatic.java.
    • threads/MasterWorkerParDynamic.java.
  • Client-server versions using sockets:
    • Supporting code: sockets/Master.java.
    • Worker: sockets/Worker.java.
    • Master using "parallel" approach (it knows how many workers): sockets/Master1.java.
    • Master using "distributed" approach (it continues to accept workers until all tasks are done): sockets/Master2.java
    To execute: Compile everything, and then run either sockets/Master1 or sockets/Master2 as the master, plus one or more workers. The start-sockets-workers script below may be useful.
  • Client-server version using RMI:
    • Supporting code: rmi/MasterWorkerRMIMasterInterface.java, rmi/MasterWorkerRMIWorkerInterface.java.
    • Master: rmi/Master.java.
    • Worker: rmi/Worker.java.
    To execute: Compile everything, start an instance of the registry with the command "rmiregistry" (in the directory containing the .class files), and then run one master and one or more workers. The start-rmi-workers script below may be useful.
  • Scripts start-socket-workers start-rmi-workers to start "worker" processes on selected machine. To use, copy into a directory in your search path (current directory will probably work) and make executable with chmod u+x start-*-workers. Executing with no arguments gives a usage message.

  OpenCL

  • My notes Using OpenCL on the Linux Lab Machines.
  • Sample makefiles Makefile.cuda (NVIDIA), Makefile.amd (AMD). (See the Note About Makefiles below.)
  • ZIP archive containing all examples OpenCL.zip (including some not listed here yet).
  • Utility functions for use with examples: timer.h, filesize.h, utility.h.
  • First program (to show information about available devices): semi-hello.c.
  • Simple example (vector addition): vector-add.c (main program), vector-add.cl (code to run on GPU).
  • Programs to compute pi by numerical integration:
    • Sequential version (simple version): num-int-seq-float.c.
    • Sequential version (improved version using Kahan algorithm): num-int-seq-kahan.c.
    • Parallel version using OpenCL: num-int-par.c, num-int-par.cl.

  POSIX Threads (P-Threads)

  • Sample makefile Makefile. (See the Note About Makefiles below.)
  • ZIP archive containing all examples P-Threads.zip.
  • "Hello, world" program: hello.c.
  • Simple example of synchronization among threads:
    • Without synchronization: synch-1.c.
    • Using lock for synchronization: synch-2.c.
  • Utility functions for use with remaining examples: timer.h,
  • Programs to compute pi by numerical integration:
    • Sequential version: num-int-seq.c.
    • Parallel versions, somewhat ugly version: num-int-par-1.c.
    • Parallel version, revised to maybe be less ugly: num-int-par-2.c.

    C++11 Threads

    • Sample makefile Makefile. (See the Note About Makefiles below.)
    • ZIP archive containing all examples C++Threads.zip.
    • "Hello, world" program: hello.cpp.
    • Simple example of synchronization among threads:
      • Without synchronization: synch-1.cpp.
      • Using lock for synchronization, one way: synch-2.cpp.
      • Using lock for synchronization, another way: synch-3.cpp.
    • Utility functions for use with remaining examples: timer.h,
    • Programs to compute pi by numerical integration:
      • Sequential version: num-int-seq.cpp.
      • Parallel version using shared variable and lock: num-int-par-1.cpp.
      • Parallel version using C++ "futures": num-int-par-2.cpp.

  Note About Makefiles

  To obtain a makefile, use your browser's download function; cutting and pasting will likely turn the tabs in the makefile into spaces and cause it to not work. Put the makefile in the directory in which you want to compile. All the makefiles linked from this page begin with brief documentation on their use (the "usage examples").

  Other files

  • Script to mail files for homework submission mail-files. Instructions for usage here.