PETSc Installation

PETSc (Portable, Extensible Toolkit for Scientific Computing) is a library of scalable linear and non-linear equation solvers and ODE solvers, developed and maintained at the Mathematics & Computer Science Division at Argonne National Laboratory. The official webpage of PETSc provides the download and installation instructions, as well as extensive documentation. Most of the codes I work with use the TS module of PETsc for advanced high-order time-integration schemes and it will be useful to briefly go through the documentation for this module to understand and utilize them:

  • User Manual (PDF) (see section on TS: Scalable ODE and DAE Solvers)
  • TS Examples

    • To download the official or development releases of PETSc, go here.

    Complete installation instructions, including all options, are available here. This page contains brief instructions that I find useful to quickly download and install PETSc to use with the codes I am working with.

    Download

    git clone -b release https://gitlab.com/petsc/petsc.git petsc

    Installation

    The following steps should install PETSc:

  • Set environment variables PETSC_DIR and PETSC_ARCH (Can be set in .bashrc too)
      • export PETSC_DIR=/path/to/petsc/ (Eg. PETSC_DIR=/home/ghosh/petsc)
      export PETSC_ARCH=(name of build) (Eg. arch-debug or arch-opt. It can be anything)
  • Move to the PETSc directory
      • cd $PETSC_DIR
  • Configure with options for debug (slow) or optimized (fast)
    Example: If GNU compilers exist but PETSc needs to download and compile MPICH,
      ./configure --with-cc=gcc --with-fc=gfortran --download-mpich=1 --with-shared-libraries --with-debugging=1 (debug)
      ./configure --with-cc=gcc --with-fc=gfortran --download-mpich=1 --with-shared-libraries --with-debugging=0 (optimized)
    Example: If MPICH already exists and PETSc needs to use that,
      ./configure --with-cc=mpicc --with-fc=mpif90 --with-cxx=mpicxx --with-shared-libraries --with-debugging=1 (debug)
      ./configure --with-cc=mpicc --with-fc=mpif90 --with-cxx=mpicxx --with-shared-libraries --with-debugging=0 (optimized)
    Note: Use --with-batch for configuring on machines that use a job scheduler.

    Note: PETSc can download and compile additional packages by adding --download-packagename=yes or use existing installations of those packages --with-packagename-include=/path/to/package/include and --with-packagename-lib=/path/to/package/lib or --with-packagename-dir=/path/to/package/ (use --help to see a list of these options and packages that PETSc can download). This page has examples of installing PETSc with downloading or using existing installations of HDF5, NetCDF and METIS packages.

  • Compilation
      make all

    Some machine specific configure commands

  • LLNL LC Quartz (With Hypre)
    • ./configure --with-batch --with-cc=mpicc --with-fc=mpif90 --with-cxx=mpicxx COPTFLAGS="-O2 -std=c99" FOPTFLAGS="-O2" CXXOPTFLAGS="-O2" --with-shared-libraries --with-debugging=0 --download-make --download-hypre --with-cxx-dialect=C++11
  • LLNL LC Quartz (With ParMETIS, Hypre and SuperLU)
    • ./configure --with-batch --with-cc=mpicc --with-fc=mpif90 --with-cxx=mpicxx COPTFLAGS="-O2 -std=c99" FOPTFLAGS="-O2" CXXOPTFLAGS="-O2" --with-shared-libraries --with-debugging=0 --download-make --download-cmake --download-hypre --download-superlu --download-superlu_dist --download-parmetis --download-metis --with-cxx-dialect=C++11
  • LLNL LC Lassen (With CUDA, ParMETIS, Hypre and SuperLU)
    • ./configure --with-batch --with-cc=mpicc --with-fc=mpif90 --with-cuda --with-cudac=nvcc --with-cxx=mpicxx COPTFLAGS="-O2 -std=c99" FOPTFLAGS="-O2" CXXOPTFLAGS="-O2" -CUDAOPTFLAGS="-O2" --with-shared-libraries --with-debugging=0 --download-hypre --download-superlu --download-superlu_dist --download-parmetis --download-metis --with-cxx-dialect=C++11
  • NERSC Cori (With ParMETIS, Hypre and SuperLU)
    • ./configure --with-cc=cc --with-fc=ftn --with-cxx=CC --with-shared-libraries --with-debugging=0 --with-batch COPTFLAGS="-O2" FOPTFLAGS="-O2" CXXOPTFLAGS="-O2" --download-cmake --download-hypre --download-superlu --download-superlu_dist --download-parmetis --download-metis --with-cxx-dialect=C++11