PB3D  [2.45]
Ideal linear high-n MHD stability in 3-D
Installation

Table of Contents

Introduction

PB3D is written in Fortran, and makes use of multiple numerical libraries:

  • blas / lapack
    • for basic linear algebra
  • pblas / blacs / scalapack
    • for parallelized basic linear algebra
  • HDF5
    • for storage files
    • works in parallel
  • NetCDF
    • to read input of VMEC
    • sequential
  • PETSc / SLEPc
    • for linear algebra of large, sparse matrices
    • can reach \(\sim \mathcal{O} (n)\) complexity
    • Minimal installation instructions:
      1. Configure PETSc using ./configure PETSC_ARCH=complex COPTFLAGS=-O3 CXXOPTFLAGS=-O3 FOPTFLAGS=-O3 \
        --with-scalar-type=complex \
        --download-metis \
        --download-mumps \
        --download-parmetis \
        --with-scalapack-lib="-L[SCALAPACKDIR]/lib -lscalapack" \
        --with-valgrind-dir=/usr \
        --with-debugging=no \
        --with-fortran-kernels=1
        (If you want debug, remove --with-debugging=no, --with-fortran-kernels=1, COPTSFLAGS, CXXOPTFLAGS and FOPTFLAGS, and change PETSC_ARCH to debug-complex.)
        (If you use a different HDF5, add --with-hdf5-dir=[HDF5DIR].)
      2. follow instructions to do makes and tests.
      3. Set global variables export SLEPC_DIR=[SLEPC DIR], export PETSC_DIR=[PETSC_DIR] and export PETSC_ARCH=[debug-]complex (depending on whether it is debug or not).
      4. Configure SLEPc using ./configure
      5. follow instructions to do makes and tests
  • StrumPack
    • for linear algebra of structured matrices [1]
    • the vacuum potential energy perturbation calculations use such matrices as they are generated with a \(1/r\) kernel.
    • can reach \(\sim \mathcal{O} (n \log(n))\) complexity
    • we use the dense version of the package, which for Fortran is 1.1.1.
    • Minimal installation instructions:
      1. Download, untar and go to root folder.
      2. Go to examples, copy Makefile.gnu to Makefile.inc and adapt:
        1. for example, it might be necessary to add -lblacs before -lscalapack
        2. if you use intel, have a look at the link line advisor, and update the LIB
        3. possibly, you might have to set the CC and FC compiler commands to, for example, mpicc respectively mpiifort
      3. Create directories lib and inc.
      4. Run make f90_example
      5. Make a library of StrumpackDensePackage.o using ar -rcs libstrumpack.a *.o in src.
      6. In inc, create a symlink libstrumpack.a to libstrumpack.a in src.
      7. In lib, create a symlink strumpackdensepackage.mod to strumpackdensepackage.mod in examples.
  • libstell
    • part of Stellopt suite, which contains VMEC
    • provides routines to read VMEC output data
  • pspline
    • Princeton Spline and Hermite Cubic Interpolation Routines
    • Minimal installation instructions:
      1. Export FFLAGS en CFLAGS if you want to optimize:
        • copy share/Make.overwrite.sample to share/Make.overwrite
        • Run the gmake below and note the flags, then edit these and put them in share/Make.overwrite
        • if GCC:
          • FFLAGS = -c -O3 -m64 -fno-range-check -fdollar-ok -cpp ; export FFLAGS
          • CFLAGS = -c -O3 -m64 ; export CFLAGS
          • CXXFLAGS = -c -O3 -m64 ; export CXXFLAGS
        • if Intel:
          • FFLAGS = -c -O3 -nowarn -ftz -auto-scalar -traceback -align dcommons
          • MPI_FFLAGS = -c -O3 -nowarn -ftz -traceback -align dcommons -auto-scalar (possibly add FC=mpiifort, CXX=mpicpc, CC=mpiicc if on a cluster)
      2. Compile with gmake NETCDF_DIR= FORTRAN_VARIANT=[VARIANT] with [VARIANT] either GCC or Intel (add DEBUG=1 and remove the FLAGS if debug wanted). Possibly also add OBJ=$[COMP_DIR] if on a cluster, with [COMP_DIR] the directory where to put the resulting library.
      3. Run tests from README_Pspline.

They should probably be installed in this order. On linux distributions such as Ubuntu, they may be available as packages.

Furthermore, PB3D comes bundled with some other, smaller libraries:

  • fftpack
    • to calculate the fast Fourier transform
  • foul
    • the Fortran Output Library

These do not have to be installed separately.

Compilation

When all dependencies are satisfied, the program is then compiled in the standard way:

  • Including the headers of all the libraries in the compilation of the object files:
    • This is done using -I[path_to_library].
    • Make sure you add the -o option to create only object files.
  • Linking with the actual libraries
    • This is done using -L[path_to_library] -l[library_name].

Makefile Example

1 ##############################################################################
2 #
3 # Example makefile for the program PB3D (Peeling Ballooning in 3D)
4 # \author Author: Toon Weyens
5 #
6 # Don't forget to set the directories:
7 # - LIBSTELL_DIR
8 # - HDF5_DIR
9 # - NETCDFF_DIR (note: Fortran library)
10 # - PETSC_DIR
11 # - SLEPC_DIR
12 # - STRUMPACK_DIR
13 ##############################################################################
14 
15 ##############################################################################
16 # Include
17 ##############################################################################
18 ## [PETSc and SLEPc trick]
19 include $(PETSC_DIR)/lib/petsc/conf/variables
20 include $(SLEPC_DIR)/lib/slepc/conf/slepc_variables
21 ## [PETSc and SLEPc trick]
22 
23 ## [PETSc and SLEPc trick inc]
24 INCLUDE = $(PETSC_FC_INCLUDES) $(SLEPC_INCLUDE)
25 ## [PETSc and SLEPc trick inc]
26 ## [Libstell special]
27 INCLUDE += -I$(LIBSTELL_DIR)/libstell_dir
28 ## [Libstell special]
29 INCLUDE += -I$(STRUMPACK_DIR)/include
30 ## [PB3D include]
31 INCLUDE += -I$(PB3D_DIR)/include
32 ## [PB3D include]
33 INCLUDE += -I/usr/include/hdf5/openmpi
34 
35 ##############################################################################
36 # Link
37 ##############################################################################
38 ## [PB3D libraries]
39 LIB_INTERNAL = libdfftpack.a libfoul.a libbspline.a
40 ## [PB3D libraries]
41 
42 LINK := $(LIB_INTERNAL)
43 
44 ## [PETSc and SLEPc trick lib]
45 LINK += $(PETSC_LIB)
46 LINK += $(SLEPC_LIB)
47 ## [PETSc and SLEPc trick lib]
48 LINK += $(LIBSTELL_DIR)/libstell.a
49 LINK += -L$(STRUMPACK_DIR)/lib -lstrumpack
50 LINK += -L$(HDF5_DIR) -lhdf5_fortran -lhdf5
51 LINK += -L$(NETCDFF_DIR)/lib -lnetcdff
52 LINK += -Wl,-R$(NETCDFF_DIR)/lib
53 LINK += -lscalapack -lblacs -lblas -lm
54 LINK += -lstdc++ -lmpi_cxx
55 
56 
57 ##############################################################################
58 # Compiler
59 ##############################################################################
60 COMPILER=mpifort
61 
62 
63 ##############################################################################
64 # Linker
65 ##############################################################################
66 LINKER=mpifort
67 
68 
69 ##############################################################################
70 # Compiler flags
71 # options (used with -D[name]):
72 # ldebug: debug
73 # lIB: infiniband
74 # lwith_gnu: use GNU compiler [default]
75 # lwith_intel: use INTEL compiler, (checked for version 12.0.2)
76 # note: INTEL warning 6536 is suppressed, which informs about extra "USE".
77 # note: INTEL warning 6843 is suppressed, which informs about empty
78 # intent(out) variables
79 ##############################################################################
80 COMP_FLAGS = -finit-real=snan -g -Og -Wall -Wextra -pedantic \
81  -fimplicit-none -fbacktrace -fno-omit-frame-pointer \
82  -fcheck=all -cpp -Dldebug# debug, profiling with gprof2dot, GCC
83 #COMP_FLAGS = -O3 -fbacktrace -g -fimplicit-none -fno-omit-frame-pointer \
84  #-cpp# optimized, GCC
85 
86 #COMP_FLAGS = -O0 -DlIB -Dldebug -g -heap-arrays 100 -recursive \
87  #-ftrapuv -check bounds -check uninit -traceback -implicitnone \
88  #-fno-omit-frame-pointer -cpp -Dlwith_intel -diag-disable 6536 \
89  #-diag-disable 6843# debug, profiling with gprof2dot, INTEL
90 #COMP_FLAGS = -O3 -DlIB -traceback -g -heap-arrays 100 -recursive \
91  #-implicitnone -fno-omit-frame-pointer -cpp -Dlwith_intel \
92  #-diag-disable 6536 -diag-disable 6843# optimized, INTEL
93 
94 COMP_FLAGS_EX= -O2 -w
95 
96 COMP_FLAGS_F= -O2 -funroll-loops -fexpensive-optimizations
97 
98 
99 ##############################################################################
100 # Link flags
101 ##############################################################################
102 LINK_FLAGS = -fPIC -finit-real=snan# debug
103 #LINK_FLAGS = -fPIC# optimized
104 
105 
106 ##############################################################################
107 # Prepare
108 ##############################################################################
109 # Add "Modules" and "Libraries" to the search path for the prerequisites
110 VPATH = Modules:Libraries
111 
112 # Contains list of source files (.o) and dependencies
113 DEPLIST = PB3D.dep
114 OBJLIST = ObjectList# defines "ObjectFiles"
115 
116 # Includes source files and dependency list
117 include $(DEPLIST)# Dependencies of all the objects
118 include $(OBJLIST)# Names of all the objects
119 
120 
121 ##############################################################################
122 # Rules
123 ##############################################################################
124 all: PB3D POST
125 
126 PB3D: $(ObjectFiles) $(LIB_INTERNAL) PB3D.o
127  $(LINKER) -o $@ $(ObjectFiles) PB3D.o $(LINK) $(LINK_FLAGS)
128 
129 POST: $(ObjectFiles) $(LIB_INTERNAL) POST.o
130  $(LINKER) -o $@ $(ObjectFiles) POST.o $(LINK) $(LINK_FLAGS)
131 
132 libdfftpack.a: dfft.o
133  ar -rcs libdfftpack.a dfft.o
134 
135 libfoul.a: foul.o
136  ar -rcs libfoul.a foul.o
137 
138 libbspline.a: bspline_sub_module.o
139  ar -rcs libbspline.a bspline_sub_module.o
140 
141 %.o: %.f90
142  $(COMPILER) $(INCLUDE) $(COMP_FLAGS) -c $<
143 
144 %.o: %.f
145  $(COMPILER) $(COMP_FLAGS_F) -c $<
146 
147 dfft.o: dfft.f
148  $(COMPILER) $(COMP_FLAGS_EX) -c $<
149 
150 foul.o: foul.f90
151  $(COMPILER) $(COMP_FLAGS_EX) -c $<
152 
153 bspline_sub_module.o: bspline_sub_module.f90
154  $(COMPILER) $(COMP_FLAGS_EX) -c $<
155 
156 clean:
157  @rm -f *.o *.a *.mod *~ fort.*
158 
159 clean_all:
160  @rm -f *.o *.mod *~ fort.* PB3D POST
Note
  1. PETSc and SLEPc don't like to be included in another makefile. The trick is to include two files:
    19 include $(PETSC_DIR)/lib/petsc/conf/variables
    20 include $(SLEPC_DIR)/lib/slepc/conf/slepc_variables
    which will load the variables PETSC_FC_INCLUDES and SLEPC_INCLUDE, used in
    24 INCLUDE = $(PETSC_FC_INCLUDES) $(SLEPC_INCLUDE)
    as well as the variables PETSC_LIB and SLEPC_LIB, used in
    45 LINK += $(PETSC_LIB)
    46 LINK += $(SLEPC_LIB)
  2. There are versions of libstell that do not use the standard convention. In this case you have to look for the *.mod files. In the example makefile this is done with
    27 INCLUDE += -I$(LIBSTELL_DIR)/libstell_dir
    instead of the standard inc directory.
  3. In
    31 INCLUDE += -I$(PB3D_DIR)/include
    there are includefiles that contain macros and wrappers specifically for PB3D.
  4. In
    39 LIB_INTERNAL = libdfftpack.a libfoul.a libbspline.a
    linking is done with external libraries that are bundled with PB3D.
this documentation was created with Doxygen 1.8.17 by Toon Weyens