Below I have listed the bashrc, jobscript, and Makefile needed to the run the NON-GPU version of the GTC Code for Tokamak and Stellarator in Param Pravega.
If your shell is BASH, then add the below code in a .bashrc
file in your HOME directory of Param Pravega Supercomputer
# .bashrc # Source global definitions if [ -f /etc/bashrc ]; then . /etc/bashrc fi # User specific aliases and functions module load spack . /home-ext/apps/spack/share/spack/setup-env.sh spack load /mxkpjli netcdf-fortran@4.5.3%intel@18.0.5 spack load petsc@3.16.2%intel@2021.4.0 /qb3bzz
For CSH shell replace the statements of above after "User specific aliases.." by:
module load spack source /home-ext/apps/spack/share/spack/setup-env.csh spack load /mxkpjli netcdf-fortran@4.5.3%intel@18.0.5 spack load petsc@3.16.2%intel@2021.4.0 /qb3bzz
Then save the file and source the file from your home directory as,
source .bashrc
Below is a list of different simulations one can submit in Param-Pravega for GTC.
Tokamak
Stellarator
Tokamak:
Stellarator:
For Stellarators,
#!/bin/bash #SBATCH -N 8 #SBATCH --ntasks-per-node=48 #SBATCH --ntasks-per-core=2 #SBATCH --time=00:04:00 #SBATCH --job-name=gtcx #SBATCH --error=%J.error #SBATCH --output=%J.output #SBATCH --partition=highmemory cd $SLURM_SUBMIT_DIR module load spack . /home-ext/apps/spack/share/spack/setup-env.sh spack load /mxkpjli netcdf-fortran@4.5.3%intel@18.0.5 spack load petsc@3.16.2%intel@2021.4.0 /qb3bzz spack load openmpi fabrics=auto /nncedca export LD_LIBRARY_PATH=/home/ext/apps/spack/opt/spack/linux-centos7-skylake_avx512/intel-18.0.5/netcdf-fortran-4.5.3-mxkpjlifwez6uij2qnz2pmn5kavlmpng/lib:/home/ext/apps/spack/opt/spack/linux-centos7-cascadelake/intel-2021.4.0/petsc-3.16.2-qb3bzznpghhokvmeitoebasgnvct5sau/lib:$LD_LIBRARY_PATH export I_MPI_FALLBACK=disable export I_MPI_FABRICS=shm:dapl export I_MPI_DEBUG=9 export OMP_NUM_THREADS=1 mkdir -p restart_dir1 mkdir -p restart_dir2 mkdir -p phi_dir mkdir -p trackp_dir ulimit -s unlimited # stack ulimit -d unlimited # data area ulimit -c unlimited mpirun -n 378 ./gtc
In above 378 MPI processes are submitted as mtoroidal = 9 (For W7-X), i.e. number of MPI process is a multiple of mtoroidal value.
Follow the link to get the code for Makefile_Param for running GTCv4.6U (Commit - 17b4a12) for the case of ICONFIG = TOROIDAL3D(Stellarators)
Last Updated: 31 July 2023