Previous: Running in parallel Up: Running in parallel Next: Parallel run
This is an automatically generated documentation by LaTeX2HTML utility. In case of any issue, please, contact us at info@cfdsupport.com.
Parallel decomposition
- Utility decomposePar handles the domain decomposition
- decomposePar reads from system/decomposeParDict
- An example of decomposeParDict file is e.g. in tutorial test case pitzDailyExptInlet
- Copy decomposeParDict file:
# cp $FOAM_TUTORIALS/incompressible/simpleFoam/pitzDailyExptInlet/\
/system/decomposeParDict $FOAM_RUN /pitzDaily/system - Print file on the screen:
# cat $FOAM_RUN /pitzDaily/system/decomposeParDict/*--------------------------------*- C++ -*----------------------------------*\ | ========= | | | \\ / F ield | OpenFOAM: The Open Source CFD Toolbox | | \\ / O peration | Version: dev | | \\ / A nd | Web: www.OpenFOAM.org | | \\/ M anipulation | | \*---------------------------------------------------------------------------*/ FoamFile { version 2.0; format ascii; class dictionary; location "system"; object decomposeParDict; } // * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * // numberOfSubdomains 4; method hierarchical; simpleCoeffs { n ( 2 1 1 ); delta 0.001; } hierarchicalCoeffs { n ( 2 2 1 ); delta 0.001; order xyz; } manualCoeffs { dataFile ""; } distributed no; roots ( ); // ************************************************************************* // - Parameter numberOfSubdomains defines number of sub-domains (cores)
- Parameter method defines decomposition method
- Decomposition methods are following:
- simple: Splits computational domain in directions specified in simpleCoeffs
Line 22 – 26 splits mesh into 2 parts in x direction - hierarchical: same as simple, but can specified order of directions. See lines 28 – 33
- metis: uses METIS algorithm, optimizes to minimal communication among processors
- scotch: uses SCOTCH algorithm, optimizes to minimal communication among processors
- manual: user can specify whole decomposition, each cell is assigned to some processor
- simple: Splits computational domain in directions specified in simpleCoeffs
- delta is factor of skewness of cells, standard value is 0.001
- Parameters distributed and roots are used for distributed systems e.g.:
distributed yes; roots 4 ( "path_1" "path_2" "path_3" "path_4" );
- Run decomposePar:
# decomposePar - Output is following:
/*---------------------------------------------------------------------------*\ | ========= | | | \\ / F ield | OpenFOAM: The Open Source CFD Toolbox | | \\ / O peration | Version: dev | | \\ / A nd | Web: www.OpenFOAM.org | | \\/ M anipulation | | \*---------------------------------------------------------------------------*/ Build : dev-e2ccbbbb Exec : decomposePar Date : Jun 16 2017 Time : 12:02:00 Host : $HOSTNAME PID : $$ Case : $FOAM_RUN/pitzDaily nProcs : 1 sigFpe : Enabling floating point exception trapping (FOAM_SIGFPE). fileModificationChecking : Monitoring run-time modified files using timeStampMaster (fileModificationSkew 10) allowSystemOperations : Allowing user-supplied system call operations // * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * // Create time Decomposing mesh region0 Create mesh Calculating distribution of cells Selecting decompositionMethod hierarchical Finished decomposition in 0.01 s Calculating original mesh data Distributing cells to processors Distributing faces to processors Distributing points to processors Constructing processor meshes Processor 0 Number of cells = 3056 Number of faces shared with processor 1 = 30 Number of faces shared with processor 2 = 119 Number of processor patches = 2 Number of processor faces = 149 Number of boundary faces = 6259 Processor 1 Number of cells = 3056 Number of faces shared with processor 0 = 30 Number of faces shared with processor 3 = 109 Number of processor patches = 2 Number of processor faces = 139 Number of boundary faces = 6249 Processor 2 Number of cells = 3056 Number of faces shared with processor 0 = 119 Number of faces shared with processor 3 = 28 Number of processor patches = 2 Number of processor faces = 147 Number of boundary faces = 6253 Processor 3 Number of cells = 3057 Number of faces shared with processor 1 = 109 Number of faces shared with processor 2 = 28 Number of processor patches = 2 Number of processor faces = 137 Number of boundary faces = 6249 Number of processor faces = 286 Max number of cells = 3057 (0.0245399% above average 3056.25) Max number of processor patches = 2 (0% above average 2) Max number of faces between processors = 149 (4.1958% above average 143) Time = 0 Processor 0: field transfer Processor 1: field transfer Processor 2: field transfer Processor 3: field transfer End - Decomposed mesh information
- Mesh is decomposed into 4 processors
- In $FOAM_RUN /pitzDaily directory are 4 more directories:
processor0, processor1, processor2, processor3 - Each processor* directory contains subdirectories 0 and constant
- In 0 directory there are local initial and boundary conditions
- In constant directory there is the local mesh