MPI Groups Not Assigning Correct Processor Rank
I have the following MPI / fortran code for creating two groups which contains the first 2/3 of the total number of processors and the second which includes the remaining 1/3. It compiles without issue, but when I print out the new rank (in the newly created group), only the second group displays the correct ranks, the processes in the first group show negative numbers.
Do you have any comments on this issue? Thank.
program test
implicit none
include "mpif.h"
integer, allocatable :: rs_use(:),ks_use(:)
integer numnodes,myid,mpi_err
integer ijk,new_group,old_group,num_used,used_id
integer proc_rs,proc_ks
integer RSPA_COMM_WORLD !Real Space communicator
integer KSPA_COMM_WORLD !Recip. Space communicator
! initialize mpi
call MPI_INIT( mpi_err )
call MPI_COMM_SIZE( MPI_COMM_WORLD, numnodes, mpi_err )
call MPI_Comm_rank(MPI_COMM_WORLD, myid, mpi_err)
proc_rs = 2*numnodes/3 !Nr. of processors for Real Space
proc_ks = numnodes - proc_rs !Nr. of processors for Recip. Space
write(6,*) 'processors rs',proc_rs,'ks',proc_ks
! get our old group from MPI_COMM _WORLD
call MPI_COMM_GROUP(MPI_COMM_WORLD,old_group,mpi_err)
! Real Space group that will contain 2*N/3 processors
allocate(rs_use(0:proc_rs-1))
do ijk=0,proc_rs-1
rs_use(ijk)=ijk
enddo
call MPI_GROUP_INCL(old_group,proc_rs,rs_use,new_group,mpi_err)
! create the new communicator
call MPI_COMM_CREATE(MPI_COMM_WORLD,new_group,RSPA_COMM_WORLD, mpi_err)
! test to see if I am part of new_group.
call MPI_GROUP_RANK(new_group,used_id, mpi_err)
! Recip. Space group that will contain N/3 processors
allocate(ks_use(proc_rs:numnodes-1))
do ijk=proc_rs,numnodes-1
ks_use(ijk)=ijk
enddo
call MPI_GROUP_INCL(old_group,proc_ks,ks_use,new_group,mpi_err)
! create the new communicator
call MPI_COMM_CREATE(MPI_COMM_WORLD,new_group,KSPA_COMM_WORLD, mpi_err)
! test to see if I am part of new_group.
call MPI_GROUP_RANK(new_group,used_id, mpi_err)
if(used_id==0) write(6,*) 'group ',used_id,myid
end program test
source to share
The problem is that only processes that belong to a group have an ID in that group. You new_group
only need to set to the appropriate process group and check the new ID after you add each process to its new group. For example, use a temporary variable tmp_group
to call MPI_COMM_CREATE
and assign it only to the group process. For the first call, MPI_COMM_CREATE
you do the following:
call MPI_GROUP_RANK(tmp_group,used_id, mpi_err)
if(myid<proc_rs) new_group = tmp_group
For the second call, MPI_COMM_CREATE
you would do the following:
call MPI_GROUP_RANK(tmp_group,used_id, mpi_err)
if(myid>=proc_rs) new_group = tmp_group
After all this, you can check the new rank for everyone:
call MPI_GROUP_RANK(new_group,used_id, mpi_err)
If you decide to check the rank in the group immediately after creating the group, make sure that only the processes that belong to the group call. But this is not a good idea, as you cannot save the new_group.
source to share