ifort + openmp + fftw problem

Hi, folks,

After two days of trying, I almost give up.

What I want to do is very simple. I wanna using openmp to do fft in
parallel. But I kept get segfaults.

I have a do loop. In each do loop, I do ONE fft. The idea is to use
openmp to do FFTs in each processor at the same time.

Here is the fortran file.

main.F
program main
implicit none
#include "fftw3.inc"
integer nx,ny
parameter(nx=128,ny=8)
double precision val(nx,ny)
double precision tp(nx), tf(nx)
integer i,j,k
real*8 deltax,rlenx
real*8 pi,amp

integer*8 plan_r2c
integer tid,OMP_GET_THREAD_NUM

pi=4.d0*atan(1.d0)
rlenx=2.d0*pi
deltax=rlenx/dble(nx)
c      write(*,*)'rlenx:',rlenx

amp=2.d0

do j=1,ny
do i=1,nx
val(i,j)=amp*dsin(2.d0*pi*2.d0*deltax*dble(i-1)/rlenx)
enddo
enddo

CALL OMP_SET_NUM_THREADS(4)

!$omp parallel do private(tid,k,plan_r2c,tp,tf)
c!$omp parallel do default(private)
do k =1,ny
tid = OMP_GET_THREAD_NUM()
write(*,*)  'Thread ID = ', tid, ' Loop # = ', k

do i=1,nx
tp(i)=val(i,k)
tf(i)=dble(0.0)
enddo

call  dfftw_plan_dft_r2c_1d(plan_r2c,nx,tp,tf,
FFTW_MEASURE)
call dfftw_execute(plan_r2c)

call dfftw_destroy_plan(plan_r2c)
write(*,*) 'plan_r2c = ', plan_r2c

enddo

end

And fftw3.inc file
INTEGER FFTW_R2HC
PARAMETER (FFTW_R2HC=0)
INTEGER FFTW_HC2R
PARAMETER (FFTW_HC2R=1)
INTEGER FFTW_DHT
PARAMETER (FFTW_DHT=2)
INTEGER FFTW_REDFT00
PARAMETER (FFTW_REDFT00=3)
INTEGER FFTW_REDFT01
PARAMETER (FFTW_REDFT01=4)
INTEGER FFTW_REDFT10
PARAMETER (FFTW_REDFT10=5)
INTEGER FFTW_REDFT11
PARAMETER (FFTW_REDFT11=6)
INTEGER FFTW_RODFT00
PARAMETER (FFTW_RODFT00=7)
INTEGER FFTW_RODFT01
PARAMETER (FFTW_RODFT01=8)
INTEGER FFTW_RODFT10
PARAMETER (FFTW_RODFT10=9)
INTEGER FFTW_RODFT11
PARAMETER (FFTW_RODFT11=10)
INTEGER FFTW_FORWARD
PARAMETER (FFTW_FORWARD=-1)
INTEGER FFTW_BACKWARD
PARAMETER (FFTW_BACKWARD=+1)
INTEGER FFTW_MEASURE
PARAMETER (FFTW_MEASURE=0)
INTEGER FFTW_DESTROY_INPUT
PARAMETER (FFTW_DESTROY_INPUT=1)
INTEGER FFTW_UNALIGNED
PARAMETER (FFTW_UNALIGNED=2)
INTEGER FFTW_CONSERVE_MEMORY
PARAMETER (FFTW_CONSERVE_MEMORY=4)
INTEGER FFTW_EXHAUSTIVE
PARAMETER (FFTW_EXHAUSTIVE=8)
INTEGER FFTW_PRESERVE_INPUT
PARAMETER (FFTW_PRESERVE_INPUT=16)
INTEGER FFTW_PATIENT
PARAMETER (FFTW_PATIENT=32)
INTEGER FFTW_ESTIMATE
PARAMETER (FFTW_ESTIMATE=64)
INTEGER FFTW_TIMELIMIT
PARAMETER (FFTW_TIMELIMIT=1073741824)
INTEGER FFTW_ESTIMATE_PATIENT
PARAMETER (FFTW_ESTIMATE_PATIENT=128)
INTEGER FFTW_BELIEVE_PCOST
PARAMETER (FFTW_BELIEVE_PCOST=256)
INTEGER FFTW_NO_DFT_R2HC
PARAMETER (FFTW_NO_DFT_R2HC=512)
INTEGER FFTW_NO_NONTHREADED
PARAMETER (FFTW_NO_NONTHREADED=1024)
INTEGER FFTW_NO_BUFFERING
PARAMETER (FFTW_NO_BUFFERING=2048)
INTEGER FFTW_NO_INDIRECT_OP
PARAMETER (FFTW_NO_INDIRECT_OP=4096)
INTEGER FFTW_ALLOW_LARGE_GENERIC
PARAMETER (FFTW_ALLOW_LARGE_GENERIC=8192)
INTEGER FFTW_NO_RANK_SPLITS
PARAMETER (FFTW_NO_RANK_SPLITS=16384)
INTEGER FFTW_NO_VRANK_SPLITS
PARAMETER (FFTW_NO_VRANK_SPLITS=32768)
INTEGER FFTW_NO_VRECURSE
PARAMETER (FFTW_NO_VRECURSE=65536)
INTEGER FFTW_NO_SIMD
PARAMETER (FFTW_NO_SIMD=131072)
INTEGER FFTW_NO_SLOW
PARAMETER (FFTW_NO_SLOW=262144)
INTEGER FFTW_NO_FIXED_RADIX_LARGE_N
PARAMETER (FFTW_NO_FIXED_RADIX_LARGE_N=524288)
INTEGER FFTW_ALLOW_PRUNING
PARAMETER (FFTW_ALLOW_PRUNING=1048576)


I compiled using
ifort  -c -g -C  -openmp    main.F
ifort  -o test.x -g -C  -openmp    main.o  -L/scratch/liu19/research/
fftw-3.1.2/lib -lfftw3


The output is something like:
[liu19@head new]$ ./test.x
Thread ID =            0  Loop # =            1
Thread ID =            1  Loop # =            3
Thread ID =            3  Loop # =            7
plan_r2c =                8036992
Thread ID =            0  Loop # =            2
*** glibc detected *** ./test.x: double free or corruption (!prev):
0x00002aaaac002340 ***
======= Backtrace: =========
/lib64/libc.so.6[0x396626e890]
/lib64/libc.so.6(cfree+0x8c)[0x3966271fac]

On Apr 2, 11:47 pm, foolcat <xiaofengli...@gmail.com> wrote:
> Hi, folks,
>
> After two days of trying, I almost give up.
>
> What I want to do is very simple. I wanna using openmp to do fft in
> parallel. But I kept get segfaults.

See the FFTW manual's section on thread-safety: http://www.fftw.org/doc/Thread-saf
ety.html

In particular, plan creation is not thread-safe.  You need to create
the plan(s) once and then only do fftw_execute in parallel.

Besides, if you care about performance so much that you are going to
multiple threads, you probably shouldn't be recreating the plan for
each transform, since they are all of the same size.  You can reuse
the same plan for different arrays by using dfftw_execute_dft.
However, since Fortran provides no way (that I know of) to allocate 16-
byte aligned memory (necessary for FFTW to exploit SIMD instructions),
you need to pass FFTW_UNALIGNED in the flags when creating the plan,
as explained in the FFTW manual.

Nor do you really want the completely unnecessary overhead of copying
the input to a temporary array.  Nor do you need to initialize the
output array, since FFTW overwrites it anyway.  You should just
transform with the column of your matrix as input to FFTW.

Even better would be to get rid of the loop entirely. (a) use FFTW's
advanced interface to do the multiple FFTs of the columns of your
matrix in a single call.  (b) compile FFTW with  --enable-openmp, and
then use dfftw_plan_with_nthreads to tell it to do the multiple FFTs
in parallel.

Regards,
Steven G. Johnson