c - FFTW real-to-real transform strided array -

i have matrix stored in row-major order. trying compute dct of ub-matrix using fftw, , nonsense. in following paragraphs describe problem , solution, , in understanding why not work.

given i , l, compute dct of sub-matrix consists of rows k k mod l == i. example, assume l = 3 , i = 2. in following matrix, sub-matrix wish transform marked in red (2 mod 3 = 2, 5 mod 3 = 2, 8 mod 3 = 2).

the source , destination arrays have same layout, , transformed matrix should stored in same location in destination array.

void transform(double* src, double* dest, size_t rows, size_t cols, size_t l, size_t i) {     int rank = 2;     fftw_iodim64 dims[] = {         { rows / l, l, l },         { cols, rows, rows } };     fftw_r2r_kind kind = fftw_redft10;      fftw_plan plan = fftw_plan_guru64_r2r(rank, dims, 0, null, src + l, dest + l, &kind, fftw_estimate | fftw_unaligned | fftw_preserve_input);     fftw_execute(plan);     fftw_destroy_plan(pla); } 

update

i tested simple case when i=l=1. in case nonsense. tested 3x4 matrix 1 of dct basis vectors:

a(i,j) = cos((i + 0.5)*2*pi/3) * cos((j + 0.5)*3*pi/4) 

i expected result elements (close to) 0 except one. resulting matrix looks this:

0             -2.22045e-016  1.33227e-015  2.22045e-016 2.22045e-016  -2.77556e-016  9.99201e-016  5.55112e-017 -8.88178e-016 -1.62359       7.83938       1.62359 

seems quite weird.

update 2

i tested simple matrix (0,0) element 1 , rest zero. in case, too, i=l=1 (the sub-matrix whole matrix). here result:

      2       2       2       0 1.73205 1.73205 1.73205       0       1       1       1       0 

kind supposed array of size rank. see e.g. http://www.fftw.org/doc/guru-real_002dto_002dreal-transforms.html#guru-real_002dto_002dreal-transforms.