Nth root algorithm

In computing, row-major order and column-major order describe methods for storing multidimensional arrays in linear memory. Following standard matrix notation, rows are numbered by the first index of a two-dimensional array and columns by the second index. Array layout is critical for correctly passing arrays between programs written in different languages. It is also important for performance when traversing an array because accessing array elements that are contiguous in memory is usually faster than accessing elements which are not, due to caching.

Row-major order is used in C/C++, Mathematica, PL/I, Pascal, Python, Speakeasy, SAS and others. Column-major order is used in Fortran, MATLAB, GNU Octave, R, Julia, Rasdaman, and Scilab.

Row-major order

In row-major storage, a multidimensional array in linear memory is organized such that rows are stored one after the other. It is the approach used by the C programming language, among others.

For example, consider this 2×3 array:

${\begin{bmatrix}1&2&3\\4&5&6\end{bmatrix}}$

An array declared in C as

int A[2][3] = { {1, 2, 3}, {4, 5, 6} };

is laid out contiguously in linear memory as:

1  2  3  4  5  6

To traverse this array in the order in which it is laid out in memory, one would use the following nested loop:

for (row = 0; row < 2; row++) 
    for (column = 0; column < 3; column++) 
        printf("%d\n", A[row][column]);

The difference in offset from one column to the next is 1 and from one row to the next is 3 (zero-based indexing). The linear offset from the beginning of the array to any given element A[row][column] can then be computed as:

offset = row*NUMCOLS + column

where NUMCOLS is the number of columns in the array.

The above formula only works when using the C convention of labeling the first element 0. In other words, row 1, column 2 in matrix A, is represented as A[0][1].

This technique generalizes to higher dimensions, so a 2×3×4 array looks like:

int A[2][3][4] = {{{1,2,3,4}, {5,6,7,8}, {9,10,11,12}}, {{13,14,15,16}, {17,18,19,20}, {21,22,23,24}}};

and the array is laid out in linear memory as:

1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24

Column-major order

Column-major order is a similar method of flattening arrays onto linear memory, but the columns are listed in sequence. The scientific programming languages Fortran and Julia, the matrix-oriented languages MATLAB, Octave and Scilab, the statistical languages S-Plus and R, the shading languages GLSL and HLSL (but not Cg), and the array database Rasdaman use column-major ordering. The array

${\begin{bmatrix}1&2&3\\4&5&6\end{bmatrix}}$

if stored contiguously in linear memory with column-major order looks like the following:

1  4  2  5  3  6

The memory offset could then be computed as:

offset = row + column*NUMROWS

where NUMROWS represents the number of rows in the array—in this case, 2.

Treating a row-major array as a column-major array is the same as transposing it. Because performing a transpose requires data movement, and is quite difficult to do in-place for non-square matrices, such transpositions are rarely performed explicitly. For example, software libraries for linear algebra, such as the BLAS, typically provide options to specify that certain matrices are to be interpreted in transposed order to avoid the necessity of data movement.

Generalization to higher dimensions

It is possible to generalize both of these concepts to arrays with greater than two dimensions. For higher-dimensional arrays, the ordering determines which dimensions of the array are more consecutive in memory. Any of the dimensions could be consecutive, just as a two-dimensional array could be listed column-first or row-first. The difference in offset between listings of that dimension would then be determined by a product of other dimensions. It is uncommon, however, to have any variation except ordering dimensions first to last or last to first. These two variations correspond to row-major and column-major, respectively.

More explicitly, consider a d-dimensional $N_{1}\times N_{2}\times \cdots \times N_{d}$ array with dimensions N_k (k=1...d). A given element of this array is specified by a tuple $(n_{1},n_{2},\ldots ,n_{d})$ of d (zero-based) indices $n_{k}\in [0,N_{k}-1]$.

In row-major order, the last dimension is contiguous, so that the memory-offset of this element is given by:

$n_{d}+N_{d}\cdot (n_{d-1}+N_{d-1}\cdot (n_{d-2}+N_{d-2}\cdot (\cdots +N_{2}n_{1})\cdots )))=\sum _{k=1}^{d}\left(\prod _{\ell =k+1}^{d}N_{\ell }\right)n_{k}$

In column-major order, the first dimension is contiguous, so that the memory-offset of this element is given by:

$n_{1}+N_{1}\cdot (n_{2}+N_{2}\cdot (n_{3}+N_{3}\cdot (\cdots +N_{d-1}n_{d})\cdots )))=\sum _{k=1}^{d}\left(\prod _{\ell =1}^{k-1}N_{\ell }\right)n_{k}$

Note that the difference between row-major and column-major order is simply that the order of the dimensions is reversed. Equivalently, in row-major order the rightmost indices vary faster as one steps through consecutive memory locations, while in column-major order the leftmost indices vary faster.

See also

• Matrix representation
• Vectorization (mathematics), the equivalent of turning a matrix into the corresponding column-major vector.

References

• Donald E. Knuth, The Art of Computer Programming Volume 1: Fundamental Algorithms, third edition, section 2.2.6 (Addison-Wesley: New York, 1997).