Reshaping a data.frame from wide to long format

Many people often run into the problem that a data.frame is organized in a way that is incompatible with certain statistical routines. I have struggled myself a number of times with this problem, and with the supposedly solution to it: reshape. However, because I don't deal with real data a lot, I keep forgetting how this function works, and worse, it's manual doesn't really help. Noortje Jansen recently ran into the problem, and asked for some help. Although there is some more helpful documentation on reshape (like this and this one), I decided to make a simpler version that does a lot less than reshape, but that I hope is more intuitive.

table.to.long <-
function (x, repeated.measures.groups, ...) 
# Reshapes a 'single row = single case' data.frame x  
# into a 'single row = single measurement occasion' 
# formatted data.frame
#
# x: data.frame
# repeated.measures.groups: named list of character vectors. 
#         Each character vector corresponds to a single 
#         repeated measures variable and specifies the
#         (ordered) labels of the columns corresponding to 
#         that variable
# ...: extra repeated.measures.groups lists; table.to.long is 
#       recursively applied to the result of 
#       table.to.long(x, repeated.measures.groups)      
#
# Examples
# x = 
# case age v1 v2 v3 v4 w1 w2 w3 w4 m1 m2
#    1  22  3  2  3  5  0  1  1  0  a  b
#    2  25  5  4  2  4  1  1  1  0  c  a
#    ... etc ...
# Example 1:
# table.to.long(x, list(v=c('v1','v2','v3','v4'),
#   w=c('w1','w2','w3','w4'))) yields
# case age m1 m2  v v.score  w w.score
#    1  22  a  b v1       3 w1       0
#    1  22  a  b v2       2 w2       1
#    1  22  a  b v3       3 w3       1
#    1  22  a  b v4       5 w4       0
#    2  25  c  a v1       5 w1       1
#    2  25  c  a v2       4 w2       1
#    2  25  c  a v3       2 w3       1
#    2  25  c  a v4       4 w4       0
#    ... etc ...
#
# Example 2:
# table.to.long(x, list(m=c('m1','m2')), 
#   list(v=c('v1','v2','v3','v4'),
#        w=c('w1','w2','w3','w4'))) yields
# case age  m m.score  v v.score  w w.score
#    1  22 m1       a v1       3 w1       0
#    1  22 m1       a v2       2 w2       1
#    1  22 m1       a v3       3 w3       1
#    1  22 m1       a v4       5 w4       0
#    1  22 m2       b v1       3 w1       0
#    1  22 m2       b v2       2 w2       1
#    1  22 m2       b v3       3 w3       1
#    1  22 m2       b v4       5 w4       0
#    2  25 m1       c v1       5 w1       1
#    2  25 m1       c v2       4 w2       1
#    2  25 m1       c v3       2 w3       1
#    2  25 m1       c v4       4 w4       0
#    2  25 m2       a v1       5 w1       1
#    2  25 m2       a v2       4 w2       1
#    2  25 m2       a v3       2 w3       1
#    2  25 m2       a v4       4 w4       0
#    ... etc ...
{
    r = repeated.measures.groups
    if (!is.data.frame(x)) {
        if (!is.matrix(x)) 
            stop(deparse(substitute(x)), 
            " is not a data.frame or a matrix.")
        else x = as.data.frame(x)
    }
    if (!is.list(r) || is.null(names(r))) 
        stop(deparse(substitute(repeated.measures.groups)), 
        " should be a named list.")
    if (any(diff(sapply(r, length)) != 0)) 
        stop("Number of variables in each repeated ",
             "measure group specified in\n        ", 
             deparse(substitute(repeated.measures.groups)), 
             "\n    should all be the same.")
    x1 = x[, !colnames(x) %in% unlist(r)]
    x2 = x[, r[[1]]]
    y = cbind(x1[rep(1:nrow(x1), each = ncol(x2)), ], r[[1]], 
        c(t(x2)))
    if (length(r) > 1) 
        for (i in 2:length(r)) {
            x2 = x[, r[[i]]]
            y = cbind(y, r[[i]], c(t(x2)))
        }
    colnames(y) = c(colnames(x1), c(t(outer(names(r), 
        c("", ".score"), paste, sep = ""))))
    rownames(y) = NULL
    if (length(list(...)) == 0) 
        return(y)
    else Recall(y, ...)
}
 

The block transpose of a matrix

Sometimes it's neat to be able to transpose the subblocks of a matrix: The "block transpose" of an n·k × m matrix can be defined as

Y =

⌈ Y1 ⌉ | ⋮ | ⌊ Yk ⌋

→

YBn

⌈ Y1' ⌉ | ⋮ | ⌊ Yk' ⌋

where the Y_i's are n × m matrices. The following routine performs this function in R:

# the function bt computes the block transpose
bt <-
function(y, n) {
  if(nrow(y)%%n!=0)
     stop(n,' is not a valid block size for this matrix.')
  m = ncol(y)
  k = nrow(y) %/% n
  matrix( aperm(array(t(y), c(m,n,k)), c(2,1,3)), , n, byrow=TRUE)
}

An example:

> idx = expand.grid(1:2,1:3,letters[1:4])
> m = matrix(paste(idx[,3],idx[,2],idx[,1],sep=''),nc=2,byrow=TRUE)
> m
      [,1]  [,2]
 [1,] "a11" "a12"
 [2,] "a21" "a22"
 [3,] "a31" "a32"
 [4,] "b11" "b12"
 [5,] "b21" "b22"
 [6,] "b31" "b32"
 [7,] "c11" "c12"
 [8,] "c21" "c22"
 [9,] "c31" "c32"
[10,] "d11" "d12"
[11,] "d21" "d22"
[12,] "d31" "d32"
> bt(m,3)
     [,1]  [,2]  [,3]
[1,] "a11" "a21" "a31"
[2,] "a12" "a22" "a32"
[3,] "b11" "b21" "b31"
[4,] "b12" "b22" "b32"
[5,] "c11" "c21" "c31"
[6,] "c12" "c22" "c32"
[7,] "d11" "d21" "d31"