开发者

More idiomatic way to join long-form data frame to compare against another field

开发者 https://www.devze.com 2023-04-01 05:03 出处:网络
I have a long-form data that represents a bunch of different configurations of different software builds that I\'m wanting to get both absolute and relative performance data out of.

I have a long-form data that represents a bunch of different configurations of different software builds that I'm wanting to get both absolute and relative performance data out of.

I have a bunch of parameters on a row, and then a bunch of rows with different phases (there may be many) and different software configurations (of which there may also be many).

The frame looks like this:

'data.frame':   88 obs. of  7 variables:
 $ build   : Ord.factor w/ 3 levels "a"<"b"<"c": 3 3 3 3 3 3 3 3 2 2 ...
 $ nodes   : num  1 1 1 1 1 1 1 1 1 1 ...
 $ vbuckets: num  1 1 1 1 1 1 1 1 1 1 ...
 $ items   : num  1e+05 1e+05 1e+05 1e+05 1e+06 1e+06 1e+06 1e+06 1e+05 1e+05 ...
 $ size    : num  1024 1024 10240 10240 1024 ...
 $ label   : Factor w/ 2 levels "phase1","phase2": 1 2 1 2 1 2 1 2 1 2 ...
 $ time    : num  12.348 0.576 18.814 0.708 12.654 ...

There is a particular subset of the rows that I want to consider "baseline" to compare everything against. For this, I'll add a new column called timecomp that contains the time relative to that baseline for that particular row. I did this with the following function:

buildComparison <- function(df, field, value) {
  df.m <- df[df[,field] == value,]

  for (v in unique(df$vbuckets)) {
    for (i in unique(df$items)) {
      for (s in unique(df$size)) {
        for (l in unique(df$label)) {
          replacement <- df.m[df.m$items == i &
                              df.m$size == s &
                              df.m$label == l, 'time']
          if (length(replacement) == 0) {
            replacement = NA
          }
          df[df$items == i &
             df$size == s &
             df$label == l &
             df$vbuckets == v, 'comptime'] <- replacement
        }
      }
    }
  }
  df
}

To that, I supply the identifying field and value of the baseline and then have it populate all of the related rows with the timings relative to that value. For example:

buildComparison(df2, 'build', 'c')

The function works, but it seems that there'd be something more applicative I could use here instead of nesting for loops with hard-coded values.

For every one row that contains the baseline for a particular config (here defined by the columns items, size, and phase, I have n output rows where the items, size and phase are the sam开发者_如何学Goe.

It seems like I could do this with join, but my initial attempts were not successful.

Here's some sample data in case someone wants to play around:

structure(list(build = structure(c(3L, 3L, 3L, 3L, 3L, 3L, 3L, 
3L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
1L), .Label = c("a", "b", "c"), class = c("ordered", "factor"
)), nodes = c(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 
1, 1, 1, 1, 1, 1, 4, 4, 4, 4, 4, 4, 4, 4, 1, 1, 1, 1, 1, 1, 1, 
1, 4, 4, 4, 4, 4, 4, 4, 4), vbuckets = c(1, 1, 1, 1, 1, 1, 1, 
1, 1, 1, 1, 1, 1, 1, 1, 1, 64, 64, 64, 64, 64, 64, 64, 64, 32, 
32, 32, 32, 32, 32, 32, 32, 16, 16, 16, 16, 16, 16, 16, 16, 8, 
8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 
1, 1, 1, 1, 1), items = c(1e+05, 1e+05, 1e+05, 1e+05, 1e+06, 
1e+06, 1e+06, 1e+06, 1e+05, 1e+05, 1e+05, 1e+05, 1e+06, 1e+06, 
1e+06, 1e+06, 1e+05, 1e+05, 1e+05, 1e+05, 1e+06, 1e+06, 1e+06, 
1e+06, 1e+05, 1e+05, 1e+05, 1e+05, 1e+06, 1e+06, 1e+06, 1e+06, 
1e+05, 1e+05, 1e+05, 1e+05, 1e+06, 1e+06, 1e+06, 1e+06, 1e+05, 
1e+05, 1e+05, 1e+05, 1e+06, 1e+06, 1e+06, 1e+06, 1e+05, 1e+05, 
1e+05, 1e+05, 1e+06, 1e+06, 1e+06, 1e+06, 1e+05, 1e+05, 1e+05, 
1e+05, 1e+06, 1e+06, 1e+06, 1e+06, 1e+05, 1e+05, 1e+05, 1e+05, 
1e+06, 1e+06, 1e+06, 1e+06, 1e+05, 1e+05, 1e+05, 1e+05, 1e+06, 
1e+06, 1e+06, 1e+06, 1e+05, 1e+05, 1e+05, 1e+05, 1e+06, 1e+06, 
1e+06, 1e+06), size = c(1024, 1024, 10240, 10240, 1024, 1024, 
10240, 10240, 1024, 1024, 10240, 10240, 1024, 1024, 10240, 10240, 
1024, 1024, 10240, 10240, 1024, 1024, 10240, 10240, 1024, 1024, 
10240, 10240, 1024, 1024, 10240, 10240, 1024, 1024, 10240, 10240, 
1024, 1024, 10240, 10240, 1024, 1024, 10240, 10240, 1024, 1024, 
10240, 10240, 1024, 1024, 10240, 10240, 1024, 1024, 10240, 10240, 
1024, 1024, 10240, 10240, 1024, 1024, 10240, 10240, 1024, 1024, 
10240, 10240, 1024, 1024, 10240, 10240, 1024, 1024, 10240, 10240, 
1024, 1024, 10240, 10240, 1024, 1024, 10240, 10240, 1024, 1024, 
10240, 10240), label = structure(c(1L, 2L, 1L, 2L, 1L, 2L, 1L, 
2L, 1L, 2L, 1L, 2L, 1L, 2L, 1L, 2L, 1L, 2L, 1L, 2L, 1L, 2L, 1L, 
2L, 1L, 2L, 1L, 2L, 1L, 2L, 1L, 2L, 1L, 2L, 1L, 2L, 1L, 2L, 1L, 
2L, 1L, 2L, 1L, 2L, 1L, 2L, 1L, 2L, 1L, 2L, 1L, 2L, 1L, 2L, 1L, 
2L, 1L, 2L, 1L, 2L, 1L, 2L, 1L, 2L, 1L, 2L, 1L, 2L, 1L, 2L, 1L, 
2L, 1L, 2L, 1L, 2L, 1L, 2L, 1L, 2L, 1L, 2L, 1L, 2L, 1L, 2L, 1L, 
2L), .Label = c("phase1", "phase2"), class = "factor"), time = c(12.348021, 
0.575608, 18.814306, 0.708066, 12.654132, 4.820617, 18.801025, 
5.418552, 32.871667, 38.323579, 32.708113, 62.34107, 33.598166, 
573.840734, 33.770479, 678.790893, 457.317751, 10.072406, 455.356456, 
21.282528, 506.433301, 75.11121, 512.225358, 168.57048, 236.888889, 
8.415059, 234.768481, 18.910127, 264.237635, 80.644928, 267.265349, 
160.335294, 135.459629, 8.553754, 137.933539, 18.143246, 149.846176, 
86.67074, 149.7233, 157.330215, 92.088142, 7.24865, 90.474223, 
21.981704, 96.142578, 87.830294, 98.100383, 145.455385, 64.255009, 
8.450853, 63.204967, 21.56206, 66.017211, 81.4217, 67.57036, 
143.30217, 50.546167, 11.142062, 50.294096, 31.728965, 52.567635, 
129.633426, 50.815687, 247.651576, 23.027685, 0.041731, 21.86194, 
0.042894, 22.882426, 0.049298, 21.724528, 0.045704, 42.779241, 
23.115027, 44.071894, 60.223718, 43.076999, 302.109365, 41.798205, 
542.886189, 22.691731, 0.044235, 21.342419, 0.045478, 21.52168, 
0.042024, 22.186233, 0.041645)), .Names = c("build", "nodes", 
"vbuckets", "items", "size", "label", "time"), row.names = c(696L, 
697L, 710L, 711L, 738L, 739L, 750L, 751L, 621L, 622L, 633L, 634L, 
657L, 658L, 669L, 670L, 409L, 410L, 420L, 421L, 442L, 443L, 453L, 
454L, 343L, 344L, 354L, 355L, 376L, 377L, 387L, 388L, 277L, 278L, 
288L, 289L, 310L, 311L, 321L, 322L, 211L, 212L, 222L, 223L, 244L, 
245L, 255L, 256L, 145L, 146L, 156L, 157L, 178L, 179L, 189L, 190L, 
79L, 80L, 90L, 91L, 112L, 113L, 123L, 124L, 549L, 550L, 561L, 
562L, 585L, 586L, 597L, 598L, 13L, 14L, 24L, 25L, 46L, 47L, 57L, 
58L, 477L, 478L, 489L, 490L, 513L, 514L, 525L, 526L), class = "data.frame")


I think this produces the same output as your function, although it was a little tricky to check, since merge likes to sort its output and also the columns end up in different places:

#Pull the baseline subset; assumes frame
# stored in dat
build_b <- subset(dat, build == "b")

#Change the time column name
colnames(build_b)[7] <- 'comptime'

#merge, or 'join' if you prefer
tmp2 <- merge(dat,build_b[,-(1:3)])

I 'checked' the output by comparing the sorted ratio of time/comptime of the two version and they were all equal.

0

精彩评论

暂无评论...
验证码 换一张
取 消