Data Science Toolbox Equity Model

Equity formula for the Data Science Toolbox Group Assessments

The equity formula operates on the following principles:

1. It should be unresponsive around 1, so that small deviations result in little change to marks.
2. It should prevent the mean of all scores from increasing much, to reduce gaming the system.
3. It must asymptote to prevent extreme multipliers leading to inappropriate grade classes.
4. It must operate without complex interplay between marks, and be tolerant of a range of group sizes.

It is based on a model used in the Business School which uses the same basic structure.

The equity share formula uses the following curve to map a claimed proportion of contribution (first variable) into a score multiplier (second variable).

sharecurvedata=matrix(c(0,0,
                0.8,0.7,
                0.88,0.98,
                1.12,1.02,
                1.5,1.2,
                10,1.5),byrow=T,ncol=2)

This is accessed via the interpolated curve formula:

sharecurve=function(s,tmodel=sharecurvedata){ # s as a ratio of the expected work per person
    tmp=c(0,0)
    i=2
    while(s>=tmodel[i-1,1]){
        tmp[1]=min(tmodel[i,1],s)
        slope=(tmodel[i,2]-tmodel[i-1,2])/ (tmodel[i,1]-tmodel[i-1,1])
        distance= (tmp[1]-tmodel[i-1,1])
        tmp[2]=tmp[2] +distance*slope
        i=i+1
    }
    tmp[2]
}

The curve scales a mark by the proportional contribution:

sprop=seq(0,2,by=0.05)
plot(sprop,sapply(sprop,sharecurve),type="b",
     main="Equity formula for Data Science Toolbox",
     xlab="Claimed equity proportion as a fraction of the expected",
     ylab="Mark multiplier")
abline(a=0,b=1)

text(0.95,1.05,"Unresponsive around 1",adj=1)
text(1.5,1.1,"Asymptote",adj=0)

text(0.9,0.7,"Typically worse to unevenly divide",adj=0)

This curve can be accessed using a simple interface called shareformula for the marks x and the share s. Note that marks are thresholded, though in practice it would be unusual to award marks that could exceed these limits. The function is:

shareformula=function(x,s,tmodel=sharecurvedata){
    ss=s/sum(s)*length(s)
    ss=sapply(ss,sharecurve,tmodel=tmodel)
    
    xx=x*ss
    xx[xx<0]=0
    xx[xx>100]=100
    xx
}

Usage

Use the function shareformula to turn a mark into a scaled mark. Input raw scores as the first variable x and the share as the second variable s. Use sharecurve to obtain a proportional scaling without any mark truncation. Here is an example with three members, one of whom contributed 40% whereas the others only 30%:

sharecurve((4/10)/(1/3))

[1] 1.057895

shareformula(1,c(40,30,30))

[1] 1.0578947 0.9833333 0.9833333

Some examples:

Note that because of the limitation to operate on the relative contribution, without knowledge of the number of members in the group, the formula can be sensitive if used in large chunks (e.g. 5% intervals) at large group sizes. This is because a 5% change of contribution is a larger proportion when there are 4 or 5 members than when there are 2 or 3. For example, the difference between a 15% and 25% equity in a 5 person project represents an entire half-person difference in contribution, so is heavily reweighted, but the difference between 45% and 55% in a 2 person project is negligible.

Examples with a varying number of group members and a base mark of 70%.

examples=list(
  c(45,55),
  c(40,60),
  c(40,30,30),
  c(50,25,25),
  c(30,25,25,20),
  c(30,25,23,22),
  c(25,20,20,20,15),
  c(25,20,20,18,17)
  )
printexample=function(ex){
  res=shareformula(70,ex)
  print(paste("Example share: (",
              paste(ex,collapse=","),
              ") of a mark of 70 results in a scaled mark of: (",
              paste(round(res,digits=0),collapse=","),
              ")"))
  invisible(res)
}
invisible(sapply(examples,printexample))

[1] "Example share: ( 45,55 ) of a mark of 70 results in a scaled mark of: ( 69,71 )"
[1] "Example share: ( 40,60 ) of a mark of 70 results in a scaled mark of: ( 49,74 )"
[1] "Example share: ( 40,30,30 ) of a mark of 70 results in a scaled mark of: ( 74,69,69 )"
[1] "Example share: ( 50,25,25 ) of a mark of 70 results in a scaled mark of: ( 84,46,46 )"
[1] "Example share: ( 30,25,25,20 ) of a mark of 70 results in a scaled mark of: ( 74,70,70,49 )"
[1] "Example share: ( 30,25,23,22 ) of a mark of 70 results in a scaled mark of: ( 74,70,69,69 )"
[1] "Example share: ( 25,20,20,20,15 ) of a mark of 70 results in a scaled mark of: ( 76,70,70,70,46 )"
[1] "Example share: ( 25,20,20,18,17 ) of a mark of 70 results in a scaled mark of: ( 76,70,70,69,61 )"