Back to schedule

Instructions

Advantages of Treating Dyad Members as Indistinguishable:

  • Simpler model with fewer parameters
  • More power in tests of actor and partner effects

Disadvantages of Treating Dyad Members as Indistinguishable:

  • If distinguishability makes a difference, then the model is wrong.
  • Sometimes the focus is on distinguishing variable and it is lost.
  • Some editors or reviewer will not allow you to do it.

To test for distinguishability we do two Runs:

  • Distinguishable (either interaction or two-intercept, results are the same)
    • Different Actor and Partner Effects
    • Main Effect of Distinguishing Factor
    • Heterogeneity of Variance: Add a weights = option
  • Indistinguishable (4 fewer parameters)
    • Same Actor and Partner Effects
    • No Main Effect of Distinguishing Factor
    • No weights = option

Run using method = "ML", not method = "REML" (or blank because REML is the default).

Note the number of parameters:

  • There should be 4 more than for the distinguishable run.

Note the -2LogLikelihood (deviance) for each model. Subtract the deviances and number of parameters to get a \(\chi^2\) with 4 df. Or simply have R do it for you with anova().

If \(\chi^2\) is not significant, then the data are consistent with the null hypothesis that the dyad members are indistinguishable. If however, \(\chi^2\) is significant, then the data are inconsistent with the null hypothesis that the dyad members are indistinguishable (i.e., dyad members are distinguishable in some way).

Riggs Example

Read in the pairwise dataset

library(dyadr)
library(dplyr)
library(nlme)

riggsp <- read.csv("riggsp.csv")

Indistinguishable Dyads Run

Make sure to use method = "ML".

apim_in <- gls(Sat_A ~ Anxiety_A + Anxiety_P, 
               data = riggsp,
               method = "ML",
               correlation = corCompSymm (form=~1|Dyad),
               na.action = na.omit)

summary(apim_in)
## Generalized least squares fit by maximum likelihood
##   Model: Sat_A ~ Anxiety_A + Anxiety_P 
##   Data: riggsp 
##        AIC      BIC    logLik
##   1992.925 2011.608 -991.4627
## 
## Correlation Structure: Compound symmetry
##  Formula: ~1 | Dyad 
##  Parameter estimate(s):
##       Rho 
## 0.6146424 
## 
## Coefficients:
##                Value Std.Error  t-value p-value
## (Intercept) 53.33882  1.492056 35.74854       0
## Anxiety_A   -1.78906  0.294169 -6.08174       0
## Anxiety_P   -1.27701  0.294169 -4.34109       0
## 
##  Correlation: 
##           (Intr) Anxt_A
## Anxiety_A -0.816       
## Anxiety_P -0.816  0.490
## 
## Standardized residuals:
##         Min          Q1         Med          Q3         Max 
## -3.23294978 -0.69300430  0.08146147  0.64791428  2.46536177 
## 
## Residual standard error: 6.672175 
## Degrees of freedom: 310 total; 307 residual

We can get the -2loglikelihood with the logLik() function

logLik(apim_in)
## 'log Lik.' -991.4627 (df=5)

Distingushable Dyads Run

Make sure to use method = "ML".

apim_di <- gls(Sat_A ~ Anxiety_A + Anxiety_P + Gender_A
               + Anxiety_A*Gender_A + Anxiety_P*Gender_A,
               data = riggsp,
               method = "ML",
               correlation = corCompSymm(form=~1|Dyad),
               weights = varIdent(form=~1|Genderstring),
               na.action = na.omit)

logLik(apim_di)
## 'log Lik.' -991.1211 (df=9)

Test for Distinguishablity

The following function call conducts the \(\chi^2\) test for distinguishability.

anova(apim_in, apim_di)
##         Model df      AIC      BIC    logLik   Test   L.Ratio p-value
## apim_in     1  5 1992.925 2011.608 -991.4627                         
## apim_di     2  9 2000.242 2033.871 -991.1211 1 vs 2 0.6832715  0.9534

But, you can always hand calulate it. For simplicity, in these calculations we use the customary -2*logLik, or the “deviance”.

\[\chi^2(df_{dist} - df_{indist}) = (-2)*logLik_{indist} - (-2)*logLik_{dist}\]

\(\chi^2(9 - 5) = 2*991.463 - 2*991.121 = 0.684\), \(p = .953\)

The null hypothesis is that the dyads are indistinguishable. We cannot reject the null hypothesis, so we conclude that there is no empirical evidence that dyad members should be differentiated by their gender.

In general, any two multilevel models can be compared with anova() if one model is a nested version of the other. Model A is said to nested version of Model B, if Model A is a simpler version of Model B, i.e., one with fewer parameters. You would compute maxmimum likelihood estimates for each model (using ML if the models have different fixed effect or REML if the two models have the same fixed effects) and subtract the deviance (-2*logLik) of Model B from Model A. Under the null hypothesis that the simplifications of Model A are true, that difference is distributed as chi square where the degrees of freedom is the number of extra parameters that Model B has.

Testing Model Significance

We can also test if the model as a whole explains a signicant amount of variance. To do this we need to run the two models again, except this time using Maximum Likelihood estimation (method = "ML) instead of REML. Then we simply use the anova() function to do a likelihood ratio test.

apim_di_two <- gls(Sat_A ~ Genderstring + Anxiety_A:Genderstring 
                   + Anxiety_P:Genderstring -1,
                   data = riggsp,
                   method = "ML",
                   correlation = corCompSymm(form=~1|Dyad),
                   weights = varIdent(form=~1|Genderstring),
                   na.action = na.omit)

apim_di_empty <- gls(Sat_A ~ 1,
                   data = riggsp,
                   method = "ML",
                   correlation = corCompSymm(form=~1|Dyad),
                   na.action = na.omit)

anova(apim_di_two, apim_di_empty)
##               Model df      AIC      BIC     logLik   Test  L.Ratio
## apim_di_two       1  9 2000.242 2033.871  -991.1211                
## apim_di_empty     2  3 2024.929 2036.139 -1009.4644 1 vs 2 36.68671
##               p-value
## apim_di_two          
## apim_di_empty  <.0001

Note this strategy can be employed whenever two different models, one of which is a more complex version of the other, are compared. Need to use ML if fixed parameters are dropped. Otherwise REML can be used. Note that if there are a large number of dyads, i.e., more than 400, one might want to use a measure of fit, e.g., AIC, BIC, or the SABIC, to compare the two models.

Back to schedule