Author: Walter Leite (University of Florida) & Zuchao Shen (University of Florida)
Date: 2022-08-30

# Package ‘SEMsens’

#### Walter Leite (University of Florida) & Zuchao Shen (University of Florida)

#### 2022-08-30

An analysis based on structural equation modeling may suffer from the impact of a potential missing confounder that may change its conclusions (Harring, McNeish, & Hancock, 2017). This package is a tool to evaluate the sensitivity of structural equation models to potential missing confounders using the Ant Colony Optimization (ACO; Colorni, Dorigo, & Maniezzo, 1992; Dorigo & Stützle, 2004; Socha & Dorigo, 2008) algorithm. The reference for the current package is: Leite, W., Shen, Z., Marcoulides, K., Fish, C., & Harring, J. (2022). Using ant colony optimization for sensitivity analysis in structural equation modeling. Structural Equation Modeling: A Multidisciplinary Journal, 29 (1), 47-56.

The current version includes three main functions and they are

- gen.sens.pars function: This function generates a set of required sensitivity parameters according to the rank of object function, with which the method has been developed by Socha & Dorigo (2008).

- sa.aco function: This function automatically performs the search for an omitted confounder in structural equation modeling using the ACO algorithm

- sens.table function: This function provides five summary tables of results produced in the sa.aco function.

Next, we use an example to illustrate how to perform sensitivity analysis in structural equation modeling.

## 1. Function sa.aco

Given a data set (or variance covariance matrix) and an analytic model, this function searches an omitted confounder to optimize the objective function. sa.aco function calls gen.sens.pars function to perform the algorithm together.

We need the following information/arguments to perform the search

- data: The data set used for an analysis. If there is no data available, use covariance matrix instead.
- sample.cov: Covariance matrix.
- sample.nobs: Number of observations for covariance matrix.
- model: The analytic model of interest.
- sens.model: Sensitivity analysis model template for structural equation modeling with a phantom variable. This is the model of interest with a phantom variable and sensitivity parameters added. See the example provided.
- opt.fun: Customized or preset object function for optimization.
- ……

To see more arguments, please run ?sa.aco in R or RStudio.

### An exmaple

The example is from: Kim, Y. S. G., & Schatschneider, C. (2017). Expanding the developmental models of writing: A direct and indirect effects model of developmental writing (DIEW). Journal of Educational psychology, 109(1), 35-50.

    # Load lavaan and SEMsens packages
    require(lavaan)
    require(SEMsens)
    set.seed(1)

    # STEP 1: Prepare data:
    # Lower diagonal correlation matrix in the study by Kim & Schatschneider (2017)
    lower = '
    1.00
    .40 1.00
    .40 .64 1.00
    .41 .66 .61 1.00
    .42 .52 .53 .61 1.00
    .34 .50 .46 .53 .48 1.00
    .42 .47 .41 .43 .47 .55 1.00
    .39 .46 .39 .30 .21 .30 .37 1.00
    .24 .31 .30 .31 .26 .32 .27 .56 1.00
    .33 .35 .35 .40 .31 .25 .35 .51 .42 1.00
    .30 .42 .36 .32 .24 .37 .43 .44 .37 .49 1.00'

    # Convert to full covariance matrix
    sample.cov = getCov(lower, sds = c(5.64,14.68,6.57,6.07,3.39,10.16,6.11,4.91,15.59,0.96,0.99),
                  names = c("Working_memory",
                            "Vocabulary",
                            "Grammar",
                            "Inference",
                            "ToM",
                            "TNL",
                            "Expository",
                            "Spelling",
                            "Sentence_copying",
                            "One_day",
                            "Castle"))

    # STEP 2: Set up analytic model and sensitivity anaysis model
      # The original analytic model
    model <-'Vocabulary~Working_memory
      Grammar~Working_memory
      Inference~Vocabulary+Grammar+Working_memory
      ToM~Vocabulary+Grammar+Working_memory
      Spelling~Working_memory
      Sentence_copying~Working_memory
      Discourse~Inference+ToM+Vocabulary+Grammar+Working_memory
      Writing~Spelling+Sentence_copying+Discourse

      Discourse=~TNL+Expository
      Writing=~One_day+Castle

      Vocabulary~~Grammar
      Grammar~~Sentence_copying
      Vocabulary~~Sentence_copying
      Grammar~~Spelling
      Vocabulary~~Spelling
      Inference~~ToM
      Discourse~~Sentence_copying
      Discourse~~Spelling
      Spelling~~Sentence_copying'

      # A sensitivity analysis model template, which additionally includes paths
      #  from a phantom variable to a set of variables (= number of sensitivity parameters)
      #   in the analytic model.
    sens.model <- 'Vocabulary~Working_memory
      Grammar~Working_memory
      Inference~Vocabulary+Grammar+Working_memory
      ToM~Vocabulary+Grammar+Working_memory
      Spelling~Working_memory
      Sentence_copying~Working_memory
      Discourse~Inference+ToM+Vocabulary+Grammar+Working_memory
      Writing~Spelling+Sentence_copying+Discourse

      Discourse=~TNL+Expository
      Writing=~One_day+Castle

      Vocabulary~~Grammar
      Grammar~~Sentence_copying
      Vocabulary~~Sentence_copying
      Grammar~~Spelling
      Vocabulary~~Spelling
      Inference~~ToM
      Discourse~~Sentence_copying
      Discourse~~Spelling
      Spelling~~Sentence_copying

      Working_memory ~ phantom1*phantom
      Grammar ~ phantom2*phantom
      Vocabulary ~ phantom3*phantom
      ToM ~ phantom4*phantom
      Inference ~ phantom5*phantom
      Spelling ~ phantom6*phantom
      Sentence_copying  ~ phantom7*phantom
      Discourse ~ phantom8*phantom
      Writing ~ phantom9*phantom
      phantom =~ 0 # added for mean of zero
      phantom ~~ 1*phantom'

    # STEP 3: Set up the paths of interest to be evaluated in sensitivity analysis.
      paths <- 'Vocabulary~Working_memory
      Grammar~Working_memory
      Inference~Vocabulary+Grammar+Working_memory
      ToM~Vocabulary+Grammar+Working_memory
      Spelling~Working_memory
      Sentence_copying~Working_memory
      Discourse~Inference+ToM+Vocabulary+Grammar+Working_memory
      Writing~Spelling+Sentence_copying+Discourse'

      # STEP 4: Perform sensitivity analysis.
    my.sa <-sa.aco(model = model, sens.model = sens.model, sample.cov = sample.cov,
                   sample.nobs = 193, k = 50, max.value= 2000, max.iter = 100,
                   opt.fun = 4, ## from significant to just significant
                   paths = paths, seed = 1, verbose = FALSE)
    # We set up a max iteration of 100 and solution archive length of 50 for
    # illustration purpose. Please specify a larger number of iteration (e.g., 1000),
    #  and a larger k (e.g., 100).

## 2. Function sens.tables

sens.tables function can help summarize the sensitivity analysis results. Beyond this function, investigators always can summarize the results by accessing to the results in sa.aco function.

    my.table <- sens.tables(my.sa)
    # Table 1: Summary of the sensitivity analysis for each path
    my.table[[1]]

    ##                                 model.est model.pvalue mean.est.sens
    ## Discourse~Grammar               0.1003703 2.544519e-01     0.1052304
    ## Inference~Working_memory        0.1245455 2.598216e-02     0.1232723
    ## Writing~Sentence_copying        0.1719948 4.222710e-02     0.1633071
    ## Discourse~Working_memory        0.1888038 7.941219e-03     0.1910799
    ## Discourse~Inference             0.1943264 3.608645e-02     0.1920310
    ## ToM~Working_memory              0.2036364 1.096483e-03     0.2030261
    ## Sentence_copying~Working_memory 0.2399999 3.306180e-04     0.2401460
    ## ToM~Vocabulary                  0.2565656 6.016058e-04     0.2540932
    ## Discourse~ToM                   0.2610613 1.360093e-03     0.2545997
    ## Discourse~Vocabulary            0.2589070 4.345386e-03     0.2615377
    ## ToM~Grammar                     0.2843434 1.345400e-04     0.2878169
    ## Inference~Grammar               0.2873737 1.336915e-05     0.2894558
    ## Spelling~Working_memory         0.3899999 2.976197e-11     0.3883753
    ## Writing~Spelling                0.3759830 3.442786e-05     0.3886019
    ## Grammar~Working_memory          0.3999999 5.308198e-12     0.3979180
    ## Vocabulary~Working_memory       0.3999999 5.308198e-12     0.3996722
    ## Inference~Vocabulary            0.4262626 3.005818e-11     0.4261592
    ## Writing~Discourse               0.4504663 3.396553e-07     0.4355599
    ##                                 min.est.sens max.est.sens
    ## Discourse~Grammar                 0.04912704    0.1774544
    ## Inference~Working_memory          0.10477421    0.1485761
    ## Writing~Sentence_copying          0.05199882    0.2657367
    ## Discourse~Working_memory          0.14529692    0.2331204
    ## Discourse~Inference               0.15029004    0.2484007
    ## ToM~Working_memory                0.17420102    0.2505132
    ## Sentence_copying~Working_memory   0.23129571    0.2588857
    ## ToM~Vocabulary                    0.22454770    0.3059992
    ## Discourse~ToM                     0.15889535    0.3087243
    ## Discourse~Vocabulary              0.23408924    0.3120485
    ## ToM~Grammar                       0.21758721    0.3477721
    ## Inference~Grammar                 0.23936198    0.3261455
    ## Spelling~Working_memory           0.36602307    0.4141131
    ## Writing~Spelling                  0.19253896    0.6603765
    ## Grammar~Working_memory            0.36081953    0.4178667
    ## Vocabulary~Working_memory         0.38836436    0.4096268
    ## Inference~Vocabulary              0.40745780    0.4641450
    ## Writing~Discourse                 0.16872738    0.5776849

    # Table 2:  Summary of the sensitivity parameters
    my.table[[2]]

    ##                              mean.phan    min.phan   max.phan
    ## Grammar~phantom          -0.0255901298 -0.19855864 0.14923981
    ## Working_memory~phantom   -0.0116100689 -0.21192127 0.16501694
    ## Inference~phantom        -0.0059654784 -0.15852757 0.19673890
    ## Vocabulary~phantom        0.0009162615 -0.07989683 0.06497973
    ## Sentence_copying~phantom  0.0040785197 -0.05962908 0.11357489
    ## Discourse~phantom         0.0302226186 -0.12647184 0.24583688
    ## Spelling~phantom          0.0366678782 -0.19658642 0.24672188
    ## ToM~phantom               0.0561965145 -0.23155307 0.49606813
    ## Writing~phantom           0.1132356278 -0.52889818 0.61827105

    # Table 3: The sensitivity parameters lead to the minimum coefficient for each
    my.table[[3]]

    ##                                 Working_memory~phantom Grammar~phantom
    ## Vocabulary~Working_memory                 -0.211921268     -0.18415235
    ## Grammar~Working_memory                    -0.211921268     -0.18415235
    ## Inference~Vocabulary                       0.002184957     -0.19855864
    ## Inference~Grammar                         -0.110365605     -0.15220596
    ## Inference~Working_memory                  -0.167003237     -0.04988583
    ## ToM~Vocabulary                            -0.030085236     -0.12832087
    ## ToM~Grammar                               -0.110365605     -0.15220596
    ## ToM~Working_memory                        -0.167003237     -0.04988583
    ## Spelling~Working_memory                    0.152931730     -0.01137375
    ## Sentence_copying~Working_memory           -0.155173551      0.08682558
    ## Discourse~Inference                        0.044223231     -0.06027779
    ## Discourse~ToM                             -0.034499544     -0.13024128
    ## Discourse~Vocabulary                       0.051240582     -0.06690173
    ## Discourse~Grammar                         -0.120907356      0.14923981
    ## Discourse~Working_memory                   0.165016936     -0.01754155
    ## Writing~Spelling                           0.048004382     -0.02181512
    ## Writing~Sentence_copying                   0.072626707      0.11784237
    ## Writing~Discourse                          0.072626707      0.11784237
    ##                                 Vocabulary~phantom ToM~phantom
    ## Vocabulary~Working_memory             -0.054220461  0.22938668
    ## Grammar~Working_memory                -0.054220461  0.22938668
    ## Inference~Vocabulary                  -0.030296701  0.20987620
    ## Inference~Grammar                      0.051535666 -0.21725287
    ## Inference~Working_memory              -0.030364651 -0.22577041
    ## ToM~Vocabulary                         0.003831129  0.27028636
    ## ToM~Grammar                            0.051535666 -0.21725287
    ## ToM~Working_memory                    -0.030364651 -0.22577041
    ## Spelling~Working_memory                0.049251697 -0.11155276
    ## Sentence_copying~Working_memory       -0.011157983  0.28069048
    ## Discourse~Inference                   -0.035560127 -0.08384902
    ## Discourse~ToM                         -0.079896827  0.49606813
    ## Discourse~Vocabulary                   0.062516527 -0.20132409
    ## Discourse~Grammar                     -0.013843073 -0.18486429
    ## Discourse~Working_memory              -0.017716484 -0.19524466
    ## Writing~Spelling                      -0.064244453  0.14765876
    ## Writing~Sentence_copying              -0.023845171  0.27678824
    ## Writing~Discourse                     -0.023845171  0.27678824
    ##                                 Inference~phantom Spelling~phantom
    ## Vocabulary~Working_memory              0.11169709       0.11546808
    ## Grammar~Working_memory                 0.11169709       0.11546808
    ## Inference~Vocabulary                   0.12562746       0.15355763
    ## Inference~Grammar                     -0.15852757       0.18009840
    ## Inference~Working_memory              -0.15021823      -0.05282820
    ## ToM~Vocabulary                         0.06902098       0.02181626
    ## ToM~Grammar                           -0.15852757       0.18009840
    ## ToM~Working_memory                    -0.15021823      -0.05282820
    ## Spelling~Working_memory               -0.09672352       0.17233148
    ## Sentence_copying~Working_memory       -0.07186973       0.14099539
    ## Discourse~Inference                    0.12930466       0.19994834
    ## Discourse~ToM                          0.14159692       0.24672188
    ## Discourse~Vocabulary                  -0.02696729      -0.10090585
    ## Discourse~Grammar                     -0.00735754       0.01807200
    ## Discourse~Working_memory              -0.14682888       0.06457722
    ## Writing~Spelling                      -0.09555416       0.20318504
    ## Writing~Sentence_copying               0.16155105      -0.18908332
    ## Writing~Discourse                      0.16155105      -0.18908332
    ##                                 Sentence_copying~phantom Discourse~phantom
    ## Vocabulary~Working_memory                     0.08826528        0.24583688
    ## Grammar~Working_memory                        0.08826528        0.24583688
    ## Inference~Vocabulary                          0.05705421        0.10077127
    ## Inference~Grammar                            -0.02665071       -0.08985596
    ## Inference~Working_memory                     -0.02098849       -0.08802815
    ## ToM~Vocabulary                               -0.03303541        0.07442881
    ## ToM~Grammar                                  -0.02665071       -0.08985596
    ## ToM~Working_memory                           -0.02098849       -0.08802815
    ## Spelling~Working_memory                      -0.05756797       -0.02387311
    ## Sentence_copying~Working_memory              -0.05369555        0.10337791
    ## Discourse~Inference                           0.02990099        0.13270063
    ## Discourse~ToM                                 0.11357489        0.17226044
    ## Discourse~Vocabulary                         -0.05219069        0.08786791
    ## Discourse~Grammar                            -0.02210510        0.13456352
    ## Discourse~Working_memory                      0.01005041        0.13126727
    ## Writing~Spelling                              0.05236181        0.05720109
    ## Writing~Sentence_copying                      0.05069574        0.08586779
    ## Writing~Discourse                             0.05069574        0.08586779
    ##                                 Writing~phantom
    ## Vocabulary~Working_memory           -0.12163204
    ## Grammar~Working_memory              -0.12163204
    ## Inference~Vocabulary                 0.08132341
    ## Inference~Grammar                   -0.28691915
    ## Inference~Working_memory             0.36057655
    ## ToM~Vocabulary                       0.41217884
    ## ToM~Grammar                         -0.28691915
    ## ToM~Working_memory                   0.36057655
    ## Spelling~Working_memory             -0.12680418
    ## Sentence_copying~Working_memory     -0.08211608
    ## Discourse~Inference                  0.11588441
    ## Discourse~ToM                        0.21492898
    ## Discourse~Vocabulary                 0.07410964
    ## Discourse~Grammar                    0.08140368
    ## Discourse~Working_memory             0.30589957
    ## Writing~Spelling                     0.61827105
    ## Writing~Sentence_copying             0.60389707
    ## Writing~Discourse                    0.60389707

    # Table 4: The sensitivity parameters lead to the maximum coefficient for each path
    my.table[[4]]

    ##                                 Working_memory~phantom Grammar~phantom
    ## Vocabulary~Working_memory                 -0.176061341      0.03160994
    ## Grammar~Working_memory                    -0.120907356      0.14923981
    ## Inference~Vocabulary                      -0.110365605     -0.15220596
    ## Inference~Grammar                          0.002184957     -0.19855864
    ## Inference~Working_memory                   0.165016936     -0.01754155
    ## ToM~Vocabulary                            -0.110365605     -0.15220596
    ## ToM~Grammar                                0.002184957     -0.19855864
    ## ToM~Working_memory                        -0.155173551      0.08682558
    ## Spelling~Working_memory                   -0.176061341      0.03160994
    ## Sentence_copying~Working_memory           -0.211921268     -0.18415235
    ## Discourse~Inference                       -0.014368362     -0.02012266
    ## Discourse~ToM                             -0.120907356      0.14923981
    ## Discourse~Vocabulary                      -0.110365605     -0.15220596
    ## Discourse~Grammar                         -0.211921268     -0.18415235
    ## Discourse~Working_memory                  -0.211921268     -0.18415235
    ## Writing~Spelling                           0.072626707      0.11784237
    ## Writing~Sentence_copying                  -0.138008251     -0.05072568
    ## Writing~Discourse                         -0.167003237     -0.04988583
    ##                                 Vocabulary~phantom ToM~phantom
    ## Vocabulary~Working_memory              0.055384298   0.1217345
    ## Grammar~Working_memory                -0.013843073  -0.1848643
    ## Inference~Vocabulary                   0.051535666  -0.2172529
    ## Inference~Grammar                     -0.030296701   0.2098762
    ## Inference~Working_memory              -0.017716484  -0.1952447
    ## ToM~Vocabulary                         0.051535666  -0.2172529
    ## ToM~Grammar                           -0.030296701   0.2098762
    ## ToM~Working_memory                    -0.011157983   0.2806905
    ## Spelling~Working_memory                0.055384298   0.1217345
    ## Sentence_copying~Working_memory       -0.054220461   0.2293867
    ## Discourse~Inference                    0.002323414   0.2029898
    ## Discourse~ToM                         -0.013843073  -0.1848643
    ## Discourse~Vocabulary                   0.051535666  -0.2172529
    ## Discourse~Grammar                     -0.054220461   0.2293867
    ## Discourse~Working_memory              -0.054220461   0.2293867
    ## Writing~Spelling                      -0.023845171   0.2767882
    ## Writing~Sentence_copying               0.046036288  -0.1319380
    ## Writing~Discourse                     -0.030364651  -0.2257704
    ##                                 Inference~phantom Spelling~phantom
    ## Vocabulary~Working_memory             -0.07860246       0.14301228
    ## Grammar~Working_memory                -0.00735754       0.01807200
    ## Inference~Vocabulary                  -0.15852757       0.18009840
    ## Inference~Grammar                      0.12562746       0.15355763
    ## Inference~Working_memory              -0.14682888       0.06457722
    ## ToM~Vocabulary                        -0.15852757       0.18009840
    ## ToM~Grammar                            0.12562746       0.15355763
    ## ToM~Working_memory                    -0.07186973       0.14099539
    ## Spelling~Working_memory               -0.07860246       0.14301228
    ## Sentence_copying~Working_memory        0.11169709       0.11546808
    ## Discourse~Inference                   -0.14558684       0.02211563
    ## Discourse~ToM                         -0.00735754       0.01807200
    ## Discourse~Vocabulary                  -0.15852757       0.18009840
    ## Discourse~Grammar                      0.11169709       0.11546808
    ## Discourse~Working_memory               0.11169709       0.11546808
    ## Writing~Spelling                       0.16155105      -0.18908332
    ## Writing~Sentence_copying               0.02871137       0.13761525
    ## Writing~Discourse                     -0.15021823      -0.05282820
    ##                                 Sentence_copying~phantom Discourse~phantom
    ## Vocabulary~Working_memory                    -0.02140238        0.02093407
    ## Grammar~Working_memory                       -0.02210510        0.13456352
    ## Inference~Vocabulary                         -0.02665071       -0.08985596
    ## Inference~Grammar                             0.05705421        0.10077127
    ## Inference~Working_memory                      0.01005041        0.13126727
    ## ToM~Vocabulary                               -0.02665071       -0.08985596
    ## ToM~Grammar                                   0.05705421        0.10077127
    ## ToM~Working_memory                           -0.05369555        0.10337791
    ## Spelling~Working_memory                      -0.02140238        0.02093407
    ## Sentence_copying~Working_memory               0.08826528        0.24583688
    ## Discourse~Inference                           0.02421053        0.04225371
    ## Discourse~ToM                                -0.02210510        0.13456352
    ## Discourse~Vocabulary                         -0.02665071       -0.08985596
    ## Discourse~Grammar                             0.08826528        0.24583688
    ## Discourse~Working_memory                      0.08826528        0.24583688
    ## Writing~Spelling                              0.05069574        0.08586779
    ## Writing~Sentence_copying                     -0.05513577        0.05556959
    ## Writing~Discourse                            -0.02098849       -0.08802815
    ##                                 Writing~phantom
    ## Vocabulary~Working_memory           -0.19619587
    ## Grammar~Working_memory               0.08140368
    ## Inference~Vocabulary                -0.28691915
    ## Inference~Grammar                    0.08132341
    ## Inference~Working_memory             0.30589957
    ## ToM~Vocabulary                      -0.28691915
    ## ToM~Grammar                          0.08132341
    ## ToM~Working_memory                  -0.08211608
    ## Spelling~Working_memory             -0.19619587
    ## Sentence_copying~Working_memory     -0.12163204
    ## Discourse~Inference                  0.44354405
    ## Discourse~ToM                        0.08140368
    ## Discourse~Vocabulary                -0.28691915
    ## Discourse~Grammar                   -0.12163204
    ## Discourse~Working_memory            -0.12163204
    ## Writing~Spelling                     0.60389707
    ## Writing~Sentence_copying             0.57441944
    ## Writing~Discourse                    0.36057655

    # Table 5: The sensitivity parameters lead to change in significance for each path
    my.table[[5]]

    ##                                      p.value  p.changed Working_memory~phantom
    ## Discourse~Grammar               2.544519e-01 0.04361667            -0.21192127
    ## Writing~Sentence_copying        4.222710e-02 0.05085436            -0.08329305
    ## Discourse~Inference             3.608645e-02 0.05138185            -0.05379439
    ## Inference~Working_memory        2.598216e-02 0.05154444            -0.17606134
    ## Writing~Discourse               3.396553e-07 0.06126915             0.07262671
    ## Vocabulary~Working_memory       5.308198e-12         NA                     NA
    ## Grammar~Working_memory          5.308198e-12         NA                     NA
    ## Inference~Vocabulary            3.005818e-11         NA                     NA
    ## Inference~Grammar               1.336915e-05         NA                     NA
    ## ToM~Vocabulary                  6.016058e-04         NA                     NA
    ## ToM~Grammar                     1.345400e-04         NA                     NA
    ## ToM~Working_memory              1.096483e-03         NA                     NA
    ## Spelling~Working_memory         2.976197e-11         NA                     NA
    ## Sentence_copying~Working_memory 3.306180e-04         NA                     NA
    ## Discourse~ToM                   1.360093e-03         NA                     NA
    ## Discourse~Vocabulary            4.345386e-03         NA                     NA
    ## Discourse~Working_memory        7.941219e-03         NA                     NA
    ## Writing~Spelling                3.442786e-05         NA                     NA
    ##                                 Grammar~phantom Vocabulary~phantom ToM~phantom
    ## Discourse~Grammar                  -0.184152355        -0.05422046  0.22938668
    ## Writing~Sentence_copying            0.009402667         0.02524991 -0.06890207
    ## Discourse~Inference                -0.113046928        -0.01528202  0.25107119
    ## Inference~Working_memory            0.031609938         0.05538430  0.12173446
    ## Writing~Discourse                   0.117842375        -0.02384517  0.27678824
    ## Vocabulary~Working_memory                    NA                 NA          NA
    ## Grammar~Working_memory                       NA                 NA          NA
    ## Inference~Vocabulary                         NA                 NA          NA
    ## Inference~Grammar                            NA                 NA          NA
    ## ToM~Vocabulary                               NA                 NA          NA
    ## ToM~Grammar                                  NA                 NA          NA
    ## ToM~Working_memory                           NA                 NA          NA
    ## Spelling~Working_memory                      NA                 NA          NA
    ## Sentence_copying~Working_memory              NA                 NA          NA
    ## Discourse~ToM                                NA                 NA          NA
    ## Discourse~Vocabulary                         NA                 NA          NA
    ## Discourse~Working_memory                     NA                 NA          NA
    ## Writing~Spelling                             NA                 NA          NA
    ##                                 Inference~phantom Spelling~phantom
    ## Discourse~Grammar                      0.11169709        0.1154681
    ## Writing~Sentence_copying               0.15064797       -0.1555506
    ## Discourse~Inference                    0.10060722        0.1050860
    ## Inference~Working_memory              -0.07860246        0.1430123
    ## Writing~Discourse                      0.16155105       -0.1890833
    ## Vocabulary~Working_memory                      NA               NA
    ## Grammar~Working_memory                         NA               NA
    ## Inference~Vocabulary                           NA               NA
    ## Inference~Grammar                              NA               NA
    ## ToM~Vocabulary                                 NA               NA
    ## ToM~Grammar                                    NA               NA
    ## ToM~Working_memory                             NA               NA
    ## Spelling~Working_memory                        NA               NA
    ## Sentence_copying~Working_memory                NA               NA
    ## Discourse~ToM                                  NA               NA
    ## Discourse~Vocabulary                           NA               NA
    ## Discourse~Working_memory                       NA               NA
    ## Writing~Spelling                               NA               NA
    ##                                 Sentence_copying~phantom Discourse~phantom
    ## Discourse~Grammar                             0.08826528       0.245836878
    ## Writing~Sentence_copying                     -0.05921337       0.001210933
    ## Discourse~Inference                           0.05885676       0.132617696
    ## Inference~Working_memory                     -0.02140238       0.020934066
    ## Writing~Discourse                             0.05069574       0.085867787
    ## Vocabulary~Working_memory                             NA                NA
    ## Grammar~Working_memory                                NA                NA
    ## Inference~Vocabulary                                  NA                NA
    ## Inference~Grammar                                     NA                NA
    ## ToM~Vocabulary                                        NA                NA
    ## ToM~Grammar                                           NA                NA
    ## ToM~Working_memory                                    NA                NA
    ## Spelling~Working_memory                               NA                NA
    ## Sentence_copying~Working_memory                       NA                NA
    ## Discourse~ToM                                         NA                NA
    ## Discourse~Vocabulary                                  NA                NA
    ## Discourse~Working_memory                              NA                NA
    ## Writing~Spelling                                      NA                NA
    ##                                 Writing~phantom
    ## Discourse~Grammar                    -0.1216320
    ## Writing~Sentence_copying              0.2289633
    ## Discourse~Inference                   0.2951056
    ## Inference~Working_memory             -0.1961959
    ## Writing~Discourse                     0.6038971
    ## Vocabulary~Working_memory                    NA
    ## Grammar~Working_memory                       NA
    ## Inference~Vocabulary                         NA
    ## Inference~Grammar                            NA
    ## ToM~Vocabulary                               NA
    ## ToM~Grammar                                  NA
    ## ToM~Working_memory                           NA
    ## Spelling~Working_memory                      NA
    ## Sentence_copying~Working_memory              NA
    ## Discourse~ToM                                NA
    ## Discourse~Vocabulary                         NA
    ## Discourse~Working_memory                     NA
    ## Writing~Spelling                             NA