The importance of replication
In the last years the field of psychology has focused more and more on the importance of replication, as original findings are often not replicated. To discover which findings are robust and which are more sample-specific, it important to replicate findings in an independent sample. Especially findings on neural activation using functional magnetic resonance imaging (fMRI) have been shown to have low test-retest reliability. Recently, there has also been discussion about reliability of relations between brain measures and behavior in neuroimaging research. These studies emphasize the need for large sample sizes in neuroimaging research, to be able to discover reliable brain markers that can account for individual differences between persons. Additionally, the replication of neuroimaging findings in childhood are still relatively understudied. Because of its design, the L-CID study can be used to test for replication effects in childhood, and to overcome those challenges.
Replication in the L-CID study
The design of the L-CID study allows for replication of findings in middle childhood. The early childhood cohort (ECC, followed from 3-10 year-olds) and middle childhood cohort (MCC, 7-14 years) overlap in age on two time points. Both in the fifth year of the ECC and first year of the MCC, children were 7-9 years old. In both years, children participated in an MRI lab visit. Thus, we have two large samples of 7-9 year-olds with behavioral and MRI data. Moreover, the measures and instructions that were used were similar, with a few exceptions (e.g., in the ECC, we added an fMRI self-concept task). Because of those similarities and the inclusion of these two independent cohorts, the L-CID study is an interesting project for the study of replication effects in childhood.
Development of social feedback processing and responses in childhood: an fMRI test-replication design
In a recent study, we used the data of both cohorts to test for replication effects of neural findings in childhood. More specifically, we used a test-replication design to examine replication of neural activation during social feedback processing and subsequent aggressive responses in childhood. Childhood is an important period to learn to regulate your emotions and behavior when receiving feedback from peers. For example, receiving negative peer feedback can lead to angry feelings or aggression. However, what happens in the brain when a child receives social feedback or responds to it, was not yet clear. Therefore, we tested for these neural correlates in the two independent samples of 7-9 year-olds (MCC: n=385, ECC: n=195).
When we investigated differences in neural activation when responding to positive, neutral or negative feedback in the whole group, we found similar activation patterns in both samples. Medial prefrontal regions showed most activation following positive and negative feedback, and might be responsive to feedback that is socially salient. More lateral prefrontal regions showed most activation following positive feedback and might be involved in inhibition processes.
Equivalence testing
However, there are of course differences between the two samples. For example, for children in the middle childhood cohort this was the first time they participated in the L-CID study, whereas for children in the early childhood cohort it was the fifth time. Thus, for the children in the ECC, there might have been some familiarity effects. Also, there were differences between the samples in the date of testing (2019 vs 2015) and in the number of trials in the task (60 vs 40). Therefore, we did not expect that the results in both samples were exactly the same. Indeed, we did find that some effects were no longer significant when tested in the replication sample. To still test whether these replication effects were comparable to the original results, we used equivalence testing. Equivalence testing tests whether you can reject the hypothesis that replication effects are large enough to be considered meaningful. Of course, it is important to define what is meant by ‘meaningful’, which is why you specify a smallest effect size of interest. There are different ways to specify this. For our study, we used the lower boundary of the confidence interval of the effect in the original sample. Our equivalence tests indicated that the effects that we did not replicate using significance testing were still considered meaningful.
Replication of individual differences analyses
Even though we were happy to see those similar patterns of activation in the two groups, we were even more interested to see whether we could replicate findings on individual differences. In the original sample, we found a brain-behavior correlation with the dorsolateral prefrontal cortex (DLPFC), a region that has often been found to be involved in controlling behavior. Children with more neural activation in this region when responding to feedback showed less aggression, or more behavioral control. Interestingly, we replicated this relation in the replication sample! Similarly, we were able to replicate a prior finding that children with more activation in this region during negative feedback also showed less aggression. Because of these replication effects, we could be more certain about our findings, suggesting that the DLPFC might play a regulatory role in inhibiting aggression in a social context.
Overall, I enjoyed working on this replication study. It was a good opportunity to dive into the different methods that can be used to compare replication results to original findings. For instance, we used equivalence testing but another interesting approach to consider might be correspondence testing. Whereas I had expected that it is most exciting to discover new findings for the first time, I actually was very enthusiastic when we were able to replicate findings! It gave me more confidence in the robustness of our results and therefore helped me better understand why some children are better at regulating their behavior than others.