Bishopblog catalogue (updated 26th Sept 2014)

Source: http://www.weblogcartoons.com/2008/11/23/ideas/

Those of you who follow this blog may have noticed a lack of
thematic coherence. I write about whatever is exercising my mind at the time,
which can range from technical aspects of statistics to the design of bathroom
taps. I decided it might be helpful to introduce a bit of order into this
chaotic melange, so here is a catalogue of posts by topic.

Language impairment, dyslexia and related disorders

The common childhood disorders that have been left out in the cold (1 Dec 2010)
What's in a name? (18 Dec 2010)
Neuroprognosis in dyslexia (22 Dec 2010)
Where commercial and clinical interests collide: Auditory processing disorder (6 Mar 2011)
Auditory processing disorder (30 Mar 2011)
Special educational needs: will they be met by the Green paper proposals? (9 Apr 2011)
Is poor parenting really to blame for children's school problems? (3 Jun 2011)
Early intervention: what's not to like? (1 Sep 2011)
Lies, damned lies and spin (15 Oct 2011)
A message to the world (31 Oct 2011)
Vitamins, genes and language (13 Nov 2011)
Neuroscientific interventions for dyslexia: red flags (24 Feb 2012)
Phonics screening: sense and sensibility (3 Apr 2012) What Chomsky doesn't get about child language (3 Sept 2012) Data from the phonics screen (1 Oct 2012)
Auditory processing disorder: schisms and skirmishes (27 Oct 2012)
High-impact journals (Action video games and dyslexia: critique) (10 Mar 2013) Overhyped genetic findings: the case of dyslexia (16 Jun 2013) The arcuate fasciculus and word learning (11 Aug 2013) Changing children's brains (17 Aug 2013)
Raising awareness of language learning impairments (26 Sep 2013) Good and bad news on the phonics screen (5 Oct 2013)
What is educational neuroscience? (25 Jan 2014)
Parent talk and child language (17 Feb 2014)
My thoughts on the dyslexia debate (20 Mar 2014)
Labels for unexplained language difficulties in children (23 Aug 2014)
International reading comparisons: Is England really do so poorly? (14 Sep 2014)

Autism

Autism diagnosis in cultural context (16 May 2011)
Are our ‘gold standard’ autism diagnostic instruments fit for purpose? (30 May 2011)
How common is autism? (7 Jun 2011)
Autism and hypersystematising parents (21 Jun 2011) An open letter to Baroness Susan Greenfield (4 Aug 2011)
Susan Greenfield and autistic spectrum disorder: was she misrepresented? (12 Aug 2011)

Psychoanalytic treatment for autism: Interviews with French analysts (23 Jan 2012)
The ‘autism epidemic’ and diagnostic substitution (4 Jun 2012)
How wishful thinking is damaging Peta's cause (9 June 2014)

Developmental disorders/paediatrics

The hidden cost of neglected tropical diseases (25 Nov 2010)
The National Children's Study: a view from across the pond (25 Jun 2011)
The kids are all right in daycare (14 Sep 2011) Moderate drinking in pregnancy: toxic or benign? (21 Nov 2012) Changing the landscape of psychiatric research (11 May 2014)

Genetics

Where does the myth of a gene for things like intelligence come from? (9 Sep 2010)
Genes for optimism, dyslexia and obesity and other mythical beasts (10 Sep 2010)
The X and Y of sex differences (11 May 2011)
Review of How Genes Influence Behaviour (5 Jun 2011)
Getting genetic effect sizes in perspective (20 Apr 2012) Moderate drinking in pregnancy: toxic or benign? (21 Nov 2012) Genes, brains and lateralisation (22 Dec 2012) Genetic variation and neuroimaging (11 Jan 2013) Have we become slower and dumber? (15 May 2013) Overhyped genetic findings: the case of dyslexia (16 Jun 2013)

Neuroscience

Neuroprognosis in dyslexia (22 Dec 2010) Brain scans show that… (11 Jun 2011) 
Time for neuroimaging (and PNAS) to clean up its act (5 Mar 2012)
Neuronal migration in language learning impairments (2 May 2012)
Sharing of MRI datasets (6 May 2012)
Genetic variation and neuroimaging (1 Jan 2013) The arcuate fasciculus and word learning (11 Aug 2013) Changing children's brains (17 Aug 2013)
What is educational neuroscience? ( 25 Jan 2014) Changing the landscape of psychiatric research (11 May 2014)

Statistics

Book review: biography of Richard Doll (5 Jun 2010)
Book review: the Invisible Gorilla (30 Jun 2010)
The difference between p < .05 and a screening test (23 Jul 2010)
Three ways to improve cognitive test scores without intervention (14 Aug 2010)
A short nerdy post about the use of percentiles (13 Apr 2011)
The joys of inventing data (5 Oct 2011)
Getting genetic effect sizes in perspective (20 Apr 2012) Causal models of developmental disorders: the perils of correlational data (24 Jun 2012) Data from the phonics screen (1 Oct 2012)Moderate drinking in pregnancy: toxic or benign? (1 Nov 2012) Flaky chocolate and the New England Journal of Medicine (13 Nov 2012) Interpreting unexpected significant results (7 June 2013) Data analysis: Ten tips I wish I'd known earlier (18 Apr 2014) Data sharing: exciting but scary (26 May 2014)
Percentages, quasi-statistics and bad arguments (21 July 2014)

Journalism/science communication

Orwellian prize for scientific misrepresentation (1 Jun 2010)
Journalists and the 'scientific breakthrough' (13 Jun 2010)
Science journal editors: a taxonomy (28 Sep 2010)
Orwellian prize for journalistic misrepresentation: an update (29 Jan 2011)
Academic publishing: why isn't psychology like physics? (26 Feb 2011)
Scientific communication: the Comment option (25 May 2011)
Accentuate the negative (26 Oct 2011)
Publishers, psychological tests and greed (30 Dec 2011)
Time for academics to withdraw free labour (7 Jan 2012)
Novelty, interest and replicability (19 Jan 2012)
2011 Orwellian Prize for Journalistic Misrepresentation (29 Jan 2012)
Time for neuroimaging (and PNAS) to clean up its act (5 Mar 2012)
Communicating science in the age of the internet (13 Jul 2012) How to bury your academic writing (26 Aug 2012)
High-impact journals: where newsworthiness trumps methodology (10 Mar 2013)
Blogging as post-publication peer review (21 Mar 2013) A short rant about numbered journal references (5 Apr 2013) Schizophrenia and child abuse in the media (26 May 2013) Why we need pre-registration (6 Jul 2013)
On the need for responsible reporting of research (10 Oct 2013)
A New Year's letter to academic publishers (4 Jan 2014)

Social Media

A gentle introduction to Twitter for the apprehensive academic (14 Jun 2011)
Your Twitter Profile: The Importance of Not Being Earnest (19 Nov 2011)
Will I still be tweeting in 2013? (2 Jan 2012)
Blogging in the service of science (10 Mar 2012) Blogging as post-publication peer review (21 Mar 2013)
The impact of blogging on reputation ( 27 Dec 2013) WeSpeechies: A meeting point on Twitter (12 Apr 2014)

Academic life

An exciting day in the life of a scientist (24 Jun 2010)
How our current reward structures have distorted and damaged science (6 Aug 2010)
The challenge for science: speech by Colin Blakemore (14 Oct 2010)
When ethics regulations have unethical consequences (14 Dec 2010)
A day working from home (23 Dec 2010)
Should we ration research grant applications? (8 Jan 2011)
The one hour lecture (11 Mar 2011)
The expansion of research regulators (20 Mar 2011)
Should we ever fight lies with lies? (19 Jun 2011)
How to survive in psychological research (13 Jul 2011)
So you want to be a research assistant? (25 Aug 2011)
NHS research ethics procedures: a modern-day Circumlocution Office (18 Dec 2011)
The REF: a monster that sucks time and money from academic institutions (20 Mar 2012)
The ultimate email auto-response (12 Apr 2012)
Well, this should be easy…. (21 May 2012) Journal impact factors and REF2014 (19 Jan 2013)  An alternative to REF2014 (26 Jan 2013) Postgraduate education: time for a rethink (9 Feb 2013) High-impact journals: where newsworthiness trumps methodology (10 Mar 2013)
Ten things that can sink a grant proposal (19 Mar 2013)Blogging as post-publication peer review (21 Mar 2013) The academic backlog (9 May 2013) Research fraud: More scrutiny by administrators is not the answer (17 Jun 2013) Discussion meeting vs conference: in praise of slower science (21 Jun 2013) Why we need pre-registration (6 Jul 2013)
Evaluate, evaluate, evaluate (12 Sep 2013)
High time to revise the PhD thesis format (9 Oct 2013)
The Matthew effect and REF2014 (15 Oct 2013)
Pressures against cumulative research (9 Jan 2014)
Why does so much research go unpublished? (12 Jan 2014) The University as big business: the case of King's College London (18 June 2014)
Should vice-chancellors earn more than the prime minister? (12 July 2014)

Replication and reputation: Whose career matters? (29 Aug 2014)


 

Celebrity scientists/quackery

Three ways to improve cognitive test scores without intervention (14 Aug 2010) What does it take to become a Fellow of the RSM? (24 Jul 2011)
An open letter to Baroness Susan Greenfield (4 Aug 2011)
Susan Greenfield and autistic spectrum disorder: was she misrepresented? (12 Aug 2011)
How to become a celebrity scientific expert (12 Sep 2011) The kids are all right in daycare (14 Sep 2011) 
The weird world of US ethics regulation (25 Nov 2011)
Pioneering treatment or quackery? How to decide (4 Dec 2011) Psychoanalytic treatment for autism: Interviews with French analysts (23 Jan 2012) Neuroscientific interventions for dyslexia: red flags (24 Feb 2012)

Women

Academic mobbing in cyberspace (30 May 2010)
What works for women: some useful links (12 Jan 2011)

The burqua ban: what's a liberal response (21 Apr 2011) C'mon sisters! Speak out! (28 Mar 2012)
Psychology: where are all the men? (5 Nov 2012)
Men! what you can do to improve the lot of women ( 25 Feb 2014) Should Rennard be reinstated? (1 June 2014)

Politics and Religion

Lies, damned lies and spin (15 Oct 2011) A letter to Nick Clegg from an ex liberal democrat (11 Mar 2012)
BBC's 'extensive coverage' of the NHS bill (9 Apr 2012)
Schoolgirls' health put at risk by Catholic view on vaccination (30 Jun 2012)
A letter to Boris Johnson (30 Nov 2013)
How the government spins a crisis (floods) (1 Jan 2014)






Humour and miscellaneous

Orwellian prize for scientific misrepresentation (1 Jun 2010)
An exciting day in the life of a scientist (24 Jun 2010)
Science journal editors: a taxonomy (28 Sep 2010)
Parasites, pangolins and peer review (26 Nov 2010)
A day working from home (23 Dec 2010)
The one hour lecture (11 Mar 2011)
The expansion of research regulators (20 Mar 2011)
Scientific communication: the Comment option (25 May 2011)
How to survive in psychological research (13 Jul 2011)
Your Twitter Profile: The Importance of Not Being Earnest (19 Nov 2011)
2011 Orwellian Prize for Journalistic Misrepresentation (29 Jan 2012)
The ultimate email auto-response (12 Apr 2012)
Well, this should be easy…. (21 May 2012)
The bewildering bathroom challenge (19 Jul 2012) Are Starbucks hiding their profits on the planet Vulcan? (15 Nov 2012) Forget the Tower of Hanoi (11 Apr 2013) How do you communicate with a communications company? ( 30 Mar 2014)
Noah: A film review from 32,000 ft (28 July 2014)

Parent talk and child language

© www.CartoonStock.com

There's been a lot in the media lately about the impacts of parental talk on children's language development. Some of it has been opinion, as in this piece in the Daily Telegraph, in which the headline proclaimed that children were "starting school unable to speak". This reflected the views of a head teacher, who claimed that the proportion of children with poor language skills had increased in his lifetime, and that this was the fault of parents who did not have time to talk to their children any more. There is nothing new here: versions of this story pop up every few years or so (here's one from 2003,  and a blogpost on another case from 2011): Editors know that stories about feckless parents sell newspapers: readers love the sense of complacency and moral superiority they induce.



But there is also more evidence-based stuff. Some children do have serious difficulties mastering spoken language, and there is research demonstrating links between parent talk and child language outcomes. We've known since the influential study of Hart and Risley (1995) that there is massive variation in the amount of language children are exposed to at home, and this is predicted by socio-economic status. There are many subsequent studies showing positive associations between aspects of the language that babies and toddlers hear and the rate and complexity of their language development.



When the Guardian ran a piece last week on the latest of these studies, someone tweeted "do we really need a study to demonstrate that?" – to most people it's blindingly obvious that children's language development will be determined by the language that they hear at home. This assumption is shared by many professionals in the field of language development; for instance, in a recent review, Leffel and Suskind (2013) describe poor attainment of children from disadvantaged homes and unambiguously state: "Parent linguistic input lies at the heart of the problem".



Except that it's not so simple. And the complexities become apparent when we look at the type of evidence that we have, which is mostly correlational. Students learn in Psychology #101 that correlation does not equal causation, yet when a causal interpretation seems so obvious to most people, this can get forgotten. I have lost count of the number of times I've seen a study showing that parent talk predicts child language development, where the conclusion drawn by the authors (and press offices and the media) is that limited parental language causes child language problems. No other explanation is even countenanced. Yet if we were well taught in Psychology #101, we would realise that we need to consider alternative explanations for the observed association. The figure below shows three possible causal models; these are not mutually exclusive and so all could play a role.

Different Models to account for association between parent talk and child language

Model A is the one that is typically assumed by most people: parent talk to children boosts their language development, and accordingly, if a child has poor language skills, this is likely to be caused by inadequate talk from parents.



In Model B, the association goes in the other direction. Poor language in the child leads to less talk from the parent. This could occur if, for instance, parents are discouraged from talking to a child who is unresponsive and appears not to understand. Consider too, this recent study looking at outcomes of infants in a special care baby unit . Children who were exposed to more adult language in hospital had better language outcomes; however, as the authors noted, "It could be that parents and caregivers have more opportunity to talk to infants who are less sick."



Model C explains the association without postulating a direct link from parental talk to child language. Instead it sees both of these as outcomes of some other cause. This could be an environmental factor, such as poor diet, or a genetic risk that is shared by parents and their children.



It is the job of researchers to try and find evidence to establish the relative importance of these different causal routes. In the case of child language, this is not just a theoretical exercise: it potentially makes a difference to the kinds of intervention that are likely to be effective in helping children. In particular, if model A is the main explanation for the association, then we should be able to boost poor child language by encouraging reticent parents to interact more like talkative parents. This is unlikely to be effective if model B explains the association. And if model C applies, then we would need to either modify the third factor (X) itself, or clarify how it operated in order to alter its association with poor outcomes in children.



I am concerned about the near-universal acceptance of model A as the sole explanation, because there are two lines of evidence that go against it. First, we can to some extent disentangle the impact of socioeconomic disadvantage and parental talk if we study children whose parents produce little spoken language input because they have a congenital hearing impairment. Some profoundly deaf parents have children with normal hearing. In the past there was concern about such children: how would they learn spoken language if their parents produced little intelligible speech? In fact, the studies that were done obtained unexpectedly positive results, leading to the conclusion that although young children clearly need some exposure to spoken language in order to learn to speak, they could develop normal language on the basis of exposure to other adults outside the home and language on TV (Schiff-Myers, 1988).



The second line of evidence comes from studies that disentangle genetic and environmental influences by considering language development in twins. If parental talk is an important determinant of child language, then we would expect twins growing up together in the same home to resemble each other. However, if model A is all-important, we would not expect the genetic relationship between the twins to have any effect. But it does make a difference, and on many language measures this effect is quite substantial. So we find that twins do resemble each other in general, but that resemblance is quite a bit higher if the twins are genetically identical (monozygotic) than if they are fraternal (dizygotic, and sharing around half their DNA for genes that vary between people).



I remember being struck when I first did twin studies of children's language difficulties at how different two twins growing up in the same family could be – provided they were non-identical. It was, however, unusual to find identical twin pairs where one had a significant language problem and the other was unaffected. The overall pattern of results tells us that the child's genetic makeup plays a role in determining their language development (Bishop, 2006).



So what has this to do with models A, B and C? Quite simply, the twin data support a version of model C: given that genes affect language development, we expect parents (who share around half their genes with their children) to resemble their children. We already know that parents of children with language impairments are more likely than other parents to have some kind of language or literacy problem themselves (Barry et al, 2007). This doesn't affect everyone: of course there are many literate and articulate parents whose children have language difficulties. But on balance, these kinds of difficulties run through generations, and we therefore expect to see an association between limited language ability in parents and language difficulties in their children. Note that a genetic account will also predict that language difficulties in children will predominate among those of lower social-economic status: parents who themselves are language-impaired are likely to have low levels of educational attainment and poor occupational prospects.



This kind of genetic explanation for parent-child similarities has a lot of evidential support, but people are very reluctant to accept it. If you propose that genes may play a role in children's developmental difficulties, people will tend to assume that you have a political agenda aligned with the Third Reich, with a goal of identifying a genetic underclass who should not be helped because they are just 'made that way'. This reflects a wrong-headed genetic determinism that is at odds with contemporary understanding of how genes work. Genes do not determine your fate: their impact is likely to vary according to the environment, and by modifying environments we may alter outcomes. Unlike in model A, though, model C predicts that sensitivity to specific environments may depend on one's genes. The arguments have been cogently put in a recent book by Asbury and Plomin (2013), who lament the way in which genetic influences on children's development have been ignored in favour of a political stance that blames educational and developmental difficulties on either poor parenting or poor teaching. If, as has been repeatedly shown, there is evidence that genes are important in influencing children's language development, then we may be squandering our intervention resources by ignoring this fact.



The bottom line is that we need more research. Well-conducted randomized controlled trials on the impact of modifying parent input have been thin on the ground to date, and have not generated impressive evidence of efficacy (see my earlier blogpost) . Obviously, it's early days, and I'd cheer on others who are attempting such research. Results may depend on the nature of the intervention, the aspects of language that are assessed, and the type of population the intervention is used with. My suggestion is that rather than denying the reality of genetic effects, we should be conducting research to find out what kinds of input are most effective for children who are at genetic risk. It is possible that rather than more language input, they may do best with a different kind of language input, specifically tailored to take into account their cognitive strengths and weaknesses. We are a long way from understanding how best to do this, and meanwhile, ingenious and dedicated practitioners are working hard to tackle the very real problems that some children experience. My message is simply that to lay the blame for these difficulties at the door of parents, and to anticipate that problems can be readily overcome by encouraging parents to talk more to their children may be oversimplistic.



To finish, I cannot resist adding my favourite quote from Richard Dawkins, which focuses on mathematics rather than language learning, but gets to the nub of inappropriate concerns about genetic explanations:



People seem to have little difficulty in accepting the modifiability of "environmental" effects on human development. If a child has had bad teaching in mathematics, it is accepted that the resulting deficiency can be remedied by extra good teaching the following year. But any suggestion that the child's mathematical deficiency might have a genetic origin is likely to be greeted with something approaching despair: if it is in the genes "it is written", it is "determined" and nothing can be done about it: you might as well give up attempting to teach the child mathematics. This is pernicious rubbish on an almost astrological scale ..... What did genes do to deserve their sinister juggernaut-like reputation? Why do we not make a similar bogey out of, say, nursery education or confirmation classes? Why are genes thought to be so much more fixed and inescapable in their effects than television, nuns, or books? 

Richard Dawkins (1982) The extended phenotype, Oxford University Press (p. 13) 

References 

Asbury, K., & Plomin, R. (2013). G is for genes: The impact of genetics on education and achievement. Chichester: Wiley Blackwell.

Barry, J. G., Yasin, I., & Bishop, D. V. M. (2007). Heritable risk factors associated with language impairments. Genes, Brain and Behavior, 6, 66-76.

Bishop, D. V. M. (2006). What causes specific language impairment in children? Current Directions in Psychological Science, 15, 217-221.

Caskey, M., Stephens, B., Tucker, R., & Vohr, B. (2014). Adult talk in the NICU with preterm infants and developmental outcomes Pediatrics DOI: 10.1542/peds.2013-0104

Hart, B., & Risley, T. R. (1995). Meaningful differences in the everyday experience of young American children. Baltimore, MD: Paul H. Brookes Publishing Co.

Leffel, K., & Suskind, D. (2013). Parent-directed approaches to enrich the early language environments of children living in poverty. Seminars in Speech and Language, 34(4), 267-277. doi: 10.1055/s-0033-1353443
Schiff-Myers, N. (1988). Hearing children of deaf parents. In D. Bishop & K. Mogford (Eds.), Language development in exceptional circumstances (pp. 47-61). Edinburgh: Churchill Livingstone.



This article (Figshare version) can be cited as:
Bishop, Dorothy V M (2014): Parent talk and child language. figshare.
http://dx.doi.org/10.6084/m9.figshare.1030407

The arcuate fasciculus and word learning: a critique

The arcuate fasciculus is a white matter tract linking areas in the temporal lobe involved in interpreting speech with areas in the frontal lobe that control motor movements. Its role in language was established years ago when it was proposed that conduction aphasia, characterised by poor repetition despite good understanding and fluent spontaneous speech, was a disconnection syndrome resulting from lesions of the arcuate fasciculus.



Compared with apes and monkeys, humans have much stronger structural connections between temporal and frontal regions of the brain, suggesting that evolution of the arcuate fasciculus played a key role in language evolution.



Study of white matter tracts in the brain has advanced rapidly since the advent of diffusion tensor imaging (DTI). DTI makes it possible to measure parameters such as fractional anisotropy and radial diffusivity, indirect measures of myelination and/or axonal density within white matter.



Use of DTI has revealed an intriguing aspect of the arcuate fasciculus: it shows wide individual variation. In most people, the left arcuate fasciculus is larger than the right, but in some a more bilateral pattern is seen, and in others, a right arcuate fasciculus may not be visible on DTI. This immediately raises the question of whether this individual variation corresponds to functional differences in language ability. Two studies considered whether the degree of lateralisation of the arcuate fasciculus related to language level, but they obtained conflicting results. Lebel and Beaulieu (2009) found that laterality of the arcuate fasciculus, measured on diffusion tensor imaging, was modestly correlated (r = 0.32) with receptive vocabulary in 68 children, with the highest scores for those with strong left lateralization. However, a study of adults found no relation between left lateralization of the arcuate fasciculus and vocabulary; instead, higher verbal memory was found to be associated with weak lateralisation.



A couple of weeks ago, López-Barroso et al published a paper in the Proceedings of the National Academy of Sciences claiming that structural and functional  measures of the left arcuate fasciculus predicted word learning ability. The authors started with 27 young adults who had brain scans that yielded measures of structural and functional connectivity between temporal and frontal language areas of the brain. Twenty of these individuals also did a learning task while in the scanner. They heard a rapid sequence of novel words, each consisting of three syllables, and were asked to concentrate on them, as they would be asked to recognise them later. After this learning phase, they were presented with the same nonwords mixed in with other nonwords made from the same syllables in a different order, and were asked to make a left or right keypress to indicate if each item was familiar or not. Their responses were transformed into a measure called d-prime, which indicates how well the person discriminates between familiar and unfamiliar items.

Figure 1A from López-Barroso et al, showing the learning task 

From previous research, one might have expected to see an association between nonword learning and lateralisation of the arcuate fasciculus. This was not found, but accuracy in the nonword learning task was significantly correlated with structural and functional measures of strength of connectivity in the left hemisphere. The authors’ conclusion is given in the title of the paper: “Word learning is mediated by the left arcuate fasciculus”.



Given what we know about the arcuate fasciculus, this is a plausible finding, but how robust is the evidence? I think there are at least three problems with this study, which lead me to be cautious about accepting its claims.



First, there is the perennial problem of multiple comparisons. The authors considered three different DTI measures (number of streamlines, fractional anisotropy and radial diffusivity) for left and right sides of four tracts (arcuate long, arcuate anterior, arcuate posterior, and inferior fronto-occipital fasciculus). They used, however, a Bonferroni correction appropriate for 8 correlations (p = .0062) rather than for 24 correlations (.002).  None of the reported correlations is significant if the appropriate correction is used.



Second, the authors emphasised that the correlation between word learning and radial diffusivity was significant only for the direct arcuate tract in the left hemisphere. This, however, confuses difference in significance levels with significance of differences: as Nieuwenhuis et al (2011)  remarked: "when making a comparison between two effects, researchers should report the statistical significance of their difference rather than the difference between their significance levels". Table 1 shows the correlations of radial diffusivity with nonword learning for different regions, with 95% confidence intervals added, and it is clear that there is overlap between these. In other words, these correlations do not differ significantly from one another. See here for further discussion of these issues.

Table 1: Correlations (r) between nonword learning and radial diffusivity in different pathways, with 95% confidence intervals

In this study, the problem is compounded by the fact that different subsets of individuals are included in the correlations for different brain regions. It is not unusual to have to exclude participants from DTI studies because of measurement difficulties, but this does mean that when comparing one brain region with another one is not comparing like with like. And since statistical significance depends on sample size, if this varies from brain region to brain region, this further complicates interpretation. This is evident from Figures 2 and 3 of the López-Barroso et al paper; in both cases the absolute value of the correlation is .42, yet for radial diffusivity of the right posterior segment, this is dismissed as nonsignificant (with N = 19), whereas for the  fMRI analysis it is heralded as significant (with N = 25).



To establish what results would look like if the same subset of participants was used in all analyses, I requested the raw data for radial diffusivity from the first author, who kindly provided it. There were just 13 participants with DTI data for all brain regions: if analysis was restricted to them, then just one of the correlations with word learning was significant by the authors' criterion, that with the right posterior arcuate fasciculus (r = .73, p = .005). This analysis does not prove that this pathway is important: rather, it emphasises that a similar pattern of associations is seen in all pathways, and the study  is underpowered to detect reliable associations, particularly if the interest is in selective associations with one pathway and not another.



Perhaps of greatest concern, though, is the measure of ‘word learning’. For a start, this was not word learning in the usual sense, as the participants were not required to associate speech sounds with meanings. Instead, they had to recognise familiar strings of meaningless sounds. There is a serious oddity about the results. Measures of d-prime usually range from zero (no ability to discriminate familiar from unfamiliar items, i.e. chance performance) to 2 or 3 (highly significant ability to discriminate familiar from unfamiliar items). But in this study, five of the twenty participants obtained negative values of d-prime. A negative value means performance is below chance: i.e., the person was more likely to treat the unfamiliar items as familiar, and vice versa. This is frankly weird, and makes one wonder whether some participants simply got confused about which key corresponded to which response. The authors give a different explanation: “Negative values indicate discrimination is achieved but individuals segmented incorrectly, classifying nonwords as words of the artificial language.” I find this unconvincing, as it would only make sense if the distractor items were made by taking sequences from the original input that crossed word boundaries: this does not seem to have been the case. But even if it were the explanation, does it make sense to treat those who discriminate the nonwords, but segment them wrongly, as doing worse on word learning than those who don’t discriminate the nonwords at all?



Does this matter? I re-ran the correlations excluding four participants with a negative d-prime value of less than -0.42 (which as far as I can work out corresponds to below chance performance). The correlations no longer reached conventional levels of statistical significance, and the largest value was now for a right-sided pathway. This is pretty meaningless, however, because the sample size, already small, becomes so tiny that one cannot do an adequately powered test of the association. The best one can say is that ‘further data are needed’.



I hope the authors will look further at this issue, as the role of the arcuate fasciculus in language  learning is fascinating and potentially important. One possibility would be to look at the associations between vocabulary level and analogous connectivity measures in the sample of 50 adults reported by Catani et al (2007), where the same DTI methods were used.



After I had drafted this critique, I Googled to see if anyone else had blogged about this study. I didn’t find blogs, but I did find extensive media coverage. I was astonished to see that, in discussing implications of this study, one of the authors, Marco Catani, a respected expert in tractography, appeared to be channeling Susan Greenfield. He was quoted as claiming that children’s vocabularies will be restricted by their use of iPads. The newspapers have picked up on these quotes, coming out with headlines such as: “Experts say too much time is spent learning via tablets and computers. Children's vocabulary could be stunted because they listen to teachers and parents less.”  For further sensationalist and misleading accounts, see here and here.



Just to be clear, this was a study looking at structural and functional brain connectivity in relation to a task that involved extracting syllabic patterns from auditory input. It did not feature children, vocabulary learning or iPads.



It really does a disservice to families of children with language learning problems to come out with scaremongering claims about modern technology on the basis of no hard evidence. And, for the record, auditory input is not the only way to learn new words: reading provides an  increasingly important route for vocabulary learning as children grow older.







Reference

López-Barroso D, Catani M, Ripollés P, Dell'acqua F, Rodríguez-Fornells A, & de Diego-Balaguer R (2013). Word learning is mediated by the left arcuate fasciculus. Proceedings of the National Academy of Sciences of the United States of America, 110 (32), 13168-73 PMID: 23884655

Genetic variation and neuroimaging: some ground rules for reporting research

Those who follow me on Twitter may have
noticed signs of tetchiness in my tweets over the past few weeks. In the course
of writing a review article, I’ve been reading papers linking genetic variants
to language-related brain structure and function. This has gone more slowly than I expected for
two reasons. First, the literature gets ever more complicated and technical:
both genetics and brain imaging involve huge amounts of data, and new methods
for crunching the numbers are developed all the time. If you really want to understand
a paper, rather than just assuming the Abstract is accurate, it can be a long,
hard slog, especially if, like me, you are neither a geneticist nor a
neuroimager. That’s understandable and perhaps unavoidable. The other reason,
though, is less acceptable. For all their complicated methods, many of the
papers in this area fail to tell the reader some important and quite basic
information. This is where the tetchiness comes in. Having burned my brains out
trying to understand what was done, I then realise that I have no idea about
something quite basic like the sample size. The initial assumption is that I’ve
missed it, and so I wade through the paper again, and the Supplementary Material, looking
for the key information. Only when I’m absolutely certain that it’s not there,
am I reduced to writing to the authors for the information. So
this is a plea – to authors, editors and reviewers. If a paper is concerned
with an association between a genetic variant and a phenotype (in my case the
interest is in neural phenotypes, but I suspect this applies more widely) then
could we please ensure that the following information is clearly reported in
the Methods or Results section

1. What genetic variant are we talking about?
You might think this is very simple, but it’s not: for instance, one of the
genes I’m interested in is CNTNAP2, which has been associated with a range of
neurodevelopmental disorders, especially those affecting language. The evidence
for a link between CNTNAP2 and developmental disorders comes from studies that
have examined variation in single-nucleotide polymorphisms or SNPs. These are
segments of DNA that are useful in revealing differences between people because
they are highly variable. DNA is composed of four bases, C, T, G, and A in
paired strands. So for instance, we might have a locus where some people have
two copies of C, some have two copies of T, and others have a C and a T. SNPs
are not  necessarily a functional part of
the gene itself – they may be in a non-coding region, or so close to a gene that
variation in the SNP co-occurs with variation in the gene. Many different SNPs
can index the same gene. So for CNTNAP2, Vernes et al (2008)tested 38 SNPs,
ten of which were linked to language problems. So we have to decide which SNP
to study – or whether to study all of them. And we have to decide how to do the
analysis. For instance, SNP rs2710102 can take the form CC, CT or TT. We could
look for a dose response effect (CC < CT < TT) or we could compare CC/CT with TT, or we could compare CC with CT/TT. Which of these we do may depend on whether prior research suggests the genetic effect is additive or dominant, but for brain imaging studies grouping can also be dictated by practical considerations: it’s usual to compare just two groups and to combine genotypes to give a reasonable sample size. If you’ve followed me so far, and you have some background in statistics, you will already be starting to see why this is potentially problematic. If the researcher can select from ten possible SNPs, and two possible analyses, the opportunities for finding spuriously ‘significant’ results are increased. If there are no directional predictions – i.e. we are just looking for a difference between two groups, but don’t have a clear idea of what type of difference will be associated with ‘risk’ – then the number of potentially ‘interesting’ results is doubled.

For CNTNAP2, I found two papers that had
looked at brain correlates of SNP rs2710102. Whalley et al (2011) found that adults
with the CC genotype had different patterns of brain activation from CT/TT
individuals. However, the other study, by Scott-van Zeeland et al (2010), treated
CC/CT as a risk genotype that was compared with TT. (This was not clear in the
paper, but the authors confirmed it was what they did).

 Four studies looked at another SNP -
rs7794745, on the basis that an increased risk of autism had been reported for
the T allele in males. Two of them (Tan et al, 2010; Whalley et al, 2010) compared TT vs TA/AA and two (Folia et al, 2011; Kos et al, 2012) compared
TT/TA with AA. In any case, the ground is rather cut from under the feet of
these researchers by a recent failure to replicate an association of this SNP
with autism (Anney et al, 2012).

2. Who are the participants? It’s not very
informative to just say you studied “healthy volunteers”. There are some types
of study where it doesn’t much matter how you recruited people. A study looking
at genetic correlates of cognitive ability isn’t one of them. Samples of
university students, for instance, are not representative of the general
population, and aren’t likely to include many people with significant language
problems.

3. How many people in the study had each type
of genetic variant? And if subgroup analyses are reported, how many people in
each subgroup had each type of genetic variant? I've found that papers in top-notch journals often fail to provide this basic
information.

Why is this important? For a start, likelihood
of showing significant activation of a brain region will be affected by sample
size. Suppose you have 24 people with genotype A and 8 with genotype B. You
find significant activation of brain region X in those with genotype A, but not
for those with genotype B. If you don’t do an explicit statistical comparison
of groups (you should - but many people don’t) you may be misled into concluding that brain
activation is defective in genotype B – when in fact you just have low power to
detect effects in that group because it is so small.

In addition, if you don’t report the N, then
it’s difficult to get an idea of the effect size and confidence interval for
any effect that is reported. The reasons why this is optimal are
well-articulated here. This issue has been much discussed in psychology, but seems not to have
permeated the field of genetics, where reliance on p-values seems the norm. In
neuroimaging it gets particularly complicated, because some form of correction
for ‘false discovery’ will be applied when multiple comparisons are conducted. It’s
often hard to work out quite how this was done, and you can end up staring at
a table that shows brain regions and p-values, with only a vague idea of how
big a difference there actually is between groups.

 Most of the SNPs that are being used in brain studies are ones that
were found to be associated with a behavioural phenotype in large-scale genomic
studies where the sample size would include hundreds if not thousands of
individuals, so small effects could be detected. Brain-based studies often use
sample sizes that are relatively small, but some of them find large, sometimes
very large, effects. So what does that mean? The optimistic interpretation is
that a brain-based phenotype is much closer to the gene effect, and so gives
clearer findings. This is essentially 
the argument used by those who talk of ‘endophenotypes’ or ‘biomarkers’.
There is, however, an alternative, and much more pessimistic view, which is
that studies linking genotypes with brain measures are prone to generate false
positive findings, because there are too many places in the analysis pipeline
where the researchers have opportunities to pick and choose the analysis that
brings out the effect of interest most clearly. Neuroskeptic has a nice blogpost illustrating this well-known problem in
the neuroimaging area; matters are only made worse by uncertainty re SNP classification
(point 1).

A source of concern here is the
unpublishability of null findings. Suppose you did a study where you looked at,
say, 40 SNPs and a range of measures of brain structure, covering the whole
brain. After doing appropriate corrections for multiple comparisons, nothing is
significant. The sad fact is that your study is unlikely to find a home in a
journal. But is this right? After all, we don’t want to clutter up the
literature with a load of negative results. The answer depends on your sample
size, among other things. In a small sample, a null result might well reflect
lack of statistical power to detect a small effect. This is precisely why
people should avoid doing small studies: if you find nothing, it’s
uninterpretable. What we need are studies that allow us to say with confidence
whether or not there is a significant gene effect.

4. How do the genetic/neuroimaging results relate to cognitive measures in your sample?  Your notion that ‘underactivation of brain area
X’ is an endophenotype that leads to poor language, for instance, doesn’t look
very plausible if people who have such underactivation have excellent language skills. Out
of five papers on CNTNAP2 that I reviewed, three made no mention of cognitive measures,
one gathered cognitive data but did not report how it related to genotype or
brain measures, and only one provided some relevant, though sketchy, data.

5. Report negative findings. The other kind of
email I’ve been writing to people is one that says – could you please clarify
whether your failure to report on the relationship between X and Y was because
you didn’t do that analysis, or whether you did the analysis but failed to find
anything. This is going to be an uphill battle, because editors and reviewers
often advise authors to remove analyses with nonsignificant findings. This is a
very bad idea as it distorts the literature.

And last of all....

A final plea is not so much to journal
editors as to press officers. Please be aware that studies of common SNPs aren't the same as studies of rare genetic mutations. The genetic variants in the
studies I looked at were all relatively common in the general population, and so
aren't going to be associated with major brain abnormalities. Sensationalised
press releases can only cause confusion:

This release on the Scott van-Zeeland (2010) study described neuroimaging
findings from  CNTNAP2 variants that are found in over 70% of the population. It claims that: 

• “A gene variant tied to autism rewires the
  brain"

• "Now we can begin to unravel the mystery
  of how genes rearrange the brain's circuitry, not only in autism but in many
  related neurological disorders."

• “Regardless of their diagnosis, the children
  carrying the risk variant showed a disjointed brain. The frontal lobe was
  over-connected to itself and poorly connected to the rest of the brain”

• "If we determine that the CNTNAP2
  variant is a consistent predictor of language difficulties, we could begin to
  design targeted therapies to help rebalance the brain and move it toward a path
  of more normal development."

Only at the end of the press release, are we
told that "One third of the population [sic: should be two thirds] carries this variant in its DNA.
It's important to remember that the gene variant alone doesn't cause autism, it
just increases risk." 

References

Anney, R., Klei, L.,
Pinto, D., Almeida, J., Bacchelli, E., Baird, G., . . . Devlin, B. .
Individual common variants exert weak effects on the risk for autism spectrum
disorders. Human Molecular Genetics, 21(21), 4781-4792. doi: 10.1093/hmg/dds301(2012)

V. Folia, C. Forkstam, M.
Ingvar, P. Hagoort, K. M. Petersson, Implicit artificial syntax processing:
Genes, preference, and bounded recursion. Biolinguistics 5,  (2011).

M. Kos et al., CNTNAP2
and language processing in healthy individuals as measured with ERPs. PLOS One
7,  (2012).

Scott-Van Zeeland, A., Abrahams, B., Alvarez-Retuerto, A., Sonnenblick, L., Rudie, J., Ghahremani, D., Mumford, J., Poldrack, R., Dapretto, M., Geschwind, D., & Bookheimer, S. (2010). Altered Functional Connectivity in Frontal Lobe Circuits Is Associated with Variation in the Autism Risk Gene CNTNAP2 Science Translational Medicine, 2 (56), 56-56 DOI: 10.1126/scitranslmed.3001344

G. C. Tan, T. F. Doke, J.
Ashburner, N. W. Wood, R. S. Frackowiak, Normal variation in fronto-occipital
circuitry and cerebellar structure with an autism-associated polymorphism of
CNTNAP2. Neuroimage 53, 1030 (2010).

Vernes, S. C., Newbury,
D. F., Abrahams, B., Winchester, L., Nicod, J., Groszer, M., . . . Fisher, S.  A functional genetic link between distinct developmental language
disorders. New England Journal of Medicine, 359, 2337-2345. (2008).

H. C. Whalley et al.,
Genetic variation in CNTNAP2 alters brain function during linguistic processing
in healthy individuals. Am. J. Med. Genet. B 156B, 941 (2011).