Researchers say that children who perform differently in IQ tests show different patterns of brain development.
They believe this difference is associated with changes in the brain during adolescence and possibly has more to do with how the brain develops at that time than its overall size.
The whole question of IQ as a valid measure of intelligence always creates controversy and this latest research could re-ignite the debate about intelligence.
Researchers however now appear to agree that cognitive abilities are shaped by environmental factors as well as genetic ones.
Researchers led by Philip Shaw at the National Institute of Mental Health in Bethesda, Maryland, say it is not that brainy children have more grey matter, but that intelligence is in the trajectory of brain development.
Shaw's team tracked a group of more than 300 children as they aged from 6 to 19, running them through a series of cognitive tests, both verbal and non-verbal to establish their IQ.
The team also measured the size of brain structures using magnetic resonance imaging at roughly two-year intervals: more than half the children had at least two scans, and around a third were scanned three or more times.
When the researchers split the children into three groups according to their initial IQ scores, they noticed a characteristic pattern of changes in the brains of the group with the highest scores.
The thickness of the cortex, the outer layer of the brain that controls high-level functions such as memory, started off thinner than that of the other groups, but rapidly gained depth until it was thicker than normal during the early teens.
All three groups converged, with the children having cortexes of roughly equal thickness by age 19.
The strongest effect was seen in the prefrontal cortex, which controls planning and reasoning.