Researchers say that children who perform differently in IQ tests show different patterns of brain development.
They believe this difference is associated with changes in the brain during adolescence and possibly has more to do with how the brain develops at that time than its overall size.
The whole question of IQ as a valid measure of intelligence always creates controversy and this latest research could re-ignite the debate about intelligence.
Researchers however now appear to agree that cognitive abilities are shaped by environmental factors as well as genetic ones.
Researchers led by Philip Shaw at the National Institute of Mental Health in Bethesda, Maryland, say it is not that brainy children have more grey matter, but that intelligence is in the trajectory of brain development.
Shaw's team tracked a group of more than 300 children as they aged from 6 to 19, running them through a series of cognitive tests, both verbal and non-verbal to establish their IQ.
The team also measured the size of brain structures using magnetic resonance imaging at roughly two-year intervals: more than half the children had at least two scans, and around a third were scanned three or more times.
When the researchers split the children into three groups according to their initial IQ scores, they noticed a characteristic pattern of changes in the brains of the group with the highest scores.
The thickness of the cortex, the outer layer of the brain that controls high-level functions such as memory, started off thinner than that of the other groups, but rapidly gained depth until it was thicker than normal during the early teens.
All three groups converged, with the children having cortexes of roughly equal thickness by age 19.
The strongest effect was seen in the prefrontal cortex, which controls planning and reasoning.
The brightest children showed the highest rate of change in the scans and the team of scientists believe the longer thickening time in the very brainy children could well be an indication of a longer development period for high-level cognitive circuits in the brain.
They suggest the thinning phase could involve a "use it or lose it" pruning, or killing off, of brain cells and their connections as the brain matures and becomes more efficient.
Shaw says it appears that this might be happening more efficiently in the most intelligent children and people with very agile minds might have a very agile cortex.
The study raises many questions that could prompt further research.
It is still unclear for example how genetic and environmental factors contribute to the change and the study is likely to prompt discussion of the social implications of such results.
The study is published in the journal Nature.