The frontal lobe develops during puberty, and synaptic pruning is completed by the age of 16.
The rest is just refinement/training from external stimuli, which has nothing to do with growth.
Theoretically, you can put a 16-year-old in a lab where he has to face a bunch of situations where he will need to navigate his emotions, and he can peak in one year instead of having to wait until 25.
And also, like I said, same logic of gh & hormones applies here
you don't need calorie surplus for synaptic pruning; brain is energy efficient