Abstract The development of the ventral visual stream is shaped both by an innate proto-organization and by experience. The fusiform face area (FFA), for example, has stronger connectivity to early visual regions representing the fovea and lower spatial frequencies. In adults, category-selective regions in the ventral stream (e.g. the FFA) also have distinct signatures of connectivity to widely distributed brain regions, which are thought to encode rich cross-modal, motoric, and affective associations (e.g., tool regions to the motor cortex). It is unclear whether this long-range connectivity is also innate, or if it develops with experience. We used MRI diffusion-weighted imaging with tractography to characterize the connectivity of face, place, and tool category-selective regions in neonates (N=445), 1-9 month old infants (N=11), and adults (N=14). Using a set of linear-discriminant classifiers, category-selective connectivity was found to be both innate and shaped by experience. Connectivity for faces was the most developed, with no evidence of significant change in the time period studied. Place and tool networks were present at birth but also demonstrated evidence of development with experience, with tool connectivity developing over a more protracted period (9 months). Taken together, the results support an extended proto-organizon to include long-range connectivity that could provide additional constraints on experience dependent development.