An imaging system that simultaneously performs near infrared (NIR) tomography and magnetic resonance imaging (MRI) is used to study breast tissue phantoms and a healthy woman in vivo. An NIR image reconstruction that exploits the combined data set is presented that implements the MR structure as a soft-constraint in the NIR property estimation. The algorithm incorporates the MR spatially segmented regions into a regularization matrix that links locations with similar MR properties, and applies a Laplacian-type filter to minimize variation within each region. When prior knowledge of the structure of phantoms is used to guide NIR property estimation, root mean square (rms) image error decreases from 26 to 58%. For a representative in vivo case, images of hemoglobin concentration, oxygen saturation, water fraction, scattering power, and scattering amplitude are derived and the properties of adipose and fibroglandular breast tissue types, identified from MRI, are quantified. Fibroglandular tissue is observed to have more than four times as much water content as adipose tissue, almost twice as much blood volume, and slightly reduced oxygen saturation. This approach is expected to improve recovery of abnormalities within the breast, as the inclusion of structural information increases the accuracy of recovery of embedded heterogeneities, at least in phantom studies.