Quantitative investigations of fiber orientation and structural connectivity at microscopic resolution have led to great challenges for current neuroimaging techniques. Here, we present a structure tensor (ST) analysis of ex vivo rat brain images acquired by a multicontrast (MC) serial optical coherence scanner. The ST considers the gradients of images in local neighbors to generate a matrix whose eigen-decomposition can estimate the local features such as the edges, anisotropy, and orientation of tissue constituents. This computational analysis is applied on the conventional- and polarization-based contrasts of optical coherence tomography. The three-dimensional (3-D) fiber orientation maps are computed from the image stacks of sequential scans both at mesoresolution for a global view and at high-resolution for the details. The computational orientation maps demonstrate a good agreement with the optic axis orientation contrast which measures the in-plane fiber orientation. Moreover, tractography is implemented using the directional information extracted from the 3-D ST. The study provides a unique opportunity to leverage MC high-resolution information to map structural connectivity of the whole brain.