Optimization of source and detector (SD) arrangements in a diffuse optical tomography system is helpful for improving measurements’ sensitivity to localized changes in imaging domain and enhancing the capacity of noise resistance. We introduced a rigorous and computationally efficient methodology and adapt it into the diffuse optics field to realize the optimizations of SD arrangements. Our method is based on Cramer–Rao lower bound analysis, which combines the diffusion-forward model and a noise model together. This method can be used to investigate the performance of the SD arrangements through quantitative estimations of lower bounds of the standard variances of the reconstructed perturbation depths and values. More importantly, it provides direct estimations of parameters without solving the inverse problem. Simulations are conducted in the reflection geometry to validate the effectiveness of the method on selections of the optimized SD sets, with a fixed number of sources and detectors, from an SD group on a planar probe surface. The impacts of different noise levels and target perturbation depths are considered in the simulations. It is demonstrated that the SD sets selected by this method afford better reconstructed images. This methodology can be adapted to other probe surfaces and other imaging geometries.