Optical biopsy techniques offer a minimally invasive, real-time alternative to traditional biopsy and pathology during tumor resection surgery. Diffuse reflectance spectroscopy (DRS) is a commonly used technique in optical biopsy. Optical property recovery from spatially resolved DRS data allows quantification of the scattering and absorption properties of tissue. Monte Carlo simulation methods were used to evaluate a unique fiber-optic probe design for a DRS instrument to be used specifically for optical biopsy of the brain. The probe diameter was kept to a minimum to allow usage in small surgical cavities at least 1 cm in diameter. Simulations showed that the close proximity of fibers to the edge of the probe resulted in boundary effects due to reflection of photons from the surrounding air–tissue interface. A new algorithm for rapid optical property recovery was developed that accounts for this reflection and therefore overcomes these effects. The parameters of the algorithm were adjusted for use over the wide range of optical properties encountered in brain tissue, and its precision was evaluated by subjecting it to random noise. This algorithm can be adapted to work with any probe geometry to allow optical property recovery in small surgical cavities.