We lay out the capabilities and limitations of starshade-based missions aiming to measure the reflected light spectra of temperate planets from an imaging perspective. We use the Starshade Imaging Simulation Toolkit for Exoplanet Reconnaissance to conduct high fidelity end-to-end optical simulations, taking a step forward from simplified analytical equations, exploring and quantifying the impact of an array of observational conditions, including natural parameters such as target star types, planet types, distances, planet phases, and exo-zodiacal dust, and starshade perturbations such as tilt, shift off line of sight, edge errors, and glint. We find that signal-to-noise ratio (SNR) requirements used for establishing detection and spectral characterization, is not suitable under realistic observation conditions for a wide range of targets. We show that even if we assume that the spatially distributed, time-varying background noise could be known and calibrated to a level of 1%, each target star will need its own SNR requirement based on its unique observation conditions, nearly always resulting in a higher threshold SNR, with values as high as X5 from currently established requirement, and in some cases impossible to detect. We conduct statistical analysis using end-to-end optical simulations, taking into account observationally based priors and update previously established completeness values for an array of target stars and mission configurations, accounting for starshade perturbations, and background knowledge at a level of one percent and find that completeness values are negatively impacted and reduced by up to 50% across targets even at ranges shorter than 10 pc. Finally, we utilize information from over hundreds of thousands of detailed imaging simulations to map accessible target stars for both optimistic and pessimistic scenarios, reassessing the expected capabilities of starshade-based high contrast direct imaging missions.