We demonstrate that the dispersion of the refractive index (RI) influences the retrieval accuracy of the particle size distribution (PSD) in the multi-wavelength angular scattering measurement. A numeric simulation was performed, which indicated that the dispersion results in a 10% to 50% error on scattering intensity measurement at certain scattering angles. In addition, although the dispersion showed little effect on the ill-posedness of the scattering kernel, the difference between the dispersion and no dispersion in the scattering kernel is so amplified in PSD inversion that accuracy decreases to such an extent that it becomes unacceptable. The accuracy of PSD retrieval, considering dispersion, was higher than that without dispersion. Finally, it was also proposed that variation of RI with incident wavelength must be considered in designing the scattering kernel to improve the retrieval accuracy in the multi-wavelength angular scattering measurement.