In this paper we present how charge transfer processes influences the induced optical losses (photodarkening) in ytterbium doped fiber lasers. The location of the charge transfer absorption band is strongly composition dependent and is correlated to the valence stability of the ytterbium ion in the silicate glass matrix. An improved photodarkening performance can in general be observed for a charge-transfer band shifted to shorter wavelengths, although other routes are also possible to reduce photodarkening. Other parameters that affect the laser performance, such as absorption and emission cross section, must also be considered.