Torrefaction is regarded as a promising way to improve the fuel properties of biomass. In this work, a typical agricultural biomass of cotton stalk with high supply availability was employed to reveal the correlation between torrefaction conditions and fuel quality as well as pelletizing property. Cotton stalk was torrefied at 220–300 °C with a wide oxygen concentration of 0%–21% using a fixed bed reactor. The fuel qualities of torrefied samples were analyzed and the pelletizing properties were investigated using a universal material testing machine. The results showed that both non-oxidative and oxidative torrefaction significantly improved the heating value at a maximum of 20.48%, while extreme conditions of 300 °C with 10%–21% concentration were avoided due to the excessive consumption of combustible substances. Four key pelletizing parameters, including pellet density, compressive strength, durability and hydrophobicity, were improved, while the energy consumption increased, mainly attributed to the reduction of hydrophilic functional groups and the increased friction force. Response surface methodology was introduced and it was indicated that the pelletizing properties were sensitive to the temperature, followed by oxygen. The operating conditions were optimized by central composite design and a torrefaction temperature of 260–270 °C with an oxygen concentration of 2%–3% were recommended to produce torrefied biomass pellet with good fuel and pelletizing properties.