We propose a topology optimization method that includes high-cycle fatigue as a constraint. The fatigue model is based on a continuous-time approach where the evolution of damage in each point of the design domain is governed by a system of ordinary differential equations, which employs the concept of a moving endurance surface being a function of the stress and back stress. Development of fatigue damage only occurs when the stress state lies outside the endurance surface. The fatigue damage is integrated for a general loading history that may include non-proportional loading. Thus, the model avoids the use of a cycle-counting algorithm. For the global high-cycle fatigue constraint, an aggregation function is implemented, which approximates the maximum damage. We employ gradient-based optimization, and the fatigue sensitivities are determined using adjoint sensitivity analysis. With the continuous-time fatigue model, the damage is load history dependent and thus the adjoint variables are obtained by solving a terminal value problem. The capabilities of the presented approach are tested on several numerical examples with both proportional and non-proportional loads. The optimization problems are to minimize mass subject to a high-cycle fatigue constraint and to maximize the structural stiffness subject to a high-cycle fatigue constraint and a limited mass.