Content recommendation can be tailored by not only personal interests, but also the incumbent content, namely the content that a user is currently viewing. Incumbent-aware recommendation adds a new dimension to optimizing content caching. We study this optimization problem subject to user satisfaction constraints. We prove the problem's NP-hardness, and present an integer linear programming formulation that enables global optimality for small-scale instances. On the algorithmic side, we first present a polynomial-time algorithm that delivers the global optimum of the recommendation sub-problem, by leveraging the problem's inherent graph structure. Next, we propose a fast, alternating algorithm for the overall problem. Numerical results using synthesized and real-world data show the close-to-optimal performance of the proposed algorithm.