In this paper, we study the time variation of the risk premia in U.S. Treasuries bonds. We propose a novel approach for deriving a single state factor consistent with a dynamic term-structure with unspanned risks theoretically motivated model. Using deep neural networks to uncover relationships in the full set of information from the yield curve, we derive a single state variable factor that provides a better approximation to the spanned space of all the information from the term-structure. We also introduce a way to obtain unspanned risks from the yield curve that is used to complete our state space. We show that this parsimonious number of state variables have predictive power for excess returns of bonds over 1-month holding period. Additionally, we provide an intuitive interpretation of derived factors and show what information from macroeconomic variables and sentiment-based measures they can capture.