A scalar Langevin-type process $X(t)$ that is driven by Ornstein-Uhlenbeck noise $\eta(t)$ is non-Markovian. However, the joint dynamics of $X$ and $\eta$ is described by a Markov process in two dimensions. But even though there exists a variety of techniques for the analysis of Markov processes, it is still a challenge to estimate the process parameters solely based on a given time series of $X$. Such a partially observed 2D-process could, e.g., be analyzed in a Bayesian framework using Markov chain Monte Carlo methods. Alternatively, an embedding strategy can be applied, where first the joint dynamic of $X$ and its temporal derivative $\dot X$ is analyzed. Subsequently the results can be used to determine the process parameters of $X$ and $\eta$. In this paper, we propose a more direct approach that is purely based on the moments of the increments of $X$, which can be estimated for different time-increments $\tau$ from a given time series. From a stochastic Taylor-expansion of $X$, analytic expressions for these moments can be derived, which can be used to estimate the process parameters by a regression strategy.