We consider a canonical nonlinear-constrained non-convex problem with broad applications in machine learning, theoretical computer science and signal processing. We propose a simple primal-dual splitting scheme that provably converges to a stationary point of the non-convex problem. We achieve this desideratum via an inexact augmented Lagrangian method. We also propose a novel constraint qualification for the given problem, which we name non-convex Slater's condition. The new algorithm features a convergence rate of $\mathcal{O}(1/\epsilon^2)$ in the feasibility gap and a convergence rate of $\mathcal{O}(1/\epsilon^4)$ in the gradient mapping. This relatively slow rates are counteracted by methods' cheap per-iteration complexity. We also provide numerical evidence on large-scale machine learning problems, typically modeled via semidefinite relaxations.