Non-singular exponential gravity: a simple theory for early- and late-time accelerated expansion