Coherent oscillations in small Fermi-polaron systems