Sub-Millisecond Absolute Timing: Toward an actual Gravitational Observatory