If you had an EXCESS contribution, that would be the case.  However, your contribution was not excess, it was merely NON-DEDUCTIBLE.  Not at all the same thing.  You can't do a corrective distribution just because it's non-deductible.

Leave it there and file the Form 8606 to start tracking basis.