You may like to check again that you have entered all the details correctly with city, state, and zip code.  Having a P.O. Box address should not cause the return to be rejected.  I tried entering a P.O. Box address but didn't get any diagnostic, not to say it won't cause a rejection in the end.

AFAIK, the IRS matches EIN but not street address.

What error message are you getting exactly and where is that showing up?