"then he receives a credit on his California return for the tax paid to New York."

Thats how it usually works, the resident state gives credit for taxes paid to the non-resident state. (there are a few reverse credit state relationships but CA/NY isnt one of them)