How to calculate the adjust basis of the property given up? Is mortgage paid off counted to reduce the gain realized (not recognized since it is 1031 exchange)?

Cost + Improvements less Accumulated Depreciation = Adjusted Basis.

Mortgage payoff does not affect realized gain.

1031 is mandatory if you meet all the requirements.