You do not need the tax office address.
You only need to write the tax reference number (also known as tax office reference).
This can be found on your wage slip.
If by chance your wages slips do not have this, then check you last P60, it is clearly mentioned on it.
If you have received any coding notice from HMRC, then this reference number is mentioned on that notice too.
regards