Andrea Byrne says she felt her husband would be "better off" without her during fertility treatment.