Nearly all the time, it feels weird without one? I like the less structural ones and more comfy casual ones when just at home doing nothing, like the pull over ones with padded inserts (I really wish I understood the science of how the washing machine puts both pads on one side).