there was a young woman from boobville ...

I think a lot of it is the snowball effect. He didn't call immediately, presumably because he was indisposed. When he did get the message, it was too late to call about it without looking like a dick. So he told himself he'd do it tomorrow when he worked up the nerve for the fight that was sure to occur, and just kept telling himself that he'd talk to Dean about it tomorrow until Dean was standing RIGHT THERE giving him shit about it. Yeah, it's over something serious, but that probably just made the phone call that much harder to make.