I don't feel bad about letting you go
I just feel sad about letting you know
Bragg has often been complimented for his unapologetic honesty and ability to play with words in his lyrics. These two lines offer a prime example of that.
Bragg claims that although he is ready to end the relationship, he has enough respect and admiration for his girlfriend to regret having to do so. Ironically, this proves Bragg is a selfless, mature adult capable of both committing himself to a relationship and, potentially, becoming a parent.