Is it time for us to break up?

I care about my boyfriend deeply, and intensly, and unconditionally. I love him more than I ever thought I could love another human being. I feel like it would hurt tremendously to break up, but at the same time, he has hurt me too much for me to ever be truly happy in our relationship. He always says and does uncaring things and doesn't understand my pain so I continue to feel neglected emotionally. Lately, my eyes have been wandering... desperately trying to unravel the mess that he has created in my mind. I want someone who is able to be a gentleman and treat me the way I want to be treated, and try as he might, he just can't. He is just bad at communicating, and I find so much importance in conversation and expressing oneself. I know he has tried to make me happy, and when we are alone, laughing and joking and flirting, we are happy and smiling and I feel loved. But when it comes to serious things, we are not on the same wavelength at all, and it causes rifts in our connection. We've been together for 4 years, he lives with me at my mother's house, doesn't make much money, and is pretty much dependent on me. He has never had anyone so close to him in my life as I have been (and neither have I, but I have a family who will support me when I fall). He is truly alone without me, and I don't want to abandon him at all.... But I feel abandoned by him right now, emotionally, in my heart... and it hurts all the time. I don't know what to do. Do all relationships require pain? Am I overthinking things? Our love is strong, but our relationship is not. We both have high morals and are faithful, we don't ever want to hurt eachother, I want him in my life, and I love him. But there is a big piece missing in our relationship, and after 4 years, the holes in my heart that have been waiting to be filled, remain empty. What do I do? Endure the suffering, because it is a part of life? Or learn to live without the only love I've ever known and move on? And if so, how do I explain? How do I watch him pack his belongings and move out? How do I live without seeing him everyday? I am so attatched, yet I am hurting.