Time is money, they say. However, I will argue that time is more valuable than money because it is limited to an unknown amount. Mysterious. But seriously, we don't know how much time we have on this earth and we ought to spend our time wisely. Also, there are emotions involved in time. This might sound complex or like a third dimension but it is simply that you can feel good or bad, happy or sad when and/or after spending time doing something.
Spend your time with people who love you. Genuinely love you. People who wish the best for you and want to see you smile. People who think of you constantly and want you to be happy. People who understand you or try. Love isn't smothering another person or spending 24/7 with them, it's realizing that people need space and it's loving and respecting another person enough to give them that space. But being close enough or available for you. People who truly love you don't guilt trip you into hanging out with them because your love is a mutual one, they understand that you have things going on in your life and they do, too. They also understand that you will reach out to them when you are able because you enjoy spending time with them.
It might sound selfish but it's not. It's self-preservation. And just like your time, your self is limited and must be nurtured.