In all honesty, you should leave him and find someone who fits your definition of caring or loving. You need someone who can provide for you the way you want to be provided for be it 5 bucks or 50million dollars...
There is no right or wrong, he will have his own reasons why he wanted the five...