Have you tried sitting down with him and talk things over? Let him know how you feel, why you are upset and what's your expectations of him. If both of you feel that it is worthwhile to continue, set a deadline.
If things still don't work out by then, it is better to end early. By work out, I...