Just quit.
Your story reminds of another in which my friend, who manages a restaurant, gave less and less hours to one of his employees till one day he just stopped giving him hours. He was trying to let him down easy instead of firing him, but eventually he had to tell him off.
Just quit and keep some dignity.