"Then why don't you dump her if she's mistreating you that badly?"

Oh, what's that? Because she's not! The eye-rolling is ever-present in this exchange, because he can't figure out if James is playing dumb or if he just is. Haymitch's money is on both. Come on!