Marriage counseling only works if both parties want it to. If you are not both prepared to put serious effort forth towards reconciling, it won’t work. You can pitch the idea of counseling to him, but if he says no, there isn’t much more you can do. That he is already seeing someone else makes it likely that he has moved on, or is trying to do so, but it cannot hurt to ask him.
He’s gone, and counseling would be a waste. To take him back after he has moved out with another woman would irreparably damage your relationship, and put him in control for the rest of your life together.
Until and unless HE asks to come back, just move on. IF he asks, then it’s time to consider counseling, but to chase him is to lose him for good.