If your toshiro is doing well I'd hold on to him, especially if you're managing to make progress modding him.
I'm going to hold onto toshiro until Noz Kenpachi. I'm also going to keep leveling Yachiru along with him for a few extra wins in Hueco Mundo Assault, as I noticed my vanguard always dies well before anyone else does. I might even pick up someone like Starrk mostly for an extra vanguard for HMA