Personally I would think if the garden stuff is crappy and the stuff your going to replace it with is better that the tree would thank you. .
Thanks for your answer. Yes, I am talking about a good, proper bonsai mix. But the doubt is over the slip potting; I didn't fully repoted it, this is the wrong season here. I just took the plant to a bigger pot, no root trimming, no root combing or cleaning. So the original garden soil is still there, but the remaining space in the pot has now bonsai soil.
Will the roots adapt?