There are many factors as placement and climate for example, but yes, with less organics it will dry faster, preventing fungus and such...
Just to put in context how your climate plays an important role on this:
I have all my trees outside, and my climate is Humid Subtropical, which is perfect for Ficus, even more being some of them "Ficus Organensis", that are native specifically from our region, that being said, I make their substrate with a lot of organic matter, and they love it... But that works for me, because of my climate, it's placement and so on, maybe it wouldn't do well on a more temperate zone or for a ficus being kept inside...