I think it also has to do with the fact that lsd and psilocybin have been very known and used since the 60s.. DMT, on the other hand, was limited to the few intrepid adventurers that travelled to the amazon to get to know ayahuasca, and the very very limited dmt synthesis and extractions.
Only in the last few years the dmt teks have improved and gotten spread, as well as mimosa hostilis becoming a known good source of dmt and being sold all around.
So there is this part, of being relatively new in terms of popularization.
Then there is what benzyme and others said, that DMT seems to impose a big respect on people. Mushrooms and LSD could too, but it seems they more easily lend themselves to recreational use (specially with people ´fiddling´ with small doses)
DMT is rarely sold, usually people extract it on their own and only give it away for those around them. So this decreases the amount of people actually getting contact with it. Normally those extracting will have researched a lot before, so they will have more respect to it. So without the financial incentive, plus the respect involved, this means it will rarely be passed on as the ´super best high´ or something like that..
and ayahuasca, the natural oral version, is basically impossible (imo) to be used recreationally. There´s a lot of suffering involved, with all the vomiting and purging in general. and you tend to see your mistakes, face the hard truth. Its not something the common drug user wants.