I think this might be false.

I've heard this for years.  Apparently there is sometimes asbestos in the same mines that they get the mica they make verm out of.  Mines will occasionally get shut down if asbestos is found with the mica. So when there is a story about a mine getting closed down people panic.  There were also some health concerns a while back, but I believe all of those were put to rest.

I did a google search and couldn't find anything.

If you are really worried about it, go and pick up 2-5 20lb bags from a different greenhouse supply store.  That should last you for a long, long, LONG time.

Perilite does not work any where near as well as verm for PF cakes.  There are alternatives, but verm is really the best thing found so far.