This is very well thought and argued, and I do find it convincing overall, when applied to individuals.
The problem comes with companies. Currently in most countries there is a legal fiction that companies are in some ways individuals. I'm for individuals being fully allowed to possess, trade, and use, without restrictions, for the reasons you stated. But I don't see why companies should have those rights. They are super-humanly powerful entities, and pretending that individuals can relate to them as peers is just fiction.
If you allow unrestricted advertising and sale of psychoactive substances by companies, you get a situation similar to the situation with tobacco for most of the 20th century. Manipulation, propaganda, social conditioning to make money regardless of the effects on people. Yes, each tobacco user had the ultimate responsibility for their choices, but the playing field wasn't leveled. It was each person as an individual vs. powerful entities shaping a whole culture to lead them in the direction of tobacco use since childhood.
So I think my neighbor Jim should be able to sell me any substance in a fully unrestricted way. Yes, he may be a liar and manipulator, and some cases of abuse are unavoidable. But he has no influence to shape culture, society, and regulations so I'm statistically likely to end up hooked to whatever he's selling.
In summary, I fully agree with you, as long as it doesn't apply to companies. They aren't individuals and are potentially very dangerous.