IMO "fresh" mullein is harsh too. What you want to look at (basically with everything you want to smoke) is fermentation. Now after fermentation your mullein wont be green anymore but brown, so the chlorophyll is gone. But it wasnt the chlorophyll that caused your discomfort smoking it in the first place, mostly it is carbohydrates and proteins making your smoke harsh, just imagine smoking sugar and tofu, not appealing ? Guess why
. Fermentation can deal with both, carbohydrates and proteins, when they are broken down its ready to smoke.
Its always up to experimentation and can be a highly elaborate process, but in general fermentation is easy. Pack your product that is fresh tightly, this will ensure a desirable climate for microbiological activity in your product, also keeping it slightly warm can speed up the process. The organisms that will do the job should already be on your product naturally and should be fine if you arent looking for elaborate experimentation.
If your product is clearly rotten or moldy tho, do not use it, discard and try again using a bit less humidity. Sometimes a genuine way to get the process started if your material is hesitant is making a solution of sugar and water and giving it a light spray. Many containers, especially earthenware ones, can provide and sustain a desirable microclimate without affecting the unique aroma of whatever you want to ferment.
Hope thats a rough guidance. Personally: I ferment all plant material I intend to smoke, there is a reason why cigars are brown and not green