Interesting. SWIM felt that same thing once when taking 3 Datura stramonium seeds with 75 mg of mescaline. After about 30 minutes into it, there was a slight feeling of fear just like you described with a sense that things were going to get really weird, but it passed after a few minutes.