Right!  But the everlasting soul you are referring to is not a personal everlasting self.  That's the difference.  I completely agree with your take as does Buddhism.  And ultimately so does Vedanta, but for me at least I simply can't think in terms of reaching a higher self because every time I do it turns into an ego chase.  Why does the ego joedirt want to be enlightened?  So he can be different than others?  So he can teach others?  Perhaps so he could do magic tricks?  etc, etc, etc.

For me learning to really focus on the fact that my ego is the illusion has been pivotal in my spiritual growth.  

Speaking in terms of consciousness you can't be both a separate self and  unified with everything at the same time.  Buddha realized that most religious people of the day, and even the ascetics, were ensnared with their ego chasing the higher self and thus unable to actual achieve it because their aim was wrong. 

That's right. 

and in those terms Buddha would not disagree with you as long as you didn't try to assign any part of your personality to it. 

I highly recommend looking deep into Vedanta.  I think it's a natural fit for the path you seem to be on..  IMHO of course.

Peace