I fully agree with you. You need a very strong, healthy, and developed ego to get rid of it. You need to be a mature adult in order to transcend even that phase.
For most people, no-self is an escape from life and its challenges. For Sri Ramana, it is all about the Self, which is wisdom and love. He never advocated for anyone to leave their family or work. Even the notion that the ego is an illusion is a philosophical view meant to turn you in the right direction for your investigation; it is in no way an escape tactic. Most of his devotees had fulfilling householder lives, and spirituality was a common thread through everything. He advised holding the view only for inner work but living a normal life outwardly. Ideally, no one would even know that you follow him or practice anything.