As people have mentioned, the Electric Kool-Aid Acid Test is an excellent book, following Kesey and the pranksters through the original Acid Test parties and all that.
Plants of the Gods by Shultes and Hofmann is a must-own.
The Way of the Shaman by Michael Harner is criminally underrepresented, and definitely worth reading. Provides not only information and context, but some practical shamanistic exercises. If you like it, he also has a follow-up, Hallucinogens and Shamanism
If you're interested in psychology, anything by Stanislov Grof will do you well.
It isn't strictly tied to psychedelics, but Alan Watt's The Book on the Taboo Against Knowing Who You Are really helped me integrate my early DMT experiences and serves as an excellent introduction for a westerner into eastern philosophy. If you have more interest in that direction, PM me-- I have no shortage of reading material in that field.
On the chemistry side, TiHKAL and PiHKAL are must-owns. DMT:The Spirit Molecule is worth reading as well, though I definitely can tell the people on this site aren't the intended audience. Psychedelic Medicine by Richard Miller gives an excellent overview of the state of modern entheogen-assisted therapy.
If you'd like to get more into practical chemistry, Zubrick's Organic Chemistry Lab Survival Guide is the best resource you can have.
I realize I've just dumped a lot out on you, so let me pull back: if you only read one book on here, it should be Alan Watt's The Book on the Taboo Against Knowing Who You Are.