Normally when performing a steam distillation on plant material you would only get essential oils, and these are called "oils" for the reason of not being an Alkaloid, having no base and therefore not existing as an Ion inside of the material.
Common stuff that can be extracted are Terpenes and these are all either just Hydrocarbons or maybe carry an alcohol or ketone.
Altogether they dont have an ionic form in physiological environment and so they can migrate over when a plant soup is heated at boiling temp.
Harmalas get evaporated with the steam, but in trace amounts. (Based on my own observation while using soxhlet extractor on rue seeds and UV light)
But regarding alkaloids, they should not be vaporizing at all, so it wonders me how you saw it with UV light - did you place a UV source behind the vapor and it was slightly glowing? Normally I would just except them to do so in solution. But still then this may be indeed a hint that there would be some in the vapor phase, but normally I would strongly say alkaloids wont ever migrate to the gas phase.
But more precise on your question you are totally good to go to just reduce you soup to whatever amount. You will not loose any amount of the Harmalas - or at least not noticable. Cooking the seeds 2x for 2 h is totally no problem.
But if you want to be super safe, just boil them at 80 - 90 °C.
But regarding concentration:
Why do you even want to concentrate them quite much? Is it for a brew or do you just want to perform a regular Harmala extraction?
In the later case just basify to 11+ and then decant as much water as possible, then filling up with new water and let the Harmalas settle down. Repeat until you have practically clear water above the light tan alkaloids and also a pH of around 7-8. That would be a very easy way to get good Harmalas, the back-salting step to precipitate them as *HCl or *OAc is not even needed and just increases time + decreases yield.
