"They" never left, "they" are still here, "we" might be "them" or at least a part of "them". We would be ashamed if we would know everything from the times which are lost and if we would see how it was when "they" were living here "creating" for a long time what we consider as ours, we would be shocked by surprise.
If you start searching you will see more clearly everyday.
If you´re afraid of something then it´s possible that you could be afraid of everything.