Possibly the most common atheist cavil is that atheism is a default position. That the reason someone is an atheist is that they haven't been presented a ground-up case for belief in God founded on evidence, and therefore they are logically committed to atheism until such a case is plausibly presented. In framing the issue in such a manner the atheist attempts to take the higher ground intellectually, but they are actually just playing word games.
If God created the universe then one can correctly say that everything that has ever existed or occurred is evidence--in the proper, modern, scientific sense--of God's existence. If He didn't, then everything is not evidence. Therefore saying that 'there is no evidence for the existence of God' is identically equivalent to saying 'I do not believe in God'. In other words, it is a priori disbelief dressed up pretty. This puts the propositions 'God does exist' and 'God does not exist' at worst on equal footing as mere assertions, and when the evidence is examined more closely--using our philosophical, historical, and scientific analytical tools-- firmly on the side of belief. Of course the atheist doesn't agree with the latter part, and he is entitled to that position, but he has no rights to claim that his disbelief is not actually disbelief but merely an objective surveillance of the evidence and that 'none exists in favor of God', as that merely re-words his disbelief. For if God exists then all is evidence.
One wonders what would even possibly could be presented to the 'rationalist' atheist of this sort that would constitute evidence in the sense that they are looking for. Perhaps if, inspected under some impossible microscope, an electron could be examined and we find that every electron has 'made by Yahweh' stamped on it that might constitute natural evidence for God. Anything short of that probably doesn't fit the criteria for 'evidence', in their mind.
But if God does exist and did create all things, is there any reason we should expect to find any evidence other than what we do find? I can't see any reason we should. If God created the natural world, and it is itself evidence, then there's nothing we should expect to see under our microscopes or in our telescopes other than precisely what we do in fact see. If God acted through miracles supernaturally within history, as Christians assert that he has, would we expect to be able to find any residue of the supernatural left behind in the natural order? No, at least not in any way that is scientifically observable (though in the case of Christianity we would certainly expect to see significant historical'residue', which we do in fact see). So again this lack of 'evidence' in the way that the skeptics mean doesn't even argue for disbelief; it's what we would expect to see whether theism is true or untrue. Not to mention there is much historical evidence that exists and which is not easily dismissed in favor of disbelief, indeed which also argues powerfully for belief, at least in the Christian God specifically.
From our human perspective all contingency and causality only points to a previous event, itself contingent and causal in nature, and so on. Aquinas correctly saw that a series of such events can never account for itself, and therefore the fact that there is such a series of events can be seen as powerful evidence for the existence of God. The atheist wants to follow the chain of contingency back to God as if God himself were also a contingent being, which is not what believers contend.
The fascinating thing about modern cosmology and physics is we may have followed contingency back to its limit, but there we find a brick wall. Either the multiverse theories are correct and we have only more contingency, which itself must still be grounded on something, or--if the beginning of the universe is also the beginning of existence--we find where the contingent meets the absolute. Though we can't see beyond the boundary of the contingent as the atheist apparently demands. In either case, a string of contingency with no known beginning which cannot account for it's own existence, or the contingent having a found boundary both argue against a strictly materialist worldview.
All we have that can be examined empirically is the contingent. And since contingency begets contingency the fact that this is all that we have is mistaken for a lack of evidence. When the reality of the matter is that either this very fact itselfis evidence, or, at worst, whether or not something constitutes evidence is dependent upon whether or not you believe. A good child of the Enlightenment would scoff at this, but it's demonstrably the case, as more contemporary philosophy highlights. One might think that therefore I'm saying evidence and science have no place in a search for truth, which is not the case. I don't believe that we are left adrift in an inadjudicable sea of relativist postmodernism. Only that science and evidence are only useful when their presuppositions and boundaries are properly understood and accounted for. That is, not as the sole or absolute means of divining truth, nor of divining truths of all kinds, but only of certain kinds within certain limits. Those truths can in turn be used to either defend or argue against various beliefs, such as whether or not God exists. But the belief or disbelief comes first and the examination of it second. There is no 'default position'.