While there are physical forms within the human brain that map pretty well to mathematics, there is no direct evidence outside of our minds of the existence of mathematical forms.
Depends on what kind of metaphysics you submit to. For instance, Frege makes a good argument for the existence of numbers. When you really dive into the subject, I actually find it very hard to argue otherwise. (Also, "evidence" is kind of a higher level notion.)
As an example, you won't find anything that would perfectly give you the number pi in nature; however you constantly see things that revolve around it and statistically converge to it. Why pi, if it is only an idea that only exists in our minds?
The truth is, pi is the name we assign to a fact of nature. Its existence is more convincing to me than most other things I accept as existing. I have no reason to believe that the relations inherent to geometric shapes the universe consistently produces happened somehow after humanity's arrival.
Can you show me how nature uses cryptography outside of humanity? If it isn't used by nature apart from humans, how can if function at all without humans?
I find this question tangential. Does the nature make use of wheels for example? But it is still a fact that round objects rotating on straight surfaces make movement easier where there is a lot of friction, and the ideal round shape for this is the circle. Now, nature could use this fact (probably it does but I don't know enough biology to provide an example) or it might not. Does it change the fact?
You could argue that cryptography is more abstract, and I'd say that nature is making effective use of it through Bitcoin, and the debate won't go anywhere until I spend a few million years travelling through the galaxy to bring you the perfect example outside humanity. Either way, is it more relevant than the mathematical tricks proteins use? If you put a prime number of balls in a basket and send it to space, will aliens be able to divide them equally among themselves without breaking one?