There's a lot of historical accounts from non-christian people of characters from the New Testament
One even accused Jesus of being a magician, indicating that he did preform miracles
There's the Dead Dea scrolls, which contained almost the entire Old Testament, indicating that the Old Testament we read is not altered in any way
Archeoligists had found Jericho, which did have it's walls collapse in the past, which were from the time of the Old Testament, they fell in a way almost like it made a ramp for people to get up on