http://www.scholastic.com/teachers/article/origins-and-functions-political-parties
You'll want to read the section titled "Early U.S. Parties".
I was mistaken. I remembered that the first parties were not called Democrat nor Republican. I remembered incorrectly the origins of our current main parties. The Democratic-Republican (that would become the Democrats) party was the second party formed (the Federalists were the first). The Republican party was a fourth party, not a third (the others being Northern Democrat, Southern Democrat, and the Constitutional-Union Party).