One need only read his letter to Hubbard Taylor written 1794 Nov 15 in reference to the outcome of the Whiskey Rebellion, to see what he thought of insurrection, which is what many so-called patriots are talking about.

https://founders.archives.gov/documents/Madison/01-15-02-0291