There is a difference between those who can't work and those who don't. When people talk about eliminating welfare, they are usually referring to programs whose only qualification is that you don't earn money. I, personally, think that disabled and vets are sustained at a higher level than people who "can't find work."
Almost no one advocates eliminating social security. The problem is that it's current structure is not sustainable, at least not indefinitely.
A major problem with the "welfare" discussion, is that people use "welfare" as a general umbrella term. Some programs specifically help those who can't (or at least shouldn't) work. Some are specifically paid into, like unemployment/disabillity insurance or social security (but often more is paid out than was paid into the program), and some simply require that you earn under a certain amount. Each type if program has issues that should be discussed separately and not under the umbrella term of "welfare".