From: https://incompetech.com/gallimaufry/care_less.html
When one usually states "I could care less", they usually mean "I could not care less". [e.g. "I could care less about linguistics."]
In order for one to "care less" about a subject, they must first care about it somewhat. Saying "I could care less about ... " does indeed imply, nay dictate, that there is some degree of care.
I've put together a handy chart to help visualize...
The Caring Continuum. vertical chart shows amount of caring - 'zero' and 'couldn't care less' synonymous at the bottom
It can easily be seen that, in absolute terms, there is no caring at the zero marker. It is impossible to care less than that amount.
Let us use a different example...
I could have less food.
I couldn't have less food.
Which statement above means "I have at least some food", and which means "I don't have any food"?