Yes, we are inferring what people mean in the Bible when they say this. Your argument doesn't really have much of a base.
What happened between us is that I made an expression and you then misinterpreted it. If you didn't misinterpret it, I wouldn't have a problem with you interpreting my expression because you'd be interpreting correctly. I have a problem because I'm saying this, and you're telling me, "no, you're actually saying this..."
This kind of thing can happen in the Bible. James, for example, says, "faith without works is dead", to which people are saying, "well, he's saying that we are saved by works." He never said that!
Here's the comparison between your accusation and my defense:
My defence: You're telling ME what I mean,
Your attack: I'm also telling HIM what that other guy means
The fallacious thinking that you've found is not really fallicious.