When you're dead and I'm in jail, not really, no.
I mean does context ever matter? Sure. But where does your line of thinking end? Possible context is nearly infinite.
Maybe you are 80 years old and had poor eyesite & poor hearing that enabled me to sneak up un-noticed, maybe you are relatively thin so that made the shove easier, maybe it was rainy and harder for you to regain footing, maybe the cliff was 300 feet deep, maybe it was a Wednesday which is when you often go near said cliff, maybe the soil was loamy etc..
None of those details are required. Sure my lawyer and the prosecution would argue over them but equating a general conversation with a trial is ridiculous. Failing to mention however much context you deem essential does not make anyone dishonest for blaming me for your death.
Your family yelling at me in court: "YOU KILLED HIM!"
Me: "well.... to be fair, it was a Wednesday and the soil was loamy, so I'm actually not that bad."
:P