I guess I'm one of those people that looks at ethics solely with regard to consequences (or at least, I give consequences a great deal of weight).
In the scenario, it's true that you are intending him harm, but it could also be said that if you are the only free agent there who's aware of the situation and you don't push him (or quickly find another solution at least as viable), you are (at least indirectly) intending the harm of the passengers. In other words, you're counting their lives as less valuable than your ability to stay within your ethical comfort zone, which is extremely selfish.
I haven't thought about it at length though, and there may be variables I'm not considering.