If it isn't injuring you in any way I don't see why it would be a bad thing to do. If you are in a position where you want to play it cool though, it would give away that you are nervous.
When I am stressed or worried I clean furiously. I busy myself in some way. I pace the floor and all kinds of other "busy" things.