
Merely staying late does not immediately qualify you as a good project manager. A truly exceptional one will manager a project such that you normally do not have to stay later. However there are often circumstances beyond the best intentioned manager's control. So at the minimum you should be willing to stick around or pull the all nighter if your staff is doing the same. Otherwise the staff will be asking themselves why they should be making the sacrifice when you are not willing to do the same.
This principle is a specific case of the general one to lead your troops by example. You can make all the speeches in the world. However if your actions betray your words, even simpletons will see through your hypocrisy. Step up to the plate. Roll up your sleeves. Stay in the trenches with your developers. They will take notice.