Observe his body language. For example, if he intently looks into your eyes and smiles a lot while talking to you, he is likely flirting. If he appears to be checking out your physical appearance he may be flirting. A nervous demeanor can be an indicator that he's concerned about saying or doing something inappropriate.
Determine whether or not his comments are intended to make you laugh. Humor is often a part of flirting. If the humor is of a sexual or intimate nature, the odds are even greater that he's flirting.
Gauge comparisons to his wife. Comments like, "I like short skirts. My wife wears mostly long skirts, but I enjoy the beauty of female legs. That skirt really shows off your legs." This kind of comment is flirty because he is giving you a compliment, while also making a slightly, negative comparison to his wife.
Notice if he begins to share with you personal information. For example, if he is working out more and makes it a point to bring it up in conversation with you--but not necessarily anyone else--he is likely fishing for a compliment and some extra attention. He may even make comments like "I've been working out more lately, in the event that I find myself on the market again, I want to be able to attract women as beautiful as you." With this type of statement, he lets you know he notices you, pays you a compliment and touts his physical attributes.