St James' Restaurant, on the fourth floor is one of London's most famous spots for Afternoon and High Tea, with teas specially selected from teas from around the world, and even a regular pianist.
It's not cheap, however, with prices ranging from £32 to £38 per person. Still, it's ok, and they were doing fine in your author's eyes until he read that they request that 'both sexes lean more towards elegance' in dress. That doesn't really mean anything at all.
For more, see http://www.fortnumandmason.com/stjames-restaurant.aspx