The building was a markedly different step from the other diplomatic missions of the day in that it was a modernist skyscraper, succeeding New Zealand's former, more traditional, base at 415 the Strand. This was also to be the first tower block to be built in Central London since the war, so it had to be a good one.
No expense was spared in its construction or in the design, and despite difficulties in planning permission, the 18 storey building was eventually begun after permission was granted directly by the British Cabinet. To this day, it is a rare example of such a skyscraper (albeit a small one) in an area of London where planning is strictly controlled.
Upstairs, the building has a number of outside spaces, including a spectacular terrace around the penthouse, with views over Buckingham Palace, the Houses of Parliament and Westminster Abbey, and two internal gardens planted with native New Zealand plants, though sadly these are not open to the public. If you want to get up close or inside the building, your only real options are the Sports Cafe (your author suggests not bothering), or the Royal Opera Arcade (which isn't actually part of it but runs down one side and is very pleasant).
For more information on New Zealand House, see the Twentieth Century Society website here: http://www.c20society.org.uk/docs/building/newzealand.html