The Centre of New Zealand is located on top of Botanical Hill and can be reach in a nice short 20-30 minutes uphill exercise from the Botanical Reserve. At the top you will be rewarded with a great view over the city and Tasman Bay. Next to the needle monument are some benches where you can enjoy the view and catch a breath. Several walkways are meeting at the summit and you can decide to take another route down to Botanical Reserve, take a detour and walk further to Walter's Bluff to come down opposite Founder's Park or take the route to Branford Park to come down in the Maitai Valley.
Despite its name, the 'Centre of New Zealand' is not actually the geographical centre of NZ. Back in the days in the 1870's when surveys throughout New Zealand where isolated and not connected, Nelson's chief surveyor John S. Browning was the only one with the skills to do it and he chose his starting point on Botanical Hill. However, the actual centre of New Zealand is not too far away and lies about 50km South West in the Spooner Range.
The Botanical Hill Reserve also has some interesting history from the same era. It's basically the birthplace of New Zealand's Rugby tradition. The first game was played here in May 1870 between the Nelson College and the Nelson club team.
A good starting point is the Botanic Sports Field at the bottom of Botanic Hill. It's located at the corner of Milton St and Hardy St.