To register a company in NZ, you first have to reserve the name, and then register it.

Reserve a Name with the Companies Office (http://www.companies.govt.nz)
The initial cost for a name reservation is $10.00 online ($25.00 by post).
Complete instructions of the simple stepcan be found here.

In essense, you first use their search tools to see if the name has been taken, or is already in the pipeline (being reserved) and if not, you can proceed with reserving the name for yourself.

The Companies Office will notify you when a reserved name has been accepted, before you register the company.
A registered name must not be identical, or near identical to another reserved or registered name, and can be held for 20 days.

Once you have a reserved name, you can proceed to registering it.

Registering a Name with the Companies Office (http://www.companies.govt.nz) 
Registering a company is $150 online ($250 by post).
An annual fee is 0.00 online, but $30.00 by post.

Complete instructions can be found here -- where you will find a nice video that demonstrates every single step. Easy!

Note: their notes state that to do it online you will first need a scanner...(about $50.00 for a cheapo downtown). I'm not sure why they say that, as I don't see any requirement for it unless they are talking about doing the equivalent of sending back the faxs of the signed director/shareholder consent forms:

Final notes about applying online: 
Once you've filled in all the parts, and your credit card has been debited, note that there is still one thing to do before the company is probably registered: Director and ShareHolder consent forms will be emailed to you. These need to be signed and faxed back to them to complete the incorporation process.

All entities conducting a taxable activity with an annual turnover of $40,000 or more must register for GST (the Goods and Services Tax), by filling in the GST Registration Form IR360.

Note:
If you are not planning on making $40,000 the first year, or simply don't know what kind of sales you will have, don't cross that bridge until you get there. Why? Because once you've signed up, you will have to submit the paperwork at regular intervals, whether you've made money or not, until you close the company [as far as I know].

How to apply for the GST
You can register online at http://ird.govt.nz/online-services.
The GST Registration Number will generally be the same as the IRD number used for tax purposes.

GST is calculated using your company's accounting mehod
Once an entity has registed for GST, it must account for GST regulary, based on the company's accounting methods, which will be either on a payment basis (ie cash-basis accounting), an  invoice basis (ie accrual-basis accounting), or a hybrid basis. (Note that only accrual basis accounting is GAAP compliant).

Report on a Regular Basis
As for how often it has to be reported, you can choose between 1, 2, and 6 months -- within constraints that are determined by how much money you are making: 6 months is limited to companies making less than $250,000, 2 month is limited to companies making less than $24 million.

Two months seems to be a common period.

Reclaiming
If you register for GST, you can reclaim any GST you are charged by other businesses. 
GST is charged at a rate of 12.5%.

You havn't even started and you already have to discuss taxes. Yup. That's the way it is.

Limited Company
Limited Companies pay a company tax of 33% of profits.

ACC
The amount of accident insurance your business pays to the government depends on whether it has any employees. 
If it does, you pay insurance at the "employer rate" of 90c per $100 of payroll. 
Otherwise, you pay the self-employed rate of $1.79 per $100 of liable earnings. 

Other Expenses: KiwiSaver
Again, if you have employees, you'll be paying 1% (rising to 4% over time) of their salary to their KiwiSaver account.

Making Payments to the IRD and/or KiwiSaver
They've got forms to pay online -- or you can pay via post, westpac, etc....its all described in one of the links below.

What is a Merchant Account?
To accept credit cards, you must establish a Merchant Account (MA), a special bank account for handling the revenue (and fees) from credit card transactions.

Why do I need a Merchant Account?
Traditionally, without a Merchant Account (MA) it is impossible to receive funds debited from a credit card. 
Note that this is no longer written in stone: for example, Google Checkouts (and others) are willing to accept credit cards for you, saving you the trouble and expense of opening a merchant account, but for a cut of the action.

Where do I get a Merchant Account?
You optain a Merchant Account (MA) from a Merchant Account Provider (MAP), which is either your bank (an aquiring bank), or an Independent Sales Organization (ISO) that processes online credit card transactions. 

What does the Merchant Account Provider do?
The Merchant Account Provider (MAP) does the job of verify your client's credit card, processes the transaction, and deposits the results into your account, usually with 2 to 4 days.

Using your bank as an Merchant Account Holder (MAP)
If your company has a track record of reliable sales, you may be able to get a merchant account from your bank.

Banks have been in the MAP business for years, processing the Card-Present sales (via card-swipe machines) for their local community's restaurants, shops, etc.  But not all banks are willing to provide merchant accounts for internet activity: make sure that they can issue a Merchant Account that is upgraded/configured to handle Card-Not-Present transactions.

That said, without a sales history (which is the case for startups), your business will probably need to work with a reseller, usually an Independent Sales Organization (ISO).

More information about Merchant Accounts in general can be found here.

I've waited most of my adult life for the promise of a paperless world to become a reality...but we're not there yet.

So, in the meantime, you'll still need a good printer to print up stationary, envelopes, and most importantly, business cards, in order to get yourself seen by your local community.
In other words, you'll need a printing company at some point or other.

If you are in Wellington, I've tried fullstop, and liked their service very much. (I've already talked about them and listed an example of their pricing, and their contact information on the Living in Wellington page).