Small businesses today have an advantage with the plethora of technology that is available and so many cost saving alternatives to the traditional small business of yester-year.
Although it may not be practical long term for a business that needs to keep a lot of inventory on hand ..... or have clients attend regularly for hands on delivery of service .... there is no reason that a home office cannot be totally adequate in the long term. Client meetings can be held at the client's office, over breakfast at a nice venue, or informally at a Starbucks or patio cafe. Most larger centres also have virtual offices where a meeting room can be rented. These virtual offices can also provide a proper "business" mailing address and telephone answering services.
A mailbox rented at a local UPS store also provides a street mailing address, rather than using your home address. P.O. Boxes are to be avoided as they have a stigma of "something to hide", "fly-by-night".
There are many web based collaboration tools, some inexpensive, others cost free, such as Basecamp, Zoho, Clientspot and Central Desktop to name a few. There are also many such tools and programs to be had for other specific purposes. For example, a solo lawyer or small lawyer firm may want to check out GoClio.com. Clio is web based and contains everything that might be needed to manage the practice at a reasonable monthly fee, which saves the huge cost of buying a number of programs, and is becoming increasingly popular.
When some back office/admin help is needed, consider hiring a Virtual Assistant (VA). Although VA's can be found worldwide, it is recommended that you choose one who, as well as having many years experience, is in a country where English is the first language. The best scenario is to find someone in your locality, so you can meet from time to time, collaborate and form a strong working relationship.
And then of course there are local networking events - check out LinkedIn Groups for some in your area, as these have networking events from time to time - your local Chamber of Commerce and and all the social media sites. At this time it would appear that LinkedIn is the top business social site, so it may be advisable to concentrate on LinkedIn at least initially. Trying to be on, and keep up with, several social media sites at once is overwhelming and so time consuing that you'll never get any work done.