Small is beautiful. It’s also less hassle and much more agile.
As the sole director of my business I can hold my board meetings in the shower and, if I come up with a bright idea, I can often have it implemented by evening.
The downside of a small business is reduced cover for holidays and sickness. Especially if your business is just you. On the other hand even the best team will take up your time in managing their time and workload.
Before growing your business beyond you it is worth considering whether this will best suit your personal goals. Instead of taking on more clients and higher overheads you could look at taking on better clients and providing them with more profitable services. Before recruiting staff you could investigate software and other automation.
Bigger isn’t always best.