I have a similar situation. I would draft an employment agreement where you list your prior clients and retain the right to notify them if you leave the new employer. Clients obtained while employed at the new firm will usually remain clients of that firm and you will probably be bound by a non-compete / non-solicitation.

If there is no agreement, then you may not be bound, if there is an agreement, then you will need to spell out your rights and duties as to your prior clients.

Good luck.