You have to feed the plants nutrients. You can either buy nutrients at the npc in the forest or farm them from monsters. Plants take time to grow, if you hover over them you see what nutrients they need, how much of them and the time it takes them to grow.
Fast plants need to be fed 3 times before you can harvest them. Medium plants 4 times and slow plants 5 times. It will tell you when hovering the mouse over them, they also look different when they are ready to be harvested.
And don’t remove them from your estate by unfixing them or such. That will only destroy them.