Taking towers down gradually is extremely inefficient, use burst abilities along with a teammate instead of wasting time on small pushes with creeps. This way you end up having to deal less total damage due to tower regen.
Getting a beast to spit + reg using mines for example will down a tower very, very quickly during the mid game.
Vlad makes a good point, and even on a map like Cataract, if you need to be right at the enemy gates to farm, you have a very long way to run if trouble comes your way, and a thus higher chance of getting killed. Ideally you want to be able to farm just in front of the flags, away from their towers, and getting priests certainly doesnt help with this. When you want to push, you can AoE all the enemy creeps out of the way at little cost and hit towers as a team for much greater effect than allowing the front to shift due to creep upgrades and gradually wearing them down.
Yes priests give you some small benefit in term of firepower and healing, but they give your opponent a giant sack of cash they can spend on better items to kill you with. If you can't stand toe to toe with your opponent becauase his items are a lot better than yours, you are in trouble. Yes you can run around capping flags and backdooring, but a decent opponent will not let you do that to any effect. Eg. on Cataract, once one team has a decent combat advantage they will take the health flag and the opposition gold flag, and these may well remain locked for the rest of the match. This happens if you buy priests and give your opponents a gold advantage from them...
.'. don't buy priests until you're ready to win. They are good for pushing the enemy citadel in conjuction with catas, not for pushing towers early/mid game.
Maybe those of you who say priests arent so bad early/mid play the game completely different to me, but I have to say every time someone buys early priests I end up many many levels ahead of them and my team inevitably gains total control of the game once we have all bought mageslayer/other stuff from all the gold we've farmed. My experience is therefore that buying priests before you're ready to backdoor and lock -> end the game is not beneficial.