Agreed.
Also, swarm bonus should be countered by opponents swarm bonus.
Thus, if you receive a 3 bonus from having 3 units adjacent to you when you attack, the opponents 2 bonus from having two other units adjacent should counter against your three. 3 minus 2 equals 1. Thus your net swarm bonus should be 1. (numbers are very simple just to get the idea across)