Throwing knives are hard to balance. Against my champion with no armor they did 3 damage each. So I took 9 damage from 3x throwing knives, thats a large amount. However, once you get a little bit of defense suddenly they only do 1 damage. Maybe if you gave them 66% armor piercing that would be enough, but I also think it should multiply the damage by the number of figures in the unit.
I do think that if you increase the damage too much then bandits will be extremely dangerous to unarmored units... but maybe thats a good thing? It would certainly add strategy if the types of neutral units you fought depended on what your army composition was.