Ive encountered this in multiple games ive played and multiple fleets.  When I have a fleet size over about 30ships and I select them all and right click to attack a target only about half respond with an attack order. The other half act as if it was a move command and either stay where they are or move to the general location of where I told them to attack.  In order to get the other half of my fleet to attack I have to select them separately and order them to attack.  So far it does this when my ships are all in one fleet and when theyre not.  Basically when I have a large group selected under any circumstances is does this.  I havent had a chance to try using the button to click attack just when I right click.  Thanks let me know if you need more details.