The best way to understand the mechanism is to look at the existing ones.
First, how do you change an ability that targets a single ship to an AoE?
For that have a look at the Ability "Cease Fire". It consists of three files:
AbilityCeaseFire.entity If you want to use this ability you just have to add this name to the bluebprint of the unit. It determines things like Anitmatter-cost and Cooldown-rate. The ability file calls a "Buff"->
BuffCeaseFireSpawner.entity This one dertermines who will be affected by the buff und calls another buff ->
BuffCeaseFireTarget.entity THis one the actual effect(In this case everyone stop fireing)
AOE-Abilities normaly work that way. Passive and direkt abilities mostly have just one ability and one Buff.
Is it possible to alter the hangar squad research the Advent have so that it affects star base squadrons as well?
No, at least not with the current method. The number of the fighters is increased in this file: RESEARCHSUBJECT_PLANETMODULEIMPROVEMENT_PSIHANGARDEFENSE.entity
Especially this lines:
"
researchModifier
modifierType "HangarFightersPerSquadAdjustment"
baseValue 0.000000
perLevelValue 1.000000
"
There is only a limited number of research modifiers made by the devs and as far as i know there is no way to change them(In this case wich target is affected).
But ist still might be possible to get the effect, by inventing a new Ability-Buff-Buff combination , working on starbases, but you have to figure that out yourself...
How do you create a research subject that boosts the speed of ships?
This one not so easy for the beginning. I dug out an old mod of mine recently that does basically only that. So have a look at it: http://www.filefront.com/14134019/HitRunLF.zip
In short you have to make a new research subject, a icon for it, a desription(english.str)and a .brush file to get the icon from the picture. Further you need a new ability and a new buff wich you can use on the units later.
And is it possible to change the fleet command cost of a capital ship, so you could build two of a ship before having to research an upgrade for capital ship slots?
Hmm, tricky one. I never stumbled upon the initial value of "CapitalShipMaxSlots". Might be changeable, but i don`t know. May be someone else knows. A workaround would be to make the first upgrade level for free.