As far as I know, we cannot limit the number of engines per hull directly. But there's several things we can do.

We could, for instance, chance engines so that instead of taking up space, they reduce the total space by a % - say 20% per engine. This would prevent you stacking more than 5 engines on a hull, without breaking small ships. I suspect you can do this by opening up gal civ 3\data\game,  finding the Shipcomponentdefs.xml file, opening it in notepad and then finding every instance of MovesCapMass, then changing the BonusType to 'Multiplier'. You may have to rework some blueprints, though.

Alternatively, the approach I used in IAB is to add newtonian physics to hulls - adding movement reduction modifiers to bigger ship types. Making tiny hulls have +50% speed, and then gradually reducing this per class until huge hulls have -70% means that as you start to get more space to stack, you need to stack to maintain reasonable speed - in this case, using up about half the ship's hull with engines will result in the same total speed regardless of hull size. Again, tough, you'd likely have to re-write the default blueprints for the AI to produce effective ships.

You could also just massively increase the mass used by engines, but be aware that this will cripple smaller hulls completely.

Well, you could preclude them based on each other's presence. That'd limit you to 1 per hull, so you'd probably want to double or triple the movement points per engine. But yeah, you'd once again still need to rewrite the blueprints heavily, since you'd be changing a fairly default assumption in the designs.

That would be a possibility for preventing the ship from having more than one of a specific drive component on it, yes. Do be aware that if you don't have anything scaling the hull capacity required for the drive systems this still favors larger hulls, as larger hulls will still use up relatively less of their capacity on the drive components. Also note that if you want to preclude stacking different types of drive component (e.g. Warp Drive + Stellar Folding), you have to add a Preclusions list that includes each component you don't want the drive to be able to stack with.

That will not work. You'd need to use an EffectType of MassCap to affect the hull capacity directly; MovesCapMass with a BonusType of Multiplier just modifies the capacity required by each component that uses MovesCapMass. Also remember that simply changing the EffectType from a flat MovesCapMass to a multiplier MassCap will render several of the specialization choices somewhat worthless, as if there are no worthwhile components with a MovesCapMass to modify then the -X% drive mass options have no value, and 'correcting' this issue by changing the techs to a hull capacity bonus has the issue of helping everything rather than simply reducing the cost of adding drive components.

I can confirm that using an EffectType of MassCap with a BonusType of Multiplier and a negative Value does indeed work on ship components; a drive component with a -10% hull capacity modifer did indeed reduce a Tiny hull's capacity by 2.5 for every drive component added. Remember, though, that this modifier stacks additively with the hull capacity bonuses from techs, some improvements, events, etc, and so if you choose, say, -10% hull capacity per drive component, then every +10% hull capacity bonus you get is another drive component you can add to the ship; you're not looking at a system where you can guarantee that no ship can have more than 10 drives installed.

Using an EffectType of MovesCapMass with a BonusType of Multiplier and a positive Value in addition to an EffectType of MovesCapMass with a BonusType of Flat and a positive Value on the drive components works as a soft cap; each additional drive component will require more hull capacity than the previous drive component.

Adding an EffectType of MovesCapMass with a BonusType of Multiplier to the hull also works; adding MovesCapMass multipliers with Values of 0, 1, 2, 2.6, 3, and 9 to the Tiny, Small, Medium, Cargo, Large, and Huge hulls will keep the size of each drive component constant relative to the capacity of the ship, at least until other MovesCapMass multipliers are in play (remember that modifiers of the same type stack additively, not multiplicatively); you can add MovesCapManufacturingCost multipliers with the same values to keep the relative costs the same. MovesCap multipliers on the hulls can accomplish much the same thing; the values you'd need for keeping the fraction of the hull capacity used to attain a given speed roughly constant would be 0, -0.5, -0.67, -0.72, -0.75, and -0.9 for Tiny, Small, Medium, Cargo, Large, and Huge hulls, though note that the game will round fractional movement values up. Personally, I'd rather scale the capacity and manufacturing required per component than make each component give fractional speed bonuses.