Are you manually queuing up ships in your shipyard build list, or are you queuing up constructor requests in your starbases? If you are manually queuing up constructors, they won't know where to go. If you are queuing up constructor requests, they should show up in your shipyard build lists as "Starbase Factory (for Starbase Draginol 1)", or something like that. They don't keep those labels when they get launched (darn), but if you are tracking to see if it is working, it can be helpful to see where the next constructor thinks it should be going.
One thing I do to help keep things straight is name my default starbase upgrade constructor designs, "up1", "up2", etc, according to how many constructor modules they have. That keeps them different from the constructors I manually build in order to have them go out and create starbases.
Otherwise, I can't account for what you are seeing. Any more details available?