Refineries boost production of your ore and crystal mines on the planet at which they are built, and all planets 1 jump away. Each mine on a colonized planet can support 3 refineries, and neutrals can support 4. Any more than that won't have any effect.
The Advent don't have a separate building, but you research a tech that allows their trade port to change into a refinery. It only works in grav well (no 1-jump deal), but it's a bigger boost.