What you are describing is a common occurrence during this time of year which is the nesting/mating season for swans. The lone swan is probably a young juvenile male that is just learning the ropes about courtship. The other three are probably one male and two females. The second female may have been a juvenile from this year and the parents are still allowing her to stay in the area. The loner, being a male is too much of a rival once he turns white and becomes sexually mature to the original mating pair. Do not worry, he will find a mate of his own and establish his own territory for raising his future family. The Regal Swan