How do you know they are infertile? No male, past experience, etc.?

You should not try to remove them right away. When the female is fairly sure that the eggs will not hatch, she will leave the nest. If you take them too soon, she will do what is known as double-clutching. A bird is programmed to lay so many eggs each time she lays a clutch of eggs. If you remove the eggs too soon, she will try to replace them. This can be very detrimental to the female's health as she loses up to 30% of her body weight during nesting season. This would be particulary dangerous in the winter months because too much weight loss can lead to illness. If you know when she laid the eggs, count approximately 40 days. If the eggs have not hatched by then, she will probably abandon the nest anyway. Once you see her spending more than 30-45 minutes away from the nest, you can safely assume that the eggs are not viable because a mother swan usually will not stay away from the eggs more than 30 minutes. The male swan will stay by the nest to watch against predators until the mother finishes bathing and eating and returns to the nest. Once hatching is imminent, the female will spend less and less time away from the nest and will become extremely restless. The Regal Swan