If you look at a beam of light whether projected from your enlarger's negative carrier or a shaft of light shinning across a room you will notice lots of particals floating in the air. Dust over the course of time will settle on the upper surface's of your enlarger and lens.
It does not take long for a considerable amount of dust to build up on the surface of your lens greatly reducing the brilliance of light passing through it. You should make it a rule that once or twice a year you clean the lens thoughly. It is also a good idea to cover the enlarger at the end of each session with its cover if it has one or a black plastic bag this will greatly reduce the dust build up.
If you have a window in your darkroom that needs to be blacked out it is not a good idea to use curtains as these are dust traps. You are better off using a wooden shutter made of hard board painted matt black or a roller blind made of light tight material that has some way of being sealed along the sides.
What you have on the floor is also important. If it is carpet it needs to be removed, as this is a dust trap too. It should be an easy clean material like lino if you have floor boards or garage floor paint for concrete floors these keep the dust at bay making it easy to mop the floor with water, it also means that you do not have to vacuum very often leaving more time to print.