You should not be disrupting them this much when they are breeding, if the nest box is up and they have eggs or you expect eggs soon. Sometimes you have to let the cage get dirty during breeding and hatching chicks, otherwise the parents will abandon the eggs or chicks. You can clean the cage tray but just quickly change the paper or litter. If the birds are too worked up, then try another time. As far as feeding and watering them, do you have to reach in the cage? Most cages have access from the outside to the dishes. If this is not the case, try to change the water and add food when the male is in the box. You can try putting a towel over your hand or wearing gloves. Whatever you do, again be slow and move away if they get too upset. You should not be trying to remove them from the cage during this time.
If they are between clutches, the nest box should be removed from the cage or blocked off. They should be rested for a few months between clutches and not have access to the nest box. In this case, you can remove the birds or wear gloves to clean the cage and feed and water them. But honestly if they have to be removed just to clean the cage or change food and water, I would look into a better cage. Most cages are made where you can easily do these things without having contact with the birds.
Thank you for asking Lafeber.