While babies are in the mouthing stage (from infancy to 2-3 yrs) and putting everything in their mouths, all small objects need to be kept out of reach. This can be tiring, but is only for a short time.
When dealing with hazards remember that:
Never depend on telling children not to do something.