In any phone conference call, no matter what service you use, clarity will decrease as un-muted participants are added. (To maintain high quality in large calls, use Presentation mode.)
Why does quality degrade? Each open microphone adds background noise. Speakerphones can leak the room's acoustic echo. Poor connections add static. Cell phone users might be in a noisy car or outside in the wind. Cordless phones and VoIP calls can inject electrical noise or add distortion.
If someone puts their handset on hold without remembering to mute the line beforehand (by hitting *6), everyone might suddenly hear their company's music-on-hold.
Tip: If music or noise disrupts your call, as the Host you can press *7 to rotate the call into Q&A mode. This mutes everyone's line. Then tell anyone who wants to speak to press *6, which unmutes their line. You'll soon discover whose connection is noisy.