They are supposed to act as one way valves, to let any moisture out and keep out anything else. Some vehicles used to use simlar type valves (plugs) in the door drain holes, to help reduce windnoise. I don't think that is currently the case, as the proof of benefit wasn't significant and the risk of plugging up the drains and increasing corrosion was high. If you want to just leave them out, I don't see any reason to worry. And the reason they fall out so easily when barely touched has to do with them not being installed correctly in the first place. One of mine fell out almost immediately. After I studied the remaining part it was obvious they weren't installed correctly. I bought new ones and have had no further issue, even though my wife or I wipe this area dry after a car wash.