Have you ever wander why there are two capacitors (an electrolytic and a non polar cap) in the same output line of a power supply? Electrolytic capacitors have thin aluminium film rolled inside the can. This kind of construction makes it inductive like a coil. The inductive reactance is negligible at power line frequencies, but it does become significant at radio frequencies.
Such capacitors therefore, cannot bypass high frequencies to ground because of their inductive reactance. In order to solve the problem a non polar cap like polyester, ceramic and etc of smaller value in parallel with the electrolytic capacitor is use to bypass the high frequencies.