There are a lot design factors that affect capacitor choice: capacitance, voltage rating, frequency stability, size, temperature stability, series resistance (ESR), etc. etc.
The largest difference in capacitors is their dielectric composition. There are electrolytic capacitors that can have high capacitance and low ESR, but aren’t great at higher frequencies. Tantalum dielectric has better frequency and temperature stability, but with higher ESR. Ceramic capacitors tend to have lower capacitance, but can be very stable and have a much higher voltage limit.