The most significant greenhouse gas is actually water vapor, not something produced directly by humankind in significant amounts. However, even slight increases in atmospheric levels of  carbon dioxide (CO₂) can cause a substantial increase in temperature. 

Why is this? There are two reasons: First, although the concentrations of these gases are not nearly as large as that of oxygen and nitrogen (the main constituents of the atmosphere), neither oxygen or nitrogen are greenhouse gases. This is because neither has more than two atoms per molecule (i.e. their molecular forms are O₂ and N₂, respectively), and so they lack the internal vibrational modes that molecules with more than two atoms have. Both water and CO₂, for example, have these "internal vibrational modes", and these vibrational modes can absorb and reradiate infrared radiation, which causes the greenhouse effect. 

Secondly,  CO₂ tends to remain in the atmosphere for a very long time (time scales in the hundreds of years). Water vapor, on the other hand, can easily condense or evaporate, depending on local conditions. Water vapor levels therefore tend to adjust quickly to the prevailing conditions, such that the energy flows from the Sun and re-radiation from the Earth achieve a balance. CO₂ tends to remain fairly constant and therefore behave as a controlling factor, rather than a reacting factor. More CO₂ means that the balance occurs at higher temperatures and water vapor levels.