Surprisingly capers are the pickled flower buds picked from a shrub-like bush. The capers are then dried in the sun and later brined or packed in salt. Sometimes capers are allowed to mature to a fruit about the size of an olive. These are sold as caper berries and are brined to be eaten like pickles or olives. The larger size are often included in an antipasti platter.
Capers have been around since ancient times. They're grown in parts of Asia, the Middle East and the Mediterranean region, including north Africa, southern Europe and Turkey. Capers are also grown in California.
Capers are too small and delicate to be plucked by machine and therefore must be harvested by hand - why a jar is so expensive. The capers are then sorted by size and then dried, brined or salted, the smallest size, called nonpareil, is the most desirable and most often used in recipes.
(To use capers in recipes it's a good idea to rinse them first, to remove all the excess salt or brine.)