Some people have suggested that the olive was a very deliberate and well-considered choice as a metaphor for peace, because olive trees famously take years to mature. War is typically very hard on the trees because people cannot take the time to nurture them and plant new ones. Therefore, the offer of an olive branch would suggest that someone is tired of war, whether it be an actual war or a falling out between friends.
In Ancient Greek and Roman times, people would offer literal olive branches. In Rome, for example, defeated armies traditionally carried olive branches to indicate that they were surrendering, and the Greeks incorporated them into weddings and other ceremonies. In the modern era, the branch is usually metaphorical, rather than literal, not least because the plants can be a bit difficult to obtain.