The Romans named the first month starting after the winter solstice after Janus, the god of doorways, and therefore of openings and beginnings. His two-headed demeanor was appropriate for looking back at the old year and forward to the new. Roman consuls chose 1 January as the start of their term in office as of 153 BCE.
Not everyone was attached to the January date, however. The Babylonians celebrated their new year on the first new moon following the vernal equinox (the date on which night and day are of equal length in Spring). In Egypt, the new year started with the annual flooding of the Nile in Spring.
Christians in medieval Europe frequently used dates of religious significance, celebrating Christmas, or 25 March (the Feast of the Annunciation), which also coincided with the return of Spring, since it would have been shortly after the vernal equinox. It was Pope Gregory XIII who reestablished our modern New Year's Day in 1582, when he reformed the calendar.
Of course, we don't celebrate the start of the new year so much as we celebrate the end of the old year. Celebrations begin in the waning hours of the old year, on the final day of December, extend briefly into the hours of the new year, and then the celebrants usually fall asleep and spend the 1st of January recovering!
