Born August 2, 1924 in the Bronx, New York. In his early years, O'Connor directed his efforts toward stage acting, mainly performing in the European cities of Dublin, London, and Paris. During the 1960s, he appeared in a number of obscure films, including By Love Possessed (1961) and In Harm's Way (1965).
Shortly after, O'Connor was offered the lead role in an American adaptation of the popular British program Till Death Do Us Part. He accepted the part and starred in the American pilot. CBS eventually agreed to air the sitcom in 1971, after prolonged negotiations because of its controversial content. Renamed All in the Family (1971-79), the show featured O'Connor as the working-class ignoramus Archie Bunker, a character redeemed only by his endearing wife Edith (Jean Stapleton), liberal daughter Gloria (Sally Struthers), and son-in-law Michael, also known as "Meathead" (Rob Reiner).
In the post-civil rights era of the 1970s, the series, created by Norman Lear, was a forum for social commentary on uncomfortable American issues like racism and bigotry. O'Connor deftly portrayed Bunker as a somehow likable blue-collar worker just trying to get along in life. His performance earned him eight nominations and four Emmy Awards. After the show ended, he continued to portray Archie Bunker in the less-successful Archie Bunker's Place (1979-83). His other notable television role was as the liberal sheriff on the well-received In the Heat of the Night (1988-92), for which he won an Emmy in 1988.
O'Connor made headlines under tragic circumstances in the mid-90s when his son, Hugh, an actor who worked with his father on In the Heat of the Night, killed himself after an unsuccessful bout with drug addiction. An acquaintance of Hugh brought slander charges against O'Connor, after Carroll went on television claiming that he supplied drugs to his son and was partly responsible for his death. O'Connor was cleared of all charges, and became an active anti-drug spokesperson, lobbying for laws that would give individuals and states the right to sue drug dealers for damages. He has also appeared in numerous public service announcements urging parents to help their children refrain from drugs.
During the late 1990s, O'Connor had recurring roles on the Fox series Party of Five, and on Mad About You as Helen Hunt's father. Most recently, he appeared opposite Minnie Driver and David Duchovny in the romantic comedy Return to Me (2000).
On June 21, 2001, at the age of 76, O'Connor died of a heart attack brought on by complications from diabetes.