Allan is Jewish.
Allan plays the clarinet. He started playing after he heard a recording by Benny Goodman.
Allan had an advertising photography business with his first wife, Diane Arbus, in the 1950s. It was based in New York City and shot both local and national ad campaigns.
Allan is the former brother-in-law of poet laureate Howard Nemerov.
Allan was a photographer for the United States Army during the 1940s.
Allan married Mariclare Costello on 24 July 1977.
Allan had two children named Amy and Doon with his first wife and one with his second Arin.
Allan married Diane Arbus in 1941 and later divorced in 1959.