George is Jewish. They reference this fact when he attempts to convert from Judaism to the Eastern Orthodox faith. Also the template for George, Larry David, is Jewish.
In "The Conversion," nobody ever states that George is converting from Judaism. He's simply converting from whatever religion he is, which could be Judaism, to Latvian Orthodox. It's unclear what religion George was before converting.