Yes, maternal side I think. He was driven by a fear of this, amongst other things. It was only refuted after death.
He was a strange cat indeed. His own “perversion” is very likely a large contributor to the suicide of his biological Niece Geili, who lived with him in Berlin.