Come to think of it, I can't really find anything that contradicts the Bible in Binding of Isaac (well, outside of the standard videogame aspect of shooting enemies). Well, okay, using the bible on Satan gets you killed but that guy's the super hard bonus boss. Other than that the faith based stuff has all the holy things give benefits while the enemies are only based on evil entities (may change with the Wrath of the Lamb expansion but we don't know that yet) and making a deal with the devil is paid in blood, often turns you into a monster and may trigger additional enemies (e.g. Krampus may show up). Of course there's a lot of non-faith-related stuff, a lot of which is based on the kinds of cautionary tales children are told by their parents (e.g. the roid rage). The game is overall based on the impression that those stories have on small children and the actual gameplay is clearly a nightmare Isaac is having. I'm not sure he's even supposed to be applying the coat hanger to himself, in a nightmare context that may just be a fear of the things based on all the horror stories about abortions he was probably told.
McMillen said in an interview that it's about how he felt about the stories he was told as a child so many of the things in it are probably manifestations of fears brought by those cautionary tales.
Of course the game is also full of videogame in jokes, including various Zelda and Mario items.