My only aversion to vice, is the price.
Meaning of the quote
The quote means that the only reason Victor Buono didn't like doing bad things is because they cost him something. He didn't care about the bad things themselves, but he didn't want to pay the price for them. In other words, he was only against vice (or bad behavior) because of the consequences it had, not because he thought the bad things were wrong.
About Victor Buono
Victor Buono was an American actor known for playing villainous and older roles, including King Tut in the TV series Batman and musician Edwin Flagg in the film What Ever Happened to Baby Jane?. He was nominated for Academy Award and Golden Globe Award for his work.