Kryptonite
From Tardis Wiki, the free Doctor Who reference
Kryptonite was a fictional crystaline substance that Superman was weak to. Colloquially, it was a synonym for a person's weakness.
Mike Yates asked the Fourth Doctor if the meteor was weakening him like Superman and Kryptonite. (AUDIO: Starfall)
Gwen Cooper knew H&M was like Kryptonite to Rhys Williams. (PROSE: The House That Jack Built)