My favorite way to ruin their lives is push them all out of their homes and replace the doors with iron doors. Then I steal all the crops, flood their gardens, and make them watch helplessly as I burn their village to the ground!
Or I just put a zombie in there, but that's not as fun.