If you stay for too long around Slaver camps (Slave Farm, South Stone Camp and Eyesocket) or UC cities, it's what I do to recruit more people for my cause, you will laugh a lot with the situation.
P.S: Outlaw (farmers and swordsman) slaves are more common to have aggro (almost 100%) but it happens with normal slaves (from Outlaw faction) and Starving Vagrants and others factions.....a way to prevent slavers attack is to pick up the slave and go away, then put the slave down out from town/camp and run for a while to trigger the slave response (stay or go)