I once chose to fight some one who would cause me to lose 1 honor if I won. I did win but not only did I lose 1 honor but I also received a stat decrease. Same with attacking someone who would cause me to win honor, I lost the battle and also received a stat decrease. Perhaps there is another factor I am missing, but it definitly showed a negative to a couple of my stats after each battle.