It does say in most of their bio's and quest logs that they do have a little permanent skill damage roughly 10%
what that means is this when you lose to them whether you attack them or they attack you and they do so based on who you attacked if you attack say Knator commander he is likely to attack you back 2 times to see if your defensive ritual can beat him
which mine didnt lol and you lose about .12 to all stats roughly what you gain is more than you lose if you do beat him
which i did lol
98% to 98% lol so close but i barely won and he kicked my butt 2 times for that lol