Got my flames! I feel all squishy inside.
***Spoilers, although it's nothing that hasn't been discussed in here already...***
On further play, I must admit, the traders are likely the Gorons. Their lips and beards looked exactly like big noses and rabbit teeth to me, but on closer inspection they look exactly like Gorons.
But I stand by my "retarded" contradictory evidence. I still believe that the Gerudos may have a hand in the Rito bloodline, and I still believe Nintendo purposely included evidence to that effect. The Koroks are clearly direct descendants of the Kokiri and nobody else, the Deku Tree essentially tells us so. However, the only evidence that the Rito are derived from Zoras is that Medli is part of the Zora bloodline, and I don't believe any other evidence was given. This means that the Rito are almost certainly descendants of the Zoras, but it doesn't preclude them being descendants of the Gerudos as well. On the contrary, I think Nintendo purposely included evidence pointing to the Gerudos.