It usually only matters in the amateur ranks. A fighter is judged on the scorecards as either red corner or blue corner. White corners are considered neutral corners where a fighter is sent in case of a knock down (furthest neutral corner) or a time out is called by the referee.
Going back to Japan, the red was always the visiting team and the blue the home team. This is an age old tradition.
When competitions are larger in terms of how many competitors keeping track of points by who / name becomes much more of a headache for the referees and score keepers. The ref can wear a blue wrist band on one hand and a red on on the other to show how many points were just scored by simply holding up the colored wrist band thus circumventing the headache and confusion of awarding points to the wrong competitor.