You can join any raid party. The side you choose won't have any permanent effects at all.
"It is not a disgrace to fail. Failing is one of the greatest arts in the world," however --
"Believe and act as if it is impossible to fail." ~ Charles Kettering