Is horse riding cled as a 'contact' sport?

Not in the true sense of the term, but most doctors will tells you not to ride either. If you were to fall or get another knock to the head it can lead to serious complications. Since horses are unpredictable, you should avoid riding until you're healed after 3 weeks.