Do we not all have the right to keep others safe?
If people want the right to do anything, then how about the right to murder anyone we don't like, or steal anything we want?
We all have the right to keep others safe, so stopping someone doing something is the right thing to do