First off, you definitely have to shut off the main. Then you will need to drain as much water from the system as possible. Open up any fixtures on lower floor levels??
The below pic is of a CPVC fixture shut off. In the picture you posted, it appears there is enough pipe extending through the wall to cut it off close to the existing valve, and glue the new one on.
Use a good quality primer and glue, and wait the appropriate amount of time before you turn the water back on. Make sure you have the shut off valve in the off position when you turn the main back on. Also, make sure there is no water dripping out of the pipe when you glue the new one on (this is fairly critical to a proper solvent joint).
When you are at the hardware store, get one of these (see below pic) as well. It will make your job much easier.
Here is a pic of the two parts together. The new toilet flex supply has rubber washers in it, so a simple adjustable wrench works.
Or you could try simply tightening the nut (closest to the wall) to the valve body. I can't guarantee nothing bad will happen if you try this one though.
Thank you sir! I am not sure I will take it on this time or not, but I really appreciate the advice.