If all connections at battery and back of alternator are tight (check for rub-through and rodent damage at obvious wiring), buy a DVOM, you can get a cheap one at Auto Parts store for 20 bucks. With the engine running, set the DVOM meter to V or 12V. Put the red lead on the positive terminal and your black lead on the negative terminal. That simple. I'm sure there are youtube vids on how to do it too.
Charging voltage should typically be 13.8 up to 14.5. If it's lower than 13.5 or dips into the 12V range while running, your battery isn't being charged which is why the MIL illuminated on the dash.
99% of the time, it's a bad alternator. If your voltage regulator is external (which I don't think is the case on the JKs), then it's most likely an internal alternator issue. Also check belt tension obviously to make sure alternator is turning and belt didn't come off.
EDIT: If the light stays on and you keep driving it, the Jeep will most likely die and a jumpstart WON'T keep it running. You will have to put a charged battery in to drive it and then that one will leave you stranded once it's done with the charge that's in it. So if it's under warranty, get it to the dealer quickly and safely.