I've owned two different XJ's. The first I got rid of the sway bars completely. The second I kept the front sway bar and made my own disconnects for it.
Removing the rear sway bar made almost no difference at all in on-road handling but did allow a bit more articulation in the rear. Not much due to the ability of the leaf springs, but did help.
Up front, though, there is a BIG difference. Without the front sway bar, the XJ has a LOT more body roll even at stock height. When lifted it was downright scary doing things as simple as changing lanes on the freeway. Thank goodness I never had to swerve or I'm pretty sure it would have gone over. That rig was pretty much trail only so I didn't worry about it that much and made sure to give myself plenty of room around other drivers. But was bad enough that the next XJ kept the sway bar.