Made the switch a couple years ago and haven't looked back. As a side sleeper, getting rid of that sleeping bag tube was a game changer for me. The freedom to move my legs without fighting the fabric is the best part.
Honestly, the pad is everything with a quilt. I went with a wide one and it made a huge difference for drafts. That extra width keeps the edges sealed down so much better.
Quilts really nail it for moving around at night. I shift from my side to my back a lot, and it just moves with me instead of getting twisted up. @the_harper is right about ditching the tube, though you gotta have a good pad setup to stop drafts. That's the only real trade-off.