When you setup your USB stick it asks you for additional space to use in order to keep your data. There you can store data that you may use between sessions. For example, you can "try" Ubuntu, connect to a network and then restart and boot again from the USB. The network you connected to will be remembered from data saved into the USB itself.

Main Advantages:

Not another Operating System into your hard disk. You don't have to deal with partitioning and it doesn't slow a bit your boot time by showing the grub menu.

Portable. You can boot from that USB from any PC that supports booting from USB.

Main Disadvantages:

Things generally run slower than usual because of lower read/write speed in USB sticks.

Every time you have to insert your USB and wait for the Operating System to load through it. This can be very slow if your USB is slow itself.

Very limited disk space. The 'disk space' is essentially the space you've configured to use inside the USB (while writing the ISO to the USB). So things can get tight.

All of the above considering that you are booting Ubuntu from a USB stick.

Of course, you can install Ubuntu alongside Windows and select from a menu your OS of choice on each PC restart.