An X Windows server provides a very basic graphical environment, usually connected to a monitor, a mouse and a keyboard.

A display manager (DM) is just the login screen, which runs in its own X windows server session. When you log in, the DM will usually start an entirely fresh X session and will also start the window manager (or desktop environment) that you have chosen.

There are many window managers (WM) available for X. These provide the basics of handling switching and focusing windows, positioning, resizing and decorations (or no decorations in the case of twm/ratpoison/awesome).

A desktop environment (DE) usually provides a window manager but a bunch of other stuff too: panels, start button, applets for volume, network etc. They provide an environment in which everything looks similar and easily recognisable, and provide GUI tools that allow you to do many admin tasks without having to type in a terminal. They often provide a display manager too, for a consistent end-to-end experience. And in the case of Gnome and KDE, a full suite of applications too!

So hackers can survive with just a window manager, but general computer users prefer a desktop environment where everything is integrated in a familiar style.

The fun thing is, thanks to Unix's philosophy of decoupling components, and since the only common required component is X windows, you can mix and match the rest if you want to. You could use Trinity KDE's display manager but ask it to run the Fluxbox window manager when you log in, and then startup LXDE's panel (lxpanel) to provide some icons, applets and a workspace pager. You could even run bbkeys to perform window management actions using your favourite keybindings, and keep using it if you switch to a different WM later! :)

(A setup like that can be achieved with configuration in .fluxbox/startup or in .xinitrc if you skip the DM and just run startx from the console.)

Finally, many window managers will let you switch to a different window manager via their menu (sometimes found by right-clicking the desktop), without restarting the X session or losing any of your windows! This is great when you want to try out some WMs to see which one you like most. Sadly modern Desktop Environments don't seem to bother with this cool feature.

Some distributions will offer you a failsafe login, which may provide an x-terminal with no window manager. From there you can boot a window manager, kill it, start a DE, kill that, start a new window manager etc. The X session itself will not close until that x-terminal does.