It seems they didn't read the associated KB article (941314) linked from the download page. (That's ok: I missed it too the first time, and went stumbling around for it like they did). The KB article describes what RSAT is, what tools are included, and most importantly: how to install, and uninstall it. The key section would be "Install RSAT" about 2/3 down the KB article page. Oddly enough, the Windows 7 RSAT download page includes all of this information on one page. Microsoft usually gets things right eventually.
RSAT, or "Remote Server Administration Tools", is the new name for the former "Administration Tools Pack", or "Admin Pack", for Windows Server 2003. The Microsoft RSAT team threw in GPMC, and a few other extraneous tools as well, which is very much appreciated (by me anyway).
To make it a little easier: After you install RSAT, go into Control Panel, open Programs, and select Turn Windows Features On or Off. Locate the entry for "Remote Server Administration Tools" and enable it there. You can enable individual tools within RSAT or the entire suite if you prefer. Don't forget to enable "Administration Tools" on the Start Menu also.
To verify installation on Windows Vista, go to Control Panel, Programs, and click on "View Installed Updates". Look for the entry "Update for Microsoft Windows (941314)". On Windows 7, look for entry "Update for Microsoft Windows (958830)". Alternatively, you can search for the file "rsatclient.dll" in the %WINDIR%\System32 folder.
KB934307 - Description of the Windows Update Stand-alone Installer (Wusa.exe) and of .msu files in Windows Vista and in Windows Server 2008