Slashdot | Web Pages Are Weak Links in the Chain of Knowledge (quoting the Washington Post's On the Web, Research Work Proves Ephemeral). It's true that linkrot is a serious problem. It's also true that archive.org is only a partial solution since it doesn't get anything and some big content providers — like the Washington Post — block it.
Is the only solution to make (copyright busting?) offline copies of everything? If so, where's the tool that will automate that for me, and — more importantly — index all that content on my drive, disk, or tape?