Weeks ago, I looked for a utility that would help me check for my blog's dead links, and I bumped onto LinkChecker (version on my system is 6.2). It turned out to be impressively advanced, and here's the summary from its manpage:
LinkChecker features recursive checking, multithreading, output in colored or normal text, HTML, SQL, CSV or a sitemap graph in GML or XML, support for HTTP/1.1, HTTPS, FTP, mailto:, news:, nntp:, Telnet and local file links, restriction of link checking with regular expression filters for URLs, proxy support, username/password authorization for HTTP and FTP, robots.txt exclusion protocol support, i18n support, a command line interface and a (Fast)CGI web interface (requires HTTP server).