WWW FAQ: How can I mirror part of another server?

How can I mirror part of another server?

Scripts are available to do this, but at this time they are not very friendly to the server you are attempting to mirror; their behavior resembles that of the more poorly written WWW robots. If you are trying to improve access times to a distant server, you will likely find the "proxy" capabilities of CERN's WWW server to be a more effective and general solution to your problem.

If you are interested in mirroring part of a server you control, perhaps to improve reliability by providing alternatives, then there are many available mirroring tools. Keep in mind such simple possibilities as (under Unix) a cron job that tars and compresses the website on the "master" server at a particular time of day, and another cron job on the "slave" server which takes advantage of the lynx browser to retrieve that document, then uncompresses and untars it. For many purposes, such simple mirroring setups are effective and near-foolproof.

World Wide Web FAQ