ext_40483 ([identity profile] sbisson.livejournal.com) wrote in [personal profile] green_amber 2006-01-03 02:57 pm (UTC)

Not everyone respects robots.txt :-)

The thing is, once you have content in an open format like HTML, anyone can do anything with it. You'd need to put your site content in FLash or similar.

One option would be to build you site as a content negotiated CMS and just block out the IP addresses or HTTP User Agents of scraping tools. That would work...


Post a comment in response:

This account has disabled anonymous posting.
If you don't have an account you can create one now.
HTML doesn't work in the subject.
More info about formatting