Date: 2006-01-03 02:57 pm (UTC)
Not everyone respects robots.txt :-)

The thing is, once you have content in an open format like HTML, anyone can do anything with it. You'd need to put your site content in FLash or similar.

One option would be to build you site as a content negotiated CMS and just block out the IP addresses or HTTP User Agents of scraping tools. That would work...

This account has disabled anonymous posting.
If you don't have an account you can create one now.
HTML doesn't work in the subject.
More info about formatting

Profile

green_amber: (Default)
green_amber

May 2009

S M T W T F S
     12
3 456789
10111213141516
17181920212223
24252627282930
31      

Most Popular Tags

Style Credit

Expand Cut Tags

No cut tags
Page generated Jul. 10th, 2025 01:40 am
Powered by Dreamwidth Studios