ban all spiders from the entire site for development period
This commit is contained in:
parent
8da5323fd0
commit
d5c5200b71
|
@ -1,5 +1,5 @@
|
||||||
# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
|
# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
|
||||||
#
|
#
|
||||||
# To ban all spiders from the entire site uncomment the next two lines:
|
# To ban all spiders from the entire site uncomment the next two lines:
|
||||||
# User-Agent: *
|
User-Agent: *
|
||||||
# Disallow: /
|
Disallow: /
|
||||||
|
|
Reference in New Issue