Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

This "technique" is highly recommended for your error pages also (50x). It also helps if you set the "Retry-After" HTTP header (value in seconds), that tells google (and hopefully other crawlers) that they shouldn't bother with crawling for another X seconds. Helps with the load if you are experiencing problems with your server.

Retry-After is also usefull for "down for maintenance" pages since you usually know how long your page will be down.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: