New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
redirect old urls which are found by analytics #380
Conversation
Makefile
Outdated
nixos/packages-explorer.js \ | ||
nixpkgs/packages-channels.json | ||
|
||
|
||
robots.txt: $(HTML) | ||
echo "Users-agent: *" >> $@ | ||
echo "Disallow: /" >> $@ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This seems not optimal. Why?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Disallow everything first and then allow things we generate. How would you do it otherwise?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Another solution would be to just have Users-agent: *
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess I'm worried disallowing things that don't exist would just simply make 404's stop showing up from search engines, instead of showing up in a way we could fix them.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That is actually a very good point. I'll commend the Disallow
and Allow
lines for now and let search engines catch everything.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Any **/css/*.css
in subfolders are actually caused by #354. They should be dropped from this PR if the files are available at the root level, and the 404 page be fixed rather.
to find 404 parts
No description provided.