diff options
| author | bnewbold <bnewbold@robocracy.org> | 2025-02-28 13:09:48 -0800 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2025-02-28 16:09:48 -0500 |
| commit | 462708e207627971f68b603e61ad70c5d6953cff (patch) | |
| tree | c82cc9811bd6ddf681ac71c1a3b8e07b651be4f5 /bskyweb/static | |
| parent | 36a7a8d8c68fc1d3d3fd396dd9f705c966552fcb (diff) | |
| download | voidsky-462708e207627971f68b603e61ad70c5d6953cff.tar.zst | |
bskyweb: add robots disallow and 'canonical' config flags (#4760)
Diffstat (limited to 'bskyweb/static')
| -rw-r--r-- | bskyweb/static/robots-disallow-all.txt | 3 |
1 files changed, 3 insertions, 0 deletions
diff --git a/bskyweb/static/robots-disallow-all.txt b/bskyweb/static/robots-disallow-all.txt new file mode 100644 index 000000000..65400a08c --- /dev/null +++ b/bskyweb/static/robots-disallow-all.txt @@ -0,0 +1,3 @@ +# This is an development or self-hosted instance of the bsky web app, and crawling has been disallowed by the operator team. +User-Agent: * +Disallow: / |