[kwlug-disc] Should kwlug-disc archives be private?

[Khalid Baheyeldin]

A few minutes of grepping through the logs of one of my sites ...

Drupal has this as part of the robots.txt:

Disallow: /comment/reply/

Despite that, the current log has the following bots still accessing these
paths:

SemRush, a bot for ads and backlinks, https://www.semrush.com/bot/
Sogou - A Chinese crawler https://www.sogou.com/docs/help/webmasters.htm#07
AlphaSEO Bot - +http://alphaseobot.com/bot.html

The point is: robots.txt is advisory in nature, not mandatory.

Back to the main point: many have voiced their objection to making the
archives members only. So, is the consensus to tell those who want to join
that this is the way it is, and if they don't want their messages to be
public they should not join? Will this be our collective stance?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://kwlug.org/pipermail/kwlug-disc_kwlug.org/attachments/20181207/0c091f7c/attachment.htm>