[Yanel-dev] Filtering search bots

[Balz Schreier]

first of all this only matters if you store a lot of data in the session.

if you do, it certainly hurts and is waisting memory consumption for
nothing.
in such a case, it would make sense to check the user-agent in the request
header.
if it matches a bot-value provided by a configurable agent-list, the request
would be associated with a dummy session.
all requests that would match would be associated with that session and
therefore would not create a new one.

the risk here is only that if the request matches but is actually a real
user request, strange things will happen for all users sharing the same
session :-)


On Thu, Mar 10, 2011 at 4:37 PM, Michael Wechner
<michael.wechner at wyona.com>wrote:

> Hi
>
> With the improved session manager I had a look at the number of sessions re
> one of
> our customers and noticed again that many search bots do not accept/keep
> cookies
> and hence create with each request a session, which is kind of "useless"
>
> Related to that I have found the following link:
>
>
> http://www.seomoz.org/blog/controlling-search-engine-access-with-cookies-session-ids
>
> I am not sure if it makes sense to investigate this more, but any ideas you
> want to share are appreciated.
>
> Thanks
>
> Michael
> --
> Yanel-development mailing list Yanel-development at wyona.com
> http://lists.wyona.org/cgi-bin/mailman/listinfo/yanel-development
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wyona.org/pipermail/yanel-development/attachments/20110310/d49821ed/attachment.html>