[Yanel-dev] YarepGroupManager.getGroup() very slow with 5000+ users

basZero baszero at gmail.com
Fri Apr 27 09:36:02 CEST 2012


Hi Michael,

sounds like a plan.
First I will try to find a very simple workaround (e.g. by not storing the
member ids anymore in the group XML), as I think that most functionality is
not needing that, am I wrong? I think it is enough just to have the group
definitions in group XML and have the group assignments in the user XML
only.

But to be clear: the above description is not the final solution :)

Yes we can work together on this.

Please also have a look at the pending pull requests which I posted 5
months ago, would be cool to have them finally in so that I can remove all
custom Yanel classes again...

Cheers
Balz

On Fri, Apr 27, 2012 at 9:25 AM, Michael Wechner
<michael.wechner at wyona.com>wrote:

>  Hi Balz
>
> Am 26.04.12 17:32, schrieb basZero:
>
> Hi,
>
>  I have noticed very poor performance (over 5 seconds) when using
> group = YarepGroupManager.getGroup(groupid)
>
>  and above all
> group.addMember(userObject) (this call is very slow!)
>
>  when having thousands of users.
>
>  1) YarepGroupManager.getGroup(groupid)
>
>  When looking at the code of getGroup(), it loops over ALL users and
> verifies whether the user really exists and only adds it when it exists.
>
>  From a performane point of view, I don't want the Yanel to verify the
> existance of 10'000 XMLs.
> Is there a way to speed this up? I would prefer the usage of JAXB (just
> converting XML to Java or the other way round). Then, if you really need
> verification of some items, you can loop over all items and do your logic.
>
>  2) group.addMember(userObject)
> on my local laptop, this method took 5 seconds alone with 1000 users.
>
>  the code mainly does:
> - first add the group to the user XML (which should be quick, did not
> measure it yet)
> - secondly the user id gets added to the group XML: here also
> createConfiguration() gets called which loops over all items, I think this
> is just too slow when not using a marshalling engine like JAXB.
>
>  What is your experience?
>
>
> I have the same experience, whereas it depends on how you use these
> classes. Also these were not built
> for performance, but for functionality. Whereas this shall be no excuse,
> but just an explanation.
>
> I completely agree that we need to improve this with the intention that
> these implementations should scale
> for millions of users and groups.
>
>  What can I do?
>
>
> Do your own implementation of the group manager and configure it inside
> your realm.xml configuration, e.g.
>
> <ac-identities
> class="com.zwischengas.wyona.security.impl.IdentityManagerFactoryImpl"
> xmlns:zg="http://www.zwischengas.com/wyona/security1.0.0"<http://www.zwischengas.com/wyona/security1.0.0>
> >
> ...
>
> whereas in order to save time you might want to derive all the classes
> from Yarep which work and just overwrite
> the implementations which don't scale well.
>
> Even better, you create a "pull branch" such that you and I can work
> together and I will be happy to help you on this,
> because I also need such an implementation shortly.
>
> We could call it org/wyona/security/impl/yarep2
>
> or whatever
>
> WDYT?
>
> Thanks
>
> Michael
>
>
>  Cheers
>  Balz
>
>
>
>
> --
> Yanel-development mailing list Yanel-development at wyona.com
> http://lists.wyona.org/cgi-bin/mailman/listinfo/yanel-development
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wyona.org/pipermail/yanel-development/attachments/20120427/f038d2d9/attachment.html>


More information about the Yanel-development mailing list