[Yanel-dev] YarepGroupManager.getGroup() very slow with 5000+ users
basZero
baszero at gmail.com
Fri Apr 27 09:36:02 CEST 2012
Hi Michael,
sounds like a plan.
First I will try to find a very simple workaround (e.g. by not storing the
member ids anymore in the group XML), as I think that most functionality is
not needing that, am I wrong? I think it is enough just to have the group
definitions in group XML and have the group assignments in the user XML
only.
But to be clear: the above description is not the final solution :)
Yes we can work together on this.
Please also have a look at the pending pull requests which I posted 5
months ago, would be cool to have them finally in so that I can remove all
custom Yanel classes again...
Cheers
Balz
On Fri, Apr 27, 2012 at 9:25 AM, Michael Wechner
<michael.wechner at wyona.com>wrote:
> Hi Balz
>
> Am 26.04.12 17:32, schrieb basZero:
>
> Hi,
>
> I have noticed very poor performance (over 5 seconds) when using
> group = YarepGroupManager.getGroup(groupid)
>
> and above all
> group.addMember(userObject) (this call is very slow!)
>
> when having thousands of users.
>
> 1) YarepGroupManager.getGroup(groupid)
>
> When looking at the code of getGroup(), it loops over ALL users and
> verifies whether the user really exists and only adds it when it exists.
>
> From a performane point of view, I don't want the Yanel to verify the
> existance of 10'000 XMLs.
> Is there a way to speed this up? I would prefer the usage of JAXB (just
> converting XML to Java or the other way round). Then, if you really need
> verification of some items, you can loop over all items and do your logic.
>
> 2) group.addMember(userObject)
> on my local laptop, this method took 5 seconds alone with 1000 users.
>
> the code mainly does:
> - first add the group to the user XML (which should be quick, did not
> measure it yet)
> - secondly the user id gets added to the group XML: here also
> createConfiguration() gets called which loops over all items, I think this
> is just too slow when not using a marshalling engine like JAXB.
>
> What is your experience?
>
>
> I have the same experience, whereas it depends on how you use these
> classes. Also these were not built
> for performance, but for functionality. Whereas this shall be no excuse,
> but just an explanation.
>
> I completely agree that we need to improve this with the intention that
> these implementations should scale
> for millions of users and groups.
>
> What can I do?
>
>
> Do your own implementation of the group manager and configure it inside
> your realm.xml configuration, e.g.
>
> <ac-identities
> class="com.zwischengas.wyona.security.impl.IdentityManagerFactoryImpl"
> xmlns:zg="http://www.zwischengas.com/wyona/security1.0.0"<http://www.zwischengas.com/wyona/security1.0.0>
> >
> ...
>
> whereas in order to save time you might want to derive all the classes
> from Yarep which work and just overwrite
> the implementations which don't scale well.
>
> Even better, you create a "pull branch" such that you and I can work
> together and I will be happy to help you on this,
> because I also need such an implementation shortly.
>
> We could call it org/wyona/security/impl/yarep2
>
> or whatever
>
> WDYT?
>
> Thanks
>
> Michael
>
>
> Cheers
> Balz
>
>
>
>
> --
> Yanel-development mailing list Yanel-development at wyona.com
> http://lists.wyona.org/cgi-bin/mailman/listinfo/yanel-development
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wyona.org/pipermail/yanel-development/attachments/20120427/f038d2d9/attachment.html>
More information about the Yanel-development
mailing list