[Yanel-dev] YarepGroupManager.getGroup() very slow with 5000+ users

[Michael Wechner]

Hi Balz

Am 26.04.12 17:32, schrieb basZero:
> Hi,
>
> I have noticed very poor performance (over 5 seconds) when using
> group = YarepGroupManager.getGroup(groupid)
>
> and above all
> group.addMember(userObject) (this call is very slow!)
>
> when having thousands of users.
>
> 1) YarepGroupManager.getGroup(groupid)
>
> When looking at the code of getGroup(), it loops over ALL users and
> verifies whether the user really exists and only adds it when it exists.
>
> From a performane point of view, I don't want the Yanel to verify the 
> existance of 10'000 XMLs.
> Is there a way to speed this up? I would prefer the usage of JAXB 
> (just converting XML to Java or the other way round). Then, if you 
> really need verification of some items, you can loop over all items 
> and do your logic.
>
> 2) group.addMember(userObject)
> on my local laptop, this method took 5 seconds alone with 1000 users.
>
> the code mainly does:
> - first add the group to the user XML (which should be quick, did not 
> measure it yet)
> - secondly the user id gets added to the group XML: here also 
> createConfiguration() gets called which loops over all items, I think 
> this is just too slow when not using a marshalling engine like JAXB.
>
> What is your experience?

I have the same experience, whereas it depends on how you use these 
classes. Also these were not built
for performance, but for functionality. Whereas this shall be no excuse, 
but just an explanation.

I completely agree that we need to improve this with the intention that 
these implementations should scale
for millions of users and groups.
> What can I do?

Do your own implementation of the group manager and configure it inside 
your realm.xml configuration, e.g.

<ac-identities 
class="com.zwischengas.wyona.security.impl.IdentityManagerFactoryImpl" 
xmlns:zg="http://www.zwischengas.com/wyona/security1.0.0">
...

whereas in order to save time you might want to derive all the classes 
from Yarep which work and just overwrite
the implementations which don't scale well.

Even better, you create a "pull branch" such that you and I can work 
together and I will be happy to help you on this,
because I also need such an implementation shortly.

We could call it org/wyona/security/impl/yarep2

or whatever

WDYT?

Thanks

Michael
>
> Cheers
> Balz
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wyona.org/pipermail/yanel-development/attachments/20120427/2034c878/attachment.html>