[Yanel-dev] separating content from application code & configuration (or getting rid of "app" directory in repository e.g. data/app/)

[Guillaume Déflache]

At the moment in most (all?) applications using Yanel a recurring 
pattern is creating an 'app' directory in realms' data repositories to 
hold all sort of things having nothing to do with content like:
- browser assets (CSS, JS, image, etc. files)
- realm's code (XSLT, Jelly files)
- administrative content (admin doc, help, ...)
- configuration files (I18N, other realm-specific config).

This is an issue because this complicates realm code updates for the 
application/realm administrator:
- as the realms' data repo primarily contains the content, it is in 
principle "owned" by the administrator, not by the developers
- HOWEVER when a new version of the realm code has to be installed, the 
administrator cannot simply points to the new data repo in the realm 
configuration, as it does not contain his current content, so it has to 
copy the newly delivered 'app' dir to its current data repo (which BTW 
can be highly non-trivial if the later is not file-based!!!)
- BUT the app dir also contain some configuration files, e.g. [almost] 
always I18N (which the administrator may have chosen to own), so the 
administrator's original one cannot be blindly overwritten!

So what we have been doing as of now is to keep a manual list for every 
release in some changelog file of all files in the "app" directory that 
needs to be copied to fully update the realm code. And until recently no 
one cared to also note there the files that were no longer needed, and 
whose presence already confused some administrators.


Ideas on how to gradually improve this:

1) use the VCS to get the list of changes in the "app" dir, similar to 
svn log -rPREVIOUS_RELEASE:NEXT_RELEASE data/app/ probably: this would 
relieve us of the burden to think about updating the changelog at each 
commit and checking and updating it manually just before each release
- this also gives use the deleted files
- if commits are functional ones and reference a bug number as they 
should, the intent of the addition or removal of these files should be 
clear, and anyway I still think administrators should not bother at 
least for non-configuration files

2) move configuration stuff (incl. I18N) out of the "app" dir, for 
instead into a "conf" dir: this would allow the "app"-dir-copy solution

3) move code-related stuff out of the data-repo: this would allow the 
administrator to have almost nothing to do on realm upgrades, except 
checking for changes in configuration files, of which there should be a 
known and short list: for this we should
3.1) use the 'rtyanelhdocs' protocol to move code-related files to the 
only resource-type they most of the time belong to
3.2) and as some are truly global to the realm (e.g. global.xsl) we 
should introduce a new 'realmhtdocs' protocol(*) similar to the 
'rthdocs' one, which would also need to be proxyied thru HTTP under 
<realm-URL>/<Yanel-reserved-prefix>/realm/... for access by browsers I guess

4) move configuration stuff out of the data-repo: this would allow the 
administrator to have nothing to do on realm upgrades (after initially 
overriding and fine-tuning of the configuration), except upgrading its 
own configuration to use new features (as configuration should be 
backward-compatible): for this we should
4.1) allow to configure custom repositories (this notion already exists 
in the Realm API, see 
<http://yanel.org/javadoc/org/wyona/yanel/core/map/Realm.html#getRepository(java.lang.String)> 
and the 'yanelrepo' protocol seems to support it, but AFAICR there are 
still some minor implementation issues)
4.2) add a new custom repository for configuration(*) and use it!

(*): Or we could at least use a dedicated RC file to catch access to all 
these files at the URL level


IMHO:
1) could be followed immediately.
2) is only a relatively short refactoring (say 2-4 hours per realm maybe 
depending on the number of configuration files), but this needs 
communication with the application administrators and their agreement if 
they have some related written/automated procedures on their end.
I am doing regularly 3.1) as part of ongoing refactoring, it may be 
tricky for custom resource-types if they do not use the Yanel resolver 
though
3.2) should be a one- or two-day task given it's a slight modification 
of existing but tricky code.
4.1) should be a one- or two-day task since some debugging and 
refactoring may also be needed
4.2) is simply configuration work once 4.1) is done.


WDYT?


Cheers,
    Guillaume