FreeMarker XML issues

[Yuh-Ruey Chen]

The XML support in FreeMarker is decent, but it's obvious that it's biased
toward producing XML that would render okay in a browser. However, there are a
couple issues that I've found:

1) For some reason I cannot fathom, when outputting a text node, e.g.
${node.@@text}, it only encodes '<' and '&', and none of the rest. Why this is
okay for browsers, it can cause exceptions for strict XML parsers. And it's hard
to workaround this in client code, since if I try replacing ">" with ">", the
outputter converts it to "&gt;". I have to actually split the text node,
delete the ">", and insert a new entity reference for "gt" between them.

2) ${node.@@text} doesn't recurse over entity references or entities.

3) I could not find a built-in way within a template to convert a string to an
XML fragment. I had to implement my own TemplateMethodModel called "xml" which
could be used like this: "${xml(some_string).blahelement.@@text}".

4) Nitpick: When outputting childless nodes, it uses " />" instead of just "/>".
First, nearly every browser in use out there works just fine with "/>", and
second, if my target isn't a browser, this is completely useless.

5) There isn't an easy way for me to workaround some of these little issues,
since most of them stem from NodeOutputter, which is both package-private and
hardcoded in use by NodeModel, so I can't extend it to fix the issues.

I've barely even started using XML in FreeMarker, so I might find more issues
soon...

As an aside, when looking through the FreeMarker code base, I get the feeling
that either this was designed in the very early ages of Java and is constrained
from being modernized for legacy reasons, or it suffers from a serious case of
"not-invented-here" syndrome.

Sincerely, Yuh-Ruey Chen


------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
FreeMarker-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/freemarker-devel

[Daniel Dekany]

Wednesday, September 9, 2009, 5:07:50 AM, Yuh-Ruey Chen wrote:

> The XML support in FreeMarker is decent, but it's obvious that it's biased
> toward producing XML that would render okay in a browser. However, there are a
> couple issues that I've found:
>
> 1) For some reason I cannot fathom, when outputting a text node, e.g.
> ${node.@@text}, it only encodes '<' and '&', and none of the rest.

There is some kind of misunderstanding there. @@text doesn't return
markup, but plain text. Like, if you have <p>Something <>&</p>
in your XML, p.@@text supposed to be the string "Something <>&". Just
plain text... because that's the data stored in the XML. (But I wonder
what happens in the corner-case if the XML parser you use doesn't
resolve the entity references. I didn't write this stuff, so I don't
know... but it should either resolve the entity references or throw
exception.)

> Why this is okay for browsers, it can cause exceptions for strict
> XML parsers. And it's hard to workaround this in client code, since
> if I try replacing ">" with ">",

That particular replacement you need not do unless you are inside an
attribute value. Anyway... the way it supposed to work:

1. @@text returns plain text, no &...; stuff should be in it. If there
   is such stuff in it, then that's itself the problem that should be
   worked around.

2. Then you escape it however you want. Like if you will insert it into
   another XML, with yourPlainText?xml, but if you insert it into RTF,
   then with yourPlainText?rtf, etc. Or, for mass-escaping use
   #escape.

> the outputter converts it to "&gt;". I have to actually split
> the text node, delete the ">", and insert a new entity reference for
> "gt" between them.
>
> 2) ${node.@@text} doesn't recurse over entity references or entities.

Aha! So, your XML parser doesn't resolve them. Why not? In the 99% of
applications it should (the 1% being the tools that deal with XML
itself, like XML editors). The application that processes XML seldom
supposed (or even has a chance) to do that.

> 3) I could not find a built-in way within a template to convert a string to an
> XML fragment.

That's true... although it's considered to be data-model building
concern, not a presentation concern.

> I had to implement my own TemplateMethodModel called "xml" which
> could be used like this: "${xml(some_string).blahelement.@@text}".
>
> 4) Nitpick: When outputting childless nodes, it uses " />" instead of just "/>".
> First, nearly every browser in use out there works just fine with "/>",

Nowadays maybe... not earlier.

> and second, if my target isn't a browser, this is completely
> useless.

Why? Not only <foo /> is well-formed XML, many (like myself) indeed
prefer to type that space. It's the kind of religious thing like some
write "if(...)" while others write "if (...)", and then fight which
one is the better, but both compiles anyway...

> 5) There isn't an easy way for me to workaround some of these little issues,
> since most of them stem from NodeOutputter, which is both package-private and
> hardcoded in use by NodeModel, so I can't extend it to fix the issues.
>
> I've barely even started using XML in FreeMarker, so I might find more issues
> soon...
>
> As an aside, when looking through the FreeMarker code base, I get the feeling
> that either this was designed in the very early ages of Java

It's J2SE 1.2 compatible...

> and is constrained from being modernized for legacy reasons, or it
> suffers from a serious case of "not-invented-here" syndrome.

What cases of that exactly you have seen? (Be aware that incredibly
advanced functionality, like <drum-roll /> replacing a substring of a
string was only introduced in Java 5. (Even then, if you look into how
it works, you may will find it too heavyweight for some tasks.))

> Sincerely, Yuh-Ruey Chen

--
Best regards,
 Daniel Dekany


------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
FreeMarker-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/freemarker-devel

[Yuh-Ruey Chen]

Thanks for the reply. Responses inline.

2009/9/9 Daniel Dekany <[hidden email]>

Wednesday, September 9, 2009, 5:07:50 AM, Yuh-Ruey Chen wrote:

> The XML support in FreeMarker is decent, but it's obvious that it's biased
> toward producing XML that would render okay in a browser. However, there are a
> couple issues that I've found:
>
> 1) For some reason I cannot fathom, when outputting a text node, e.g.
> ${node.@@text}, it only encodes '<' and '&', and none of the rest.

There is some kind of misunderstanding there. @@text doesn't return
markup, but plain text. Like, if you have <p>Something &lt;>&amp;</p>
in your XML, p.@@text supposed to be the string "Something <>&". Just
plain text... because that's the data stored in the XML. (But I wonder
what happens in the corner-case if the XML parser you use doesn't
resolve the entity references. I didn't write this stuff, so I don't
know... but it should either resolve the entity references or throw
exception.)

Sorry, I meant ${node.@@markup}. Following the source in FreeMarker 2.3.15:

NodeModel.java:266

            if (key.equals("@@nested_markup")) {
                StringBuffer buf = new StringBuffer();
                NodeOutputter nu = new NodeOutputter(node);
                nu.outputContent(node.getChildNodes(), buf);
                return new SimpleScalar(buf.toString());
            }

NodeOutputter.java:247

            case Node.CDATA_SECTION_NODE:
            case Node.TEXT_NODE: {
                buf.append(StringUtil.XMLEncNQG(n.getNodeValue()));
                break;
            }

StringUtil.java:222

    /**
     *  XML encoding without replacing apostrophes and quotation marks and greater-than signs.
     *  @see #XMLEnc(String)
     */
    public static String XMLEncNQG(String s) {

That function only encodes '<' and '&'.

> Why this is okay for browsers, it can cause exceptions for strict
> XML parsers. And it's hard to workaround this in client code, since
> if I try replacing ">" with "&gt;",

That particular replacement you need not do unless you are inside an
attribute value.

I haven't read the XML spec closely, so this might just be a parser bug, but I had the javax.xml.xpath XML parser die on me with a serialized XML string like this: "<formula>x > 3</formula>".
 

> 2) ${node.@@text} doesn't recurse over entity references or entities.

Aha! So, your XML parser doesn't resolve them. Why not? In the 99% of
applications it should (the 1% being the tools that deal with XML
itself, like XML editors). The application that processes XML seldom
supposed (or even has a chance) to do that.

Actually it did resolve them, but I was manually "unresolving" them to get around issue (1) for @@markup.
 

> 3) I could not find a built-in way within a template to convert a string to an
> XML fragment.

That's true... although it's considered to be data-model building
concern, not a presentation concern.

Well FreeMarker does have the capability of treating template markup as data:

[#assign markup]
<breadcrumb>foo</breadcrumb>
<breadcrumb>bar</breadcrumb>
[/#assign]

So it's not that big of a step to want to treat that data as XML:

[#list xml(markup).breadcrumb as breadcrumb]
${breadcrumb.@@nested_markup}
[/#list]
 

> As an aside, when looking through the FreeMarker code base, I get the feeling
> that either this was designed in the very early ages of Java

It's J2SE 1.2 compatible...

> and is constrained from being modernized for legacy reasons, or it
> suffers from a serious case of "not-invented-here" syndrome.

What cases of that exactly you have seen? (Be aware that incredibly
advanced functionality, like <drum-roll /> replacing a substring of a
string was only introduced in Java 5. (Even then, if you look into how
it works, you may will find it too heavyweight for some tasks.))

For example, the mismatch between TemplateModel objects and Java collections. I understand the need for an object wrapper, but I don't get why we need things like TemplateHashModel for Maps or TemplateSequenceModel for Lists - why couldn't the Map or List interfaces be used directly here? It's also doesn't handle Sets well - the default object wrapper wraps it in a TemplateSequenceModel, even though it doesn't have an indexing method, via an internal ArrayList. FreeMarker should've just used the collection interfaces (including Iterator) and throw UnsupportedOperationException on methods that cause mutation.

Also, I had to implement my own ObjectWrapper to get around memory and performance issues involved with copying collections backed by a DB. Copying a collection into an ArrayList defeats the whole purpose of lazily populated collections or proxy collections.

Regards, Yuh-Ruey Chen

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
FreeMarker-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/freemarker-devel

[Daniel Dekany]

Wednesday, September 9, 2009, 10:38:59 PM, YR Chen wrote:

[snip]
> Sorry, I meant ${node.@@markup}. Following the source in FreeMarker 2.3.15:
[snip]
> That function only encodes '<' and '&'.

And since that outputs markup (never attribute values), escaping '>'
is indeed not needed. (Actually, I just realized, there is a bug there
that you didn't run into: it should still escape "]]>"-s.)

[snip]
> I haven't read the XML spec closely, so this might just be a parser
> bug, but I had the javax.xml.xpath XML parser die on me with a
> serialized XML string like this: "<formula>x > 3</formula>".

Huh? What kind of XML parser is that which doesn't respect the basic
rules of XML? Sounds like it's time to trash it...

[snip]
> Well FreeMarker does have the capability of treating template markup as data:
>
> [#assign markup]
> <breadcrumb>foo</breadcrumb>
> <breadcrumb>bar</breadcrumb>
> [/#assign]

In this example it just treated the XML as plain text...

> So it's not that big of a step to want to treat that data as XML:
>
> [#list xml(markup).breadcrumb as breadcrumb]
> ${breadcrumb.@@nested_markup}
> [/#list]

Well, the example before this didn't put me closer to the solution...
anyway, surely this xml(...) thing doable. I wonder if Jonathan
planned to add something like this in 2.4.

[snip]
>>> As an aside, when looking through the FreeMarker code base, I get the feeling
>>> that either this was designed in the very early ages of Java
>>
>> It's J2SE 1.2 compatible...
>>
>>> and is constrained from being modernized for legacy reasons, or it
>>> suffers from a serious case of "not-invented-here" syndrome.
>
>> What cases of that exactly you have seen?
[snip]
>
> For example, the mismatch between TemplateModel objects and Java
> collections. I understand the need for an object wrapper, but I
> don't get why we need things like TemplateHashModel for Maps or
> TemplateSequenceModel for Lists - why couldn't the Map or List
> interfaces be used directly here?

I guess because they are way too complex to implement (if you want to
implement a custom wrapper class), they don't require that items are
TemplateModel-s (sure, you could extend the interface with
getTemplateModel or like, but then you again had to wrap HashMap-s and
like, so...), and they don't support multi-type values (like in
FreeMarker you can have a value that is a hash and a sequence and a
string at once; this is something the XML wrapper happens to use
extensively). (Actually, the List and Map interfaces are not
compatible, because the size() method of them clashes, so you couldn't
even have a list+map type, which is really a loss... it's so often
useful.) (Not to mention, that as an FM 1.x legacy, hashes only accept
String keys... because, back then they only meant to be collections of
variables, and variable names are strings.)

Anyway, I often wonder wether the whole intrusive-object-wrapping
business bought as much advantage than it caused headache... Surely
Attila or someone else will stand up pointing to powerful features
that wouldn't be possible without it, but then maybe X users use
those, and 10*X users suffer because of the difficulties in the more
common scenarios... (Also I'm not at all sure there is not another
way, like instead of replacing the Java type-system extending it.
Somehow... um. Anyway, most certainly such things won't happen in
FreeMarker, due to backward-compatibility... maybe in a next-gen
template engine we make.)

> It's also doesn't handle Sets well - the default object wrapper
> wraps it in a TemplateSequenceModel, even though it doesn't have an
> indexing method, via an internal ArrayList. FreeMarker should've
> just used the collection interfaces (including Iterator) and throw
> UnsupportedOperationException on methods that cause mutation.

There the rationale was, I guess, that the template author is not
supposed to know the difference between a Set and a List... and the
template should not work or fail due to changes in the data-model like
that. Actually, FTL doesn't even have sets in its type system. It's
like it doesn't have double, int, long, etc., just number. It has a
simpler type system than Java has.

> Also, I had to implement my own ObjectWrapper to get around memory
> and performance issues involved with copying collections backed by a
> DB. Copying a collection into an ArrayList defeats the whole purpose
> of lazily populated collections or proxy collections.

What wrapper are you using? Because unfortunately,
DefaultObjectWrapper has some problems... it's there mostly because
it's backward-compatible. I'm certain plain BeansWrapper wouldn't
screw that data-base thing you are talking about. Whenever I used
FreeMarker, among the first things used to be "wrapper = new
BeansWrapper(); wrapper.setSimpleMapWrapper(true);
config.setObjectWrapper(wrapper);".

> Regards, Yuh-Ruey Chen

--
Best regards,
 Daniel Dekany


------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
FreeMarker-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/freemarker-devel

[Attila Szegedi-3]

On 2009.09.09., at 22:38, YR Chen wrote:

> Also, I had to implement my own ObjectWrapper to get around memory  
> and performance issues involved with copying collections backed by a  
> DB. Copying a collection into an ArrayList defeats the whole purpose  
> of lazily populated collections or proxy collections.

Use BeansWrapper. It uses no-copying lightweight adapters for  
underlying Java objects.

Attila.

--
twitter: http://twitter.com/szegedi
weblog: http://constc.blogspot.com






------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
FreeMarker-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/freemarker-devel

[Attila Szegedi-3]

On 2009.09.10., at 10:28, Daniel Dekany wrote:

> Wednesday, September 9, 2009, 10:38:59 PM, YR Chen wrote:
>
>
>>
>> For example, the mismatch between TemplateModel objects and Java
>> collections. I understand the need for an object wrapper, but I
>> don't get why we need things like TemplateHashModel for Maps or
>> TemplateSequenceModel for Lists - why couldn't the Map or List
>> interfaces be used directly here?
>
> I guess because they are way too complex to implement (if you want to
> implement a custom wrapper class), they don't require that items are
> TemplateModel-s (sure, you could extend the interface with
> getTemplateModel or like, but then you again had to wrap HashMap-s and
> like, so...), and they don't support multi-type values (like in
> FreeMarker you can have a value that is a hash and a sequence and a
> string at once; this is something the XML wrapper happens to use
> extensively). (Actually, the List and Map interfaces are not
> compatible, because the size() method of them clashes, so you couldn't
> even have a list+map type, which is really a loss... it's so often
> useful.) (Not to mention, that as an FM 1.x legacy, hashes only accept
> String keys... because, back then they only meant to be collections of
> variables, and variable names are strings.)
>
> Anyway, I often wonder wether the whole intrusive-object-wrapping
> business bought as much advantage than it caused headache... Surely
> Attila or someone else will stand up pointing to powerful features
> that wouldn't be possible without it, but then maybe X users use
> those, and 10*X users suffer because of the difficulties in the more
> common scenarios... (Also I'm not at all sure there is not another
> way, like instead of replacing the Java type-system extending it.
> Somehow... um. Anyway, most certainly such things won't happen in
> FreeMarker, due to backward-compatibility... maybe in a next-gen
> template engine we make.)

The TemplateXxxModel business carries over from FreeMarker 1.x which  
existed before Java 2 and its collection classes. That said, *my*  
longer term plan is to replace wrappers with metaobject protocols  
anyway. I won't defend the current design.

Attila.

--
twitter: http://twitter.com/szegedi
weblog: http://constc.blogspot.com

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
FreeMarker-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/freemarker-devel

[Daniel Dekany]

Thursday, September 10, 2009, 10:45:59 AM, Attila Szegedi wrote:

> On 2009.09.10., at 10:28, Daniel Dekany wrote:
>
>> Wednesday, September 9, 2009, 10:38:59 PM, YR Chen wrote:
>>
>>
>>>
>>> For example, the mismatch between TemplateModel objects and Java
>>> collections. I understand the need for an object wrapper, but I
>>> don't get why we need things like TemplateHashModel for Maps or
>>> TemplateSequenceModel for Lists - why couldn't the Map or List
>>> interfaces be used directly here?
>>
>> I guess because they are way too complex to implement (if you want to
>> implement a custom wrapper class), they don't require that items are
>> TemplateModel-s (sure, you could extend the interface with
>> getTemplateModel or like, but then you again had to wrap HashMap-s and
>> like, so...), and they don't support multi-type values (like in
>> FreeMarker you can have a value that is a hash and a sequence and a
>> string at once; this is something the XML wrapper happens to use
>> extensively). (Actually, the List and Map interfaces are not
>> compatible, because the size() method of them clashes, so you couldn't
>> even have a list+map type, which is really a loss... it's so often
>> useful.) (Not to mention, that as an FM 1.x legacy, hashes only accept
>> String keys... because, back then they only meant to be collections of
>> variables, and variable names are strings.)
>>
>> Anyway, I often wonder wether the whole intrusive-object-wrapping
>> business bought as much advantage than it caused headache... Surely
>> Attila or someone else will stand up pointing to powerful features
>> that wouldn't be possible without it, but then maybe X users use
>> those, and 10*X users suffer because of the difficulties in the more
>> common scenarios... (Also I'm not at all sure there is not another
>> way, like instead of replacing the Java type-system extending it.
>> Somehow... um. Anyway, most certainly such things won't happen in
>> FreeMarker, due to backward-compatibility... maybe in a next-gen
>> template engine we make.)
>
> The TemplateXxxModel business carries over from FreeMarker 1.x which  
> existed before Java 2 and its collection classes.

Sure, but it's not like we attempted to deprecated them when 1.2
become a requirement... (Or at least I'm not aware of such efforts.)

> That said, *my* longer term plan is to replace wrappers with
> metaobject protocols anyway.

You mean, in FM? So with how much backward compatibility impact you
think that's practically doable?

> I won't defend the current design.
>
> Attila.

--
Best regards,
 Daniel Dekany


------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
FreeMarker-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/freemarker-devel

[Yuh-Ruey Chen]

Replies inline.

On Thu, Sep 10, 2009 at 1:28 AM, Daniel Dekany <[hidden email]> wrote:

Wednesday, September 9, 2009, 10:38:59 PM, YR Chen wrote:

[snip]

> Sorry, I meant ${node.@@markup}. Following the source in FreeMarker 2.3.15:

[snip]

> That function only encodes '<' and '&'.

And since that outputs markup (never attribute values), escaping '>'
is indeed not needed. (Actually, I just realized, there is a bug there
that you didn't run into: it should still escape "]]>"-s.)

[snip]

> I haven't read the XML spec closely, so this might just be a parser
> bug, but I had the javax.xml.xpath XML parser die on me with a
> serialized XML string like this: "<formula>x > 3</formula>".

Huh? What kind of XML parser is that which doesn't respect the basic
rules of XML? Sounds like it's time to trash it...

Well either that parser was buggy, or there was a mistake somewhere on my end :p I did check the XML grammar, and you're right:

[14]

CharData

::=

[^<&]* - ([^<&]* ']]>' [^<&]*)

 

[snip]

> Well FreeMarker does have the capability of treating template markup as data:
>
> [#assign markup]
> <breadcrumb>foo</breadcrumb>
> <breadcrumb>bar</breadcrumb>
> [/#assign]

In this example it just treated the XML as plain text...

> So it's not that big of a step to want to treat that data as XML:
>
> [#list xml(markup).breadcrumb as breadcrumb]
> ${breadcrumb.@@nested_markup}
> [/#list]

Well, the example before this didn't put me closer to the solution...
anyway, surely this xml(...) thing doable. I wonder if Jonathan
planned to add something like this in 2.4.

Good to know :)
 

> Also, I had to implement my own ObjectWrapper to get around memory
> and performance issues involved with copying collections backed by a
> DB. Copying a collection into an ArrayList defeats the whole purpose
> of lazily populated collections or proxy collections.

What wrapper are you using? Because unfortunately,
DefaultObjectWrapper has some problems... it's there mostly because
it's backward-compatible. I'm certain plain BeansWrapper wouldn't
screw that data-base thing you are talking about. Whenever I used
FreeMarker, among the first things used to be "wrapper = new
BeansWrapper(); wrapper.setSimpleMapWrapper(true);
config.setObjectWrapper(wrapper);".

Hmm, I'll give that a try. I did implement a couple extra things like wrapping an Enum class into a Hash. Since I believe Enums are a Java 5 feature, it might not be possible to add the FreeMarker code base.

Regards, Yuh-Ruey Chen

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
FreeMarker-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/freemarker-devel

[Yuh-Ruey Chen]

2009/9/10 Daniel Dekany <[hidden email]>

Thursday, September 10, 2009, 10:45:59 AM, Attila Szegedi wrote:
> The TemplateXxxModel business carries over from FreeMarker 1.x which
> existed before Java 2 and its collection classes.

I thought that might be the case.
 

> That said, *my* longer term plan is to replace wrappers with

> metaobject protocols anyway.

You mean, in FM? So with how much backward compatibility impact you
think that's practically doable?

Well a FreeMarker 3.0 would not have to be restricted with backward compatibility worries, as long as it keeps the same "look and feel" of FreeMarker. Though FreeMarker development seems to have slowed down a lot these past few years. Makes me wonder if Java template languages are getting out of vogue or something...

Regards, Yuh-Ruey Chen

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
FreeMarker-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/freemarker-devel

[Attila Szegedi-3]

On 2009.09.11., at 4:10, YR Chen wrote:

> Hmm, I'll give that a try. I did implement a couple extra things  
> like wrapping an Enum class into a Hash. Since I believe Enums are a  
> Java 5 feature, it might not be possible to add the FreeMarker code  
> base.

Well, FreeMarker 2.3.x actually has enum support -- see  
BeansWrapper.getEnumModels().

The thing is, FM 2.3.x is Java 1.2 compatible as a baseline, but we  
actually support many Java 1.3, 1.4, and 1.5 features in it - classes  
for these are compiled with appropriate target and loaded if the  
required JRE level is present; the newer Java you run it on, the more  
features you get :-) Specifically, calling getEnumModels() on a  
pre-1.5 JRE will give you an UnsupportedOperationException, but on  
Java 5 and above it works fine.

> Regards, Yuh-Ruey Chen

Attila.

--
twitter: http://twitter.com/szegedi
weblog: http://constc.blogspot.com






------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
FreeMarker-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/freemarker-devel

[Daniel Dekany]

Friday, September 11, 2009, 4:15:07 AM, YR Chen wrote:

> 2009/9/10 Daniel Dekany <[hidden email]>
>
> Thursday, September 10, 2009, 10:45:59 AM, Attila Szegedi wrote:
>>> The TemplateXxxModel business carries over from FreeMarker 1.x which
>>> existed before Java 2 and its collection classes.
>>
>> I thought that might be the case.

I'm not sure whether the original authors were use the collection
interfaces if they were part of Java back then... but whatever.

>>> That said, *my* longer term plan is to replace wrappers with
>>> metaobject protocols anyway.
>>
>> You mean, in FM? So with how much backward compatibility impact you
>> think that's practically doable?
>>
>> Well a FreeMarker 3.0 would not have to be restricted with backward
>> compatibility worries, as long as it keeps the same "look and feel"
>> of FreeMarker.

I believe that a re-design is sorely needed for FM at this point (that
I already did to some extent). Even if we don't touch the language too
much (but I believe we should), the public API should be totally
redesigned, IMO. However, it's not exactly a good message to release
something called *FreeMarker* 3 being utterly backward-incompatible...
You better change the product name then, and then start to push it as
"a new design from the authors of <OldProduct>" or something... and
ensure customers that the FM line is continued to be supported.

>> Though FreeMarker development seems to have slowed down a lot these
>> past few years. Makes me wonder if Java template languages are
>> getting out of vogue or something...

They should and do become less and less important, at least as far as
application developers are concerned. There was the time when people
started to realize that mixing the Java/PHP/whatever code with HTML is
a bad practice, and the natural step was the introduction of the
template engines like WebMacro and FreeMarker and Velocity. But people
should proceed towards component-oriented stuff... JSF, GWT, Tapestry,
and so on. I'm not saying WebMacro-style template engines become
useless, it's just that they are not the new craze anymore, and they
should be used on less and less places.

> Regards, Yuh-Ruey Chen

--
Best regards,
 Daniel Dekany


------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
FreeMarker-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/freemarker-devel

[Daniel Dekany]

Friday, September 11, 2009, 4:10:28 AM, YR Chen wrote:

> Hmm, I'll give that a try. I did implement a couple extra things
> like wrapping an Enum class into a Hash. Since I believe Enums are a
> Java 5 feature, it might not be possible to add the FreeMarker code base.

Enum support in FreeMarker:
http://freemarker.org/docs/pgui_misc_beanwrapper.html#jdk_15_enums

--
Best regards,
 Daniel Dekany


------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
FreeMarker-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/freemarker-devel

[Attila Szegedi-3]

On 2009.09.11., at 0:28, Daniel Dekany wrote:

> I believe that a re-design is sorely needed for FM at this point (that
> I already did to some extent). Even if we don't touch the language too
> much (but I believe we should), the public API should be totally
> redesigned, IMO. However, it's not exactly a good message to release
> something called *FreeMarker* 3 being utterly backward-incompatible...
> You better change the product name then, and then start to push it as
> "a new design from the authors of <OldProduct>" or something... and
> ensure customers that the FM line is continued to be supported.

Well, I don't know... One of my key takeaways from Jon's experiences  
with JavaCC is that an open source project is really a name - you can  
take the code elsewhere (fork), but you can't take the name. As such,  
a name might have a high brand value (search engine rankings etc.)

So, I'd prefer if we had FreeMarker 3.0.

I mean, hey, if Struts 2 could throw away Struts 1 completely and rely  
on WebWork codebase....

>
>>> Though FreeMarker development seems to have slowed down a lot these
>>> past few years. Makes me wonder if Java template languages are
>>> getting out of vogue or something...
>
> They should and do become less and less important, at least as far as
> application developers are concerned. There was the time when people
> started to realize that mixing the Java/PHP/whatever code with HTML is
> a bad practice, and the natural step was the introduction of the
> template engines like WebMacro and FreeMarker and Velocity. But people
> should proceed towards component-oriented stuff... JSF, GWT, Tapestry,
> and so on. I'm not saying WebMacro-style template engines become
> useless, it's just that they are not the new craze anymore, and they
> should be used on less and less places.

Ah, right, people "should" proceed. Thing is, people will use whatever  
pragmatically works for them with least learning curve and won't start  
use something because of philosophical ideas of somehow "better" design.
I wouldn't. Also, I'm honestly convinced that the design principle of  
a template engine is sound, even of our "dirty" "class 5" engines  
(short explanation: prof. Terence Parr of - among others - ANTLR fame  
has a paper on template engines, and establishes a sort-of purity  
index 1 being purest 5 being dirtiest; he also concludes that he can't  
really imagine any engine falling in 2-4 range - if you go 2, you  
cascade all the way down to 5. Of course, only his StringTemplate  
engine has index 1, every other has 5. I'm actually proud of that.)

Attila.

--
twitter: http://twitter.com/szegedi
weblog: http://constc.blogspot.com






------------------------------------------------------------------------------
Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf
_______________________________________________
FreeMarker-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/freemarker-devel

[Daniel Dekany]

Monday, September 21, 2009, 9:44:38 AM, Attila Szegedi wrote:

> On 2009.09.11., at 0:28, Daniel Dekany wrote:
>
>> I believe that a re-design is sorely needed for FM at this point (that
>> I already did to some extent). Even if we don't touch the language too
>> much (but I believe we should), the public API should be totally
>> redesigned, IMO. However, it's not exactly a good message to release
>> something called *FreeMarker* 3 being utterly backward-incompatible...
>> You better change the product name then, and then start to push it as
>> "a new design from the authors of <OldProduct>" or something... and
>> ensure customers that the FM line is continued to be supported.
>
> Well, I don't know... One of my key takeaways from Jon's experiences  
> with JavaCC is that an open source project is really a name - you can
> take the code elsewhere (fork), but you can't take the name. As such,
> a name might have a high brand value (search engine rankings etc.)
>
> So, I'd prefer if we had FreeMarker 3.0.

[snip]

Whatever... this actually doesn't mater, until I or someone else
indeed releases something redesigned. Still, it would be funny if
something that has nothing to do with FreeMarker 2.x, other than that
it's too an imperative MVC-oriented template engine, is called
FreeMarker 3.0...

--
Best regards,
 Daniel Dekany


------------------------------------------------------------------------------
Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf
_______________________________________________
FreeMarker-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/freemarker-devel

[Klotz, Leigh]

One issue that disturbs me about escaping in FM is that it doesn't know when text is being included into an attribute value, because of the way FT is parsed into text chunks.  If it knew separately about the context, it could properly output special characters.  As it is, there's a great onus on FTL authors to get the escaping rules correct, and as a result, FTL is prone to XSS problems unless carefully managed.

Leigh.

------------------------------------------------------------------------------
Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf
_______________________________________________
FreeMarker-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/freemarker-devel

[Daniel Dekany]

Monday, September 21, 2009, 10:40:46 PM, Klotz, Leigh wrote:

> One issue that disturbs me about escaping in FM is that it doesn't
> know when text is being included into an attribute value, because of
> the way FT is parsed into text chunks.  If it knew separately about
> the context, it could properly output special characters.  As it is,
> there's a great onus on FTL authors to get the escaping rules
> correct, and as a result, FTL is prone to XSS problems unless carefully managed.

I'm aware of the issue, hence HTML-awareness was always in the plan
for a new template engine... Then you can not only escape more
intelligently, but practically ensure well-formed output. Helping the
output being valid is also possible, as the names of at least some
elements and attributes can be checked.

Regarding FM, as far as you are using ?html everywhere (preferably via
<#escape x as x?html>), and you quote attribute values with ",
everything should be fine (outside CDATA sections...). Even if you
don't quote attributes as needed, since <-s will be still encoded, I
don't think the attacker can usually achieve more than upsetting the
page layout a bit... but of course that's undesirable too.

(After all that, I wonder how some template engines get away without
any built-in support for HTML-escaping... /-: Or with one that's much
more verbose than FM's...)

> Leigh.

--
Best regards,
 Daniel Dekany


------------------------------------------------------------------------------
Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf
_______________________________________________
FreeMarker-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/freemarker-devel