[JIRA] Created: (APF-1025) XML and locale properties files should be UTF-8 encoded - but are ansi

9 messages

[JIRA] Created: (APF-1025) XML and locale properties files should be UTF-8 encoded - but are ansi

Reply Threaded

XML and locale properties files should be UTF-8 encoded - but are ansi
----------------------------------------------------------------------

Key: APF-1025
URL: http://issues.appfuse.org/browse/APF-1025
Project: AppFuse
Issue Type: Bug
Components: Installation
Environment: de_DE
Reporter: Kropp, Henning
Assignee: Matt Raible
Priority: Minor

XML files and local properties files need to be utf-8 encoded but they are ansi encoded. If not you experience problems with special characters in those files.

For one the locale property files get all messed up after a appfuse:full-source.

And the ansi encoding makes it impossible to add special characters to the sample-data.xml for example writing

<value description="first_name">Mätt</value>

in the sample-data.xml throws the following error during build:

[INFO] ------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Error executing database operation: CLEAN_INSERT

Embedded error: java.io.UTFDataFormatException: Invalid byte 2 of 3-byte UTF-8 s
equence.
[INFO] ------------------------------------------------------------------------

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.appfuse.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

AppFuse - Issues mailing list

[JIRA] Updated: (APF-1025) XML and locale properties files should be UTF-8 encoded - but are ansi

Reply Threaded

[ http://issues.appfuse.org/browse/APF-1025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Matt Raible updated APF-1025:
-----------------------------

Affects Version/s: 2.0.1
Fix Version/s: 2.0.2

The i18n files should be UTF8-encoded, but I haven't checked all the XML files.

> XML and locale properties files should be UTF-8 encoded - but are ansi
> ----------------------------------------------------------------------
>
> Key: APF-1025
> URL: http://issues.appfuse.org/browse/APF-1025
> Project: AppFuse
> Issue Type: Bug
> Components: Installation
> Affects Versions: 2.0.1
> Environment: de_DE
> Reporter: Kropp, Henning
> Assignee: Matt Raible
> Priority: Minor
> Fix For: 2.0.2
>
>
> XML files and local properties files need to be utf-8 encoded but they are ansi encoded. If not you experience problems with special characters in those files.
> For one the locale property files get all messed up after a appfuse:full-source.
> And the ansi encoding makes it impossible to add special characters to the sample-data.xml for example writing
> <value description="first_name">Mätt</value>
> in the sample-data.xml throws the following error during build:
> [INFO] ------------------------------------------------------------------------
> [ERROR] BUILD ERROR
> [INFO] ------------------------------------------------------------------------
> [INFO] Error executing database operation: CLEAN_INSERT
> Embedded error: java.io.UTFDataFormatException: Invalid byte 2 of 3-byte UTF-8 s
> equence.
> [INFO] ------------------------------------------------------------------------

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.appfuse.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

AppFuse - Issues mailing list

[JIRA] Commented: (APF-1025) XML and locale properties files should be UTF-8 encoded - but are ansi

Reply Threaded

In reply to this post by AppFuse - Issues mailing list

[ http://issues.appfuse.org/browse/APF-1025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12342 ]

Kropp, Henning commented on APF-1025:
-------------------------------------

I checked and ApplicationResources_pt.properties is the only i18n file encoded in utf-8 and the only one still intact after appfuse:full-source.

AppFuse - Issues mailing list

[JIRA] Commented: (APF-1025) XML and locale properties files should be UTF-8 encoded - but are ansi

Reply Threaded

In reply to this post by AppFuse - Issues mailing list

[ http://issues.appfuse.org/browse/APF-1025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12406 ]

Matt Raible commented on APF-1025:
----------------------------------

If you can provide a patch, I can fix this in 2.0.2. If not, I'm going to have to move it to 2.1.

AppFuse - Issues mailing list

[JIRA] Commented: (APF-1025) XML and locale properties files should be UTF-8 encoded - but are ansi

Reply Threaded

In reply to this post by AppFuse - Issues mailing list

[ http://issues.appfuse.org/browse/APF-1025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12413 ]

Matt Raible commented on APF-1025:
----------------------------------

I need a way to reproduce this problem so I can fix it. Is it possible for you to checkout the source, cd into plugins/appfuse-maven-plugin and run one of the full-source tests:

$ ant -p | grep source
test-basic-source Tests extracting source into basic archetypes
test-core-source tests full-source plugin with core archetype
test-jsf-basic-source tests full-source plugin with jsf basic archetype
test-jsf-modular-source tests full-source plugin with jsf modular archetype
test-modular-source Tests extracting source into modular archetypes
test-spring-basic-source tests full-source plugin with spring basic archetype
test-spring-modular-source tests full-source plugin with spring modular archetype
test-struts-basic-source tests full-source plugin with struts basic archetype
test-struts-modular-source tests full-source plugin with struts modular archetype
test-tapestry-basic-source tests full-source plugin with tapestry basic archetype
test-tapestry-modular-source tests full-source plugin with tapestry modular archetype

If I had a way to verify file encoding from the command line or TextMate, I might be able to fix this. However, I don't want to check in a fix when I can't see the before and after difference in results.

My hunch is I can add System.setProperty("file.encoding", "UTF-8") to the execute() method in InstallSourceMojo.java, but I'm not sure.

> XML and locale properties files should be UTF-8 encoded - but are ansi
> ----------------------------------------------------------------------
>
> Key: APF-1025
> URL: http://issues.appfuse.org/browse/APF-1025
> Project: AppFuse
> Issue Type: Bug
> Components: Installation
> Affects Versions: 2.0.1
> Environment: de_DE
> Reporter: Kropp, Henning
> Assignee: Matt Raible
> Priority: Minor
> Fix For: 2.1
>
>
> XML files and local properties files need to be utf-8 encoded but they are ansi encoded. If not you experience problems with special characters in those files.
> For one the locale property files get all messed up after a appfuse:full-source.
> And the ansi encoding makes it impossible to add special characters to the sample-data.xml for example writing
> <value description="first_name">Mätt</value>
> in the sample-data.xml throws the following error during build:
> [INFO] ------------------------------------------------------------------------
> [ERROR] BUILD ERROR
> [INFO] ------------------------------------------------------------------------
> [INFO] Error executing database operation: CLEAN_INSERT
> Embedded error: java.io.UTFDataFormatException: Invalid byte 2 of 3-byte UTF-8 s
> equence.
> [INFO] ------------------------------------------------------------------------

AppFuse - Issues mailing list

[JIRA] Updated: (APF-1025) XML and locale properties files should be UTF-8 encoded - but are ansi

Reply Threaded

In reply to this post by AppFuse - Issues mailing list

> XML and locale properties files should be UTF-8 encoded - but are ansi
> ----------------------------------------------------------------------
>
> Key: APF-1025
> URL: http://issues.appfuse.org/browse/APF-1025
> Project: AppFuse
> Issue Type: Bug
> Components: Installation
> Affects Versions: 2.0.1
> Environment: de_DE
> Reporter: Kropp, Henning
> Assignee: Matt Raible
> Priority: Minor
> Fix For: 2.1
>
>
> XML files and local properties files need to be utf-8 encoded but they are ansi encoded. If not you experience problems with special characters in those files.
> For one the locale property files get all messed up after a appfuse:full-source.
> And the ansi encoding makes it impossible to add special characters to the sample-data.xml for example writing
> <value description="first_name">Mätt</value>
> in the sample-data.xml throws the following error during build:
> [INFO] ------------------------------------------------------------------------
> [ERROR] BUILD ERROR
> [INFO] ------------------------------------------------------------------------
> [INFO] Error executing database operation: CLEAN_INSERT
> Embedded error: java.io.UTFDataFormatException: Invalid byte 2 of 3-byte UTF-8 s
> equence.
> [INFO] ------------------------------------------------------------------------

AppFuse - Issues mailing list

[JIRA] Commented: (APF-1025) XML and locale properties files should be UTF-8 encoded - but are ansi

Reply Threaded

In reply to this post by AppFuse - Issues mailing list

[ http://issues.appfuse.org/browse/APF-1025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12441 ]

Kropp, Henning commented on APF-1025:
-------------------------------------

Will do till 05-17-08. Report back than.

> XML and locale properties files should be UTF-8 encoded - but are ansi
> ----------------------------------------------------------------------
>
> Key: APF-1025
> URL: http://issues.appfuse.org/browse/APF-1025
> Project: AppFuse
> Issue Type: Bug
> Components: Installation
> Affects Versions: 2.0.1
> Environment: de_DE
> Reporter: Kropp, Henning
> Assignee: Matt Raible
> Priority: Minor
> Fix For: 2.1
>
>
> XML files and local properties files need to be utf-8 encoded but they are ansi encoded. If not you experience problems with special characters in those files.
> For one the locale property files get all messed up after a appfuse:full-source.
> And the ansi encoding makes it impossible to add special characters to the sample-data.xml for example writing
> <value description="first_name">Mätt</value>
> in the sample-data.xml throws the following error during build:
> [INFO] ------------------------------------------------------------------------
> [ERROR] BUILD ERROR
> [INFO] ------------------------------------------------------------------------
> [INFO] Error executing database operation: CLEAN_INSERT
> Embedded error: java.io.UTFDataFormatException: Invalid byte 2 of 3-byte UTF-8 s
> equence.
> [INFO] ------------------------------------------------------------------------

AppFuse - Issues mailing list

[JIRA] Commented: (APF-1025) XML and locale properties files should be UTF-8 encoded - but are ansi

Reply Threaded

In reply to this post by AppFuse - Issues mailing list

[ http://issues.appfuse.org/browse/APF-1025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12479 ]

Kropp, Henning commented on APF-1025:
-------------------------------------

I believe TextMate would be perfect for this. I actually use TextPad to find out the file encoding under windows.
If you cant make TextMate show you the properties of the file you are writting as you save the file TextMate shows you the encoding as you can see here:

http://macromates.com/textmate/manual/saving_files

And there is an articel about how to handle UTF-8 here
http://blog.macromates.com/2005/handling-encodings-utf-8/

Unfortunately I have not found a proper terminal%2Fshell tool for this besides iconv which hepls you with the file encoding. Iconv is also mentioned in one of the articels above.

I am also really interested in a command line tool that simply prints out the file encoding.

> XML and locale properties files should be UTF-8 encoded - but are ansi
> ----------------------------------------------------------------------
>
> Key: APF-1025
> URL: http://issues.appfuse.org/browse/APF-1025
> Project: AppFuse
> Issue Type: Bug
> Components: Installation
> Affects Versions: 2.0.1
> Environment: de_DE
> Reporter: Kropp, Henning
> Assignee: Matt Raible
> Priority: Minor
> Fix For: 2.1
>
>
> XML files and local properties files need to be utf-8 encoded but they are ansi encoded. If not you experience problems with special characters in those files.
> For one the locale property files get all messed up after a appfuse:full-source.
> And the ansi encoding makes it impossible to add special characters to the sample-data.xml for example writing
> <value description="first_name">Mätt</value>
> in the sample-data.xml throws the following error during build:
> [INFO] ------------------------------------------------------------------------
> [ERROR] BUILD ERROR
> [INFO] ------------------------------------------------------------------------
> [INFO] Error executing database operation: CLEAN_INSERT
> Embedded error: java.io.UTFDataFormatException: Invalid byte 2 of 3-byte UTF-8 s
> equence.
> [INFO] ------------------------------------------------------------------------

AppFuse - Issues mailing list

[JIRA] Updated: (APF-1025) XML and locale properties files should be UTF-8 encoded - but are ansi

Reply Threaded

In reply to this post by AppFuse - Issues mailing list

> XML and locale properties files should be UTF-8 encoded - but are ansi
> ----------------------------------------------------------------------
>
> Key: APF-1025
> URL: http://issues.appfuse.org/browse/APF-1025
> Project: AppFuse
> Issue Type: Bug
> Components: Installation
> Affects Versions: 2.0.1
> Environment: de_DE
> Reporter: Kropp, Henning
> Assignee: Matt Raible
> Priority: Minor
> Fix For: 2.1.0-M2
>
>
> XML files and local properties files need to be utf-8 encoded but they are ansi encoded. If not you experience problems with special characters in those files.
> For one the locale property files get all messed up after a appfuse:full-source.
> And the ansi encoding makes it impossible to add special characters to the sample-data.xml for example writing
> <value description="first_name">Mätt</value>
> in the sample-data.xml throws the following error during build:
> [INFO] ------------------------------------------------------------------------
> [ERROR] BUILD ERROR
> [INFO] ------------------------------------------------------------------------
> [INFO] Error executing database operation: CLEAN_INSERT
> Embedded error: java.io.UTFDataFormatException: Invalid byte 2 of 3-byte UTF-8 s
> equence.
> [INFO] ------------------------------------------------------------------------