memory issue with Hudson server version 1.316+

[baz themail]

All,

I upgraded from 1.251 to 1.316 two weeks ago. The server stopped working because of memory usage. Does the version 1.316 use more memory than the 1.251? Any details on this?

This is my current settings:

-Xrs -Xmx1024m -Dhudson.lifecycle=hudson.lifecycle.WindowsServiceLifecycle -jar "%BASE%\hudson.war" --httpPort=80

Followings are the errors:

21-Oct-2009 11:08:41 hudson.triggers.SafeTimerTask run
SEVERE: Timer task hudson.slaves.NodeProvisioner$NodeProvisionerInvoker@11548f3 failed
java.lang.OutOfMemoryError: Java heap space
Could not create the Java virtual machine.
Could not create the Java virtual machine.

Thanks. A.

[baz themail]

Can i set Xmx more than 1024? What is the next resasonable value? 2048?

On Wed, Oct 21, 2009 at 2:17 PM, baz themail <[hidden email]> wrote:

All,

I upgraded from 1.251 to 1.316 two weeks ago. The server stopped working because of memory usage. Does the version 1.316 use more memory than the 1.251? Any details on this?

This is my current settings:

-Xrs -Xmx1024m -Dhudson.lifecycle=hudson.lifecycle.WindowsServiceLifecycle -jar "%BASE%\hudson.war" --httpPort=80

Followings are the errors:

21-Oct-2009 11:08:41 hudson.triggers.SafeTimerTask run
SEVERE: Timer task hudson.slaves.NodeProvisioner$NodeProvisionerInvoker@11548f3 failed
java.lang.OutOfMemoryError: Java heap space
Could not create the Java virtual machine.
Could not create the Java virtual machine.

Thanks. A.

[baz themail]

When i set to 2048, then the error is "Could not create the Java virtual machine.". So i wonder what should i set above 1024? Is there a limit or convention?

On Wed, Oct 28, 2009 at 7:52 PM, baz themail <[hidden email]> wrote:

Can i set Xmx more than 1024? What is the next resasonable value? 2048?

On Wed, Oct 21, 2009 at 2:17 PM, baz themail <[hidden email]> wrote:

All,

I upgraded from 1.251 to 1.316 two weeks ago. The server stopped working because of memory usage. Does the version 1.316 use more memory than the 1.251? Any details on this?

This is my current settings:

-Xrs -Xmx1024m -Dhudson.lifecycle=hudson.lifecycle.WindowsServiceLifecycle -jar "%BASE%\hudson.war" --httpPort=80

Followings are the errors:

21-Oct-2009 11:08:41 hudson.triggers.SafeTimerTask run
SEVERE: Timer task hudson.slaves.NodeProvisioner$NodeProvisionerInvoker@11548f3 failed
java.lang.OutOfMemoryError: Java heap space
Could not create the Java virtual machine.
Could not create the Java virtual machine.

Thanks. A.

[Joshua McKinnon]

You should be able to set any value in MB that you want, until you exceed the limit for a 32-bit process (just guessing, since 2048 did not work for you)

i.e. -Xmx1100m, -Xmx1150m, -Xmx1234m should all be legal.


From memory, on windows anyway, Java usually tops out by 1400-1500m max heap size as a 32-bit process, due to other memory usage outside of the heap itself adding up to the 2 gig limit imposed per process in 32-bit windows. I'm not sure what OS you're on, I'm just going by personal experience.

That said - I am also interested in how much memory usage may have increased in Hudson (or more likely, certain plugins I use). I currently run 1.316 and previously was on 1.280ish and did see a jump as well. In my case, I upped my heap for Hudson from 500m to 800m...


Joshua

On Tue, Nov 3, 2009 at 3:39 PM, baz themail <[hidden email]> wrote:

When i set to 2048, then the error is "Could not create the Java virtual machine.". So i wonder what should i set above 1024? Is there a limit or convention?

On Wed, Oct 28, 2009 at 7:52 PM, baz themail <[hidden email]> wrote:

Can i set Xmx more than 1024? What is the next resasonable value? 2048?

On Wed, Oct 21, 2009 at 2:17 PM, baz themail <[hidden email]> wrote:

All,

I upgraded from 1.251 to 1.316 two weeks ago. The server stopped working because of memory usage. Does the version 1.316 use more memory than the 1.251? Any details on this?

This is my current settings:

-Xrs -Xmx1024m -Dhudson.lifecycle=hudson.lifecycle.WindowsServiceLifecycle -jar "%BASE%\hudson.war" --httpPort=80

Followings are the errors:

21-Oct-2009 11:08:41 hudson.triggers.SafeTimerTask run
SEVERE: Timer task hudson.slaves.NodeProvisioner$NodeProvisionerInvoker@11548f3 failed
java.lang.OutOfMemoryError: Java heap space
Could not create the Java virtual machine.
Could not create the Java virtual machine.

Thanks. A.

[baz themail]

The value of -Xmx1234m seems to work. Thank you so much.

Just curious, how can i determine on the valid values? Is 1234 the max within 32-bit?

On Tue, Nov 3, 2009 at 2:22 PM, Joshua McKinnon <[hidden email]> wrote:

You should be able to set any value in MB that you want, until you exceed the limit for a 32-bit process (just guessing, since 2048 did not work for you)

i.e. -Xmx1100m, -Xmx1150m, -Xmx1234m should all be legal.


From memory, on windows anyway, Java usually tops out by 1400-1500m max heap size as a 32-bit process, due to other memory usage outside of the heap itself adding up to the 2 gig limit imposed per process in 32-bit windows. I'm not sure what OS you're on, I'm just going by personal experience.

That said - I am also interested in how much memory usage may have increased in Hudson (or more likely, certain plugins I use). I currently run 1.316 and previously was on 1.280ish and did see a jump as well. In my case, I upped my heap for Hudson from 500m to 800m...


Joshua

On Tue, Nov 3, 2009 at 3:39 PM, baz themail <[hidden email]> wrote:

When i set to 2048, then the error is "Could not create the Java virtual machine.". So i wonder what should i set above 1024? Is there a limit or convention?

On Wed, Oct 28, 2009 at 7:52 PM, baz themail <[hidden email]> wrote:

Can i set Xmx more than 1024? What is the next resasonable value? 2048?

On Wed, Oct 21, 2009 at 2:17 PM, baz themail <[hidden email]> wrote:

All,

I upgraded from 1.251 to 1.316 two weeks ago. The server stopped working because of memory usage. Does the version 1.316 use more memory than the 1.251? Any details on this?

This is my current settings:

-Xrs -Xmx1024m -Dhudson.lifecycle=hudson.lifecycle.WindowsServiceLifecycle -jar "%BASE%\hudson.war" --httpPort=80

Followings are the errors:

21-Oct-2009 11:08:41 hudson.triggers.SafeTimerTask run
SEVERE: Timer task hudson.slaves.NodeProvisioner$NodeProvisionerInvoker@11548f3 failed
java.lang.OutOfMemoryError: Java heap space
Could not create the Java virtual machine.
Could not create the Java virtual machine.

Thanks. A.

[Joshua McKinnon]

1234m is not the max heap value for java on 32-bit windows, it is just an example of a number that isn't an increment of any value in particular.

If 1234m, or 1200m, or some number works - I would stick with it until it doesn't. The max is generally 1400-1500m, but if you set it to that...the next time the memory footprint grows and that is no longer enough, you'll have a problem you can't fix immediately, and your builds will be on hold until you diagnose why memory usage grew, or you go 64-bit. If you set it to something lower, you can try to identify the problem the next time this happens before you run out of address space to use.

The only way I know of to identify the max heap you can set is by trial and error, though like I said a heap settiting in the 1400-1500m range commonly causes the limit to be hit, since the heap is only part of Java's memory usage. If you really want to find out, you can keep adjusting by 50m or so until you hit the limit, or go under the limit.


Joshua

On Tue, Nov 3, 2009 at 4:43 PM, baz themail <[hidden email]> wrote:

The value of -Xmx1234m seems to work. Thank you so much.

Just curious, how can i determine on the valid values? Is 1234 the max within 32-bit?

On Tue, Nov 3, 2009 at 2:22 PM, Joshua McKinnon <[hidden email]> wrote:

You should be able to set any value in MB that you want, until you exceed the limit for a 32-bit process (just guessing, since 2048 did not work for you)

i.e. -Xmx1100m, -Xmx1150m, -Xmx1234m should all be legal.


From memory, on windows anyway, Java usually tops out by 1400-1500m max heap size as a 32-bit process, due to other memory usage outside of the heap itself adding up to the 2 gig limit imposed per process in 32-bit windows. I'm not sure what OS you're on, I'm just going by personal experience.

That said - I am also interested in how much memory usage may have increased in Hudson (or more likely, certain plugins I use). I currently run 1.316 and previously was on 1.280ish and did see a jump as well. In my case, I upped my heap for Hudson from 500m to 800m...


Joshua

On Tue, Nov 3, 2009 at 3:39 PM, baz themail <[hidden email]> wrote:

When i set to 2048, then the error is "Could not create the Java virtual machine.". So i wonder what should i set above 1024? Is there a limit or convention?

On Wed, Oct 28, 2009 at 7:52 PM, baz themail <[hidden email]> wrote:

Can i set Xmx more than 1024? What is the next resasonable value? 2048?

On Wed, Oct 21, 2009 at 2:17 PM, baz themail <[hidden email]> wrote:

All,

I upgraded from 1.251 to 1.316 two weeks ago. The server stopped working because of memory usage. Does the version 1.316 use more memory than the 1.251? Any details on this?

This is my current settings:

-Xrs -Xmx1024m -Dhudson.lifecycle=hudson.lifecycle.WindowsServiceLifecycle -jar "%BASE%\hudson.war" --httpPort=80

Followings are the errors:

21-Oct-2009 11:08:41 hudson.triggers.SafeTimerTask run
SEVERE: Timer task hudson.slaves.NodeProvisioner$NodeProvisionerInvoker@11548f3 failed
java.lang.OutOfMemoryError: Java heap space
Could not create the Java virtual machine.
Could not create the Java virtual machine.

Thanks. A.

[baz themail]

Great info. Thanks Joshua.

On Tue, Nov 3, 2009 at 3:53 PM, Joshua McKinnon <[hidden email]> wrote:

1234m is not the max heap value for java on 32-bit windows, it is just an example of a number that isn't an increment of any value in particular.

If 1234m, or 1200m, or some number works - I would stick with it until it doesn't. The max is generally 1400-1500m, but if you set it to that...the next time the memory footprint grows and that is no longer enough, you'll have a problem you can't fix immediately, and your builds will be on hold until you diagnose why memory usage grew, or you go 64-bit. If you set it to something lower, you can try to identify the problem the next time this happens before you run out of address space to use.

The only way I know of to identify the max heap you can set is by trial and error, though like I said a heap settiting in the 1400-1500m range commonly causes the limit to be hit, since the heap is only part of Java's memory usage. If you really want to find out, you can keep adjusting by 50m or so until you hit the limit, or go under the limit.


Joshua

On Tue, Nov 3, 2009 at 4:43 PM, baz themail <[hidden email]> wrote:

The value of -Xmx1234m seems to work. Thank you so much.

Just curious, how can i determine on the valid values? Is 1234 the max within 32-bit?

On Tue, Nov 3, 2009 at 2:22 PM, Joshua McKinnon <[hidden email]> wrote:

You should be able to set any value in MB that you want, until you exceed the limit for a 32-bit process (just guessing, since 2048 did not work for you)

i.e. -Xmx1100m, -Xmx1150m, -Xmx1234m should all be legal.


From memory, on windows anyway, Java usually tops out by 1400-1500m max heap size as a 32-bit process, due to other memory usage outside of the heap itself adding up to the 2 gig limit imposed per process in 32-bit windows. I'm not sure what OS you're on, I'm just going by personal experience.

That said - I am also interested in how much memory usage may have increased in Hudson (or more likely, certain plugins I use). I currently run 1.316 and previously was on 1.280ish and did see a jump as well. In my case, I upped my heap for Hudson from 500m to 800m...


Joshua

On Tue, Nov 3, 2009 at 3:39 PM, baz themail <[hidden email]> wrote:

When i set to 2048, then the error is "Could not create the Java virtual machine.". So i wonder what should i set above 1024? Is there a limit or convention?

On Wed, Oct 28, 2009 at 7:52 PM, baz themail <[hidden email]> wrote:

Can i set Xmx more than 1024? What is the next resasonable value? 2048?

On Wed, Oct 21, 2009 at 2:17 PM, baz themail <[hidden email]> wrote:

All,

I upgraded from 1.251 to 1.316 two weeks ago. The server stopped working because of memory usage. Does the version 1.316 use more memory than the 1.251? Any details on this?

This is my current settings:

-Xrs -Xmx1024m -Dhudson.lifecycle=hudson.lifecycle.WindowsServiceLifecycle -jar "%BASE%\hudson.war" --httpPort=80

Followings are the errors:

21-Oct-2009 11:08:41 hudson.triggers.SafeTimerTask run
SEVERE: Timer task hudson.slaves.NodeProvisioner$NodeProvisionerInvoker@11548f3 failed
java.lang.OutOfMemoryError: Java heap space
Could not create the Java virtual machine.
Could not create the Java virtual machine.

Thanks. A.

[Eric Pyle]

Some javascript/style in this post has been disabled (why?)

We have a Java program (not Hudson) which we have successfully run with –Xmx 1600M on some 32-bit Windows machines with 2 GB RAM. Other machines with the same or more RAM could not allocate more than 1200M or so. A key factor I found is that if you use the Sun JVM, it attempts to allocate the entire heap as a contiguous block of memory. So you may have plenty of free virtual memory but not be able to use all of it. We experimented a bit with the JRockit JVM from Oracle and found it was able to make more memory available – however in the end we did not want to support a second JVM and instead reduced the memory usage of our program. But it may be an option if you are stuck with needing a larger Java heap.

 

-Eric

 

---

From: Joshua McKinnon [mailto:[hidden email]]
Sent: Tuesday, November 03, 2009 5:53 PM
To: [hidden email]
Subject: Re: memory issue with Hudson server version 1.316+

 

1234m is not the max heap value for java on 32-bit windows, it is just an example of a number that isn't an increment of any value in particular.

If 1234m, or 1200m, or some number works - I would stick with it until it doesn't. The max is generally 1400-1500m, but if you set it to that...the next time the memory footprint grows and that is no longer enough, you'll have a problem you can't fix immediately, and your builds will be on hold until you diagnose why memory usage grew, or you go 64-bit. If you set it to something lower, you can try to identify the problem the next time this happens before you run out of address space to use.

The only way I know of to identify the max heap you can set is by trial and error, though like I said a heap settiting in the 1400-1500m range commonly causes the limit to be hit, since the heap is only part of Java's memory usage. If you really want to find out, you can keep adjusting by 50m or so until you hit the limit, or go under the limit.


Joshua

On Tue, Nov 3, 2009 at 4:43 PM, baz themail <[hidden email]> wrote:

The value of -Xmx1234m seems to work. Thank you so much.

Just curious, how can i determine on the valid values? Is 1234 the max within 32-bit?

 

On Tue, Nov 3, 2009 at 2:22 PM, Joshua McKinnon <[hidden email]> wrote:

You should be able to set any value in MB that you want, until you exceed the limit for a 32-bit process (just guessing, since 2048 did not work for you)

i.e. -Xmx1100m, -Xmx1150m, -Xmx1234m should all be legal.


From memory, on windows anyway, Java usually tops out by 1400-1500m max heap size as a 32-bit process, due to other memory usage outside of the heap itself adding up to the 2 gig limit imposed per process in 32-bit windows. I'm not sure what OS you're on, I'm just going by personal experience.

That said - I am also interested in how much memory usage may have increased in Hudson (or more likely, certain plugins I use). I currently run 1.316 and previously was on 1.280ish and did see a jump as well. In my case, I upped my heap for Hudson from 500m to 800m...


Joshua

On Tue, Nov 3, 2009 at 3:39 PM, baz themail <[hidden email]> wrote:

When i set to 2048, then the error is "Could not create the Java virtual machine.". So i wonder what should i set above 1024? Is there a limit or convention?

 

On Wed, Oct 28, 2009 at 7:52 PM, baz themail <[hidden email]> wrote:

Can i set Xmx more than 1024? What is the next resasonable value? 2048?

 

On Wed, Oct 21, 2009 at 2:17 PM, baz themail <[hidden email]> wrote:

All,

I upgraded from 1.251 to 1.316 two weeks ago. The server stopped working because of memory usage. Does the version 1.316 use more memory than the 1.251? Any details on this?

This is my current settings:

-Xrs -Xmx1024m -Dhudson.lifecycle=hudson.lifecycle.WindowsServiceLifecycle -jar "%BASE%\hudson.war" --httpPort=80

Followings are the errors:

21-Oct-2009 11:08:41 hudson.triggers.SafeTimerTask run
SEVERE: Timer task hudson.slaves.NodeProvisioner$NodeProvisionerInvoker@11548f3 failed
java.lang.OutOfMemoryError: Java heap space
Could not create the Java virtual machine.
Could not create the Java virtual machine.

Thanks. A.

 

 

 

 

 

[baz themail]

Eric, thanks for the information on Sun JVM.

I am now restart the Hudson server with "-Xmx 1234M" and see if I am going to see the problem again.

Meanwhile, for "-Xmx 1024M", I saw the following error from hudson error log:

Before the Heap error, I saw the followings, could they be the cause? Anyone can give me some advice? May be memory leak somewhere in version 1.316?

[Winstone 2009/11/05 04:59:35] - Untrapped Error in Servlet
org.apache.commons.jelly.JellyTagException: jar:file:/E:/pl/Hudson_master/war/WEB-INF/lib/hudson-core-1.316.jar!/hudson/widgets/HistoryWidget/entry.jelly:32:174: <st:nbsp> Failed to write to client
    at org.apache.commons.jelly.impl.TagScript.handleException(TagScript.java:694)
    at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:272)
...

javax.servlet.ServletException: jar:file:/E:/pl/Hudson_master/war/WEB-INF/lib/hudson-core-1.316.jar!/hudson/widgets/HistoryWidget/entry.jelly:32:174: <st:nbsp> Failed to write to client
...

[Winstone 2009/11/05 05:09:56] - Untrapped Error in Servlet
java.lang.OutOfMemoryError: Java heap space
    at java.nio.HeapByteBuffer.<init>(Unknown Source)

...

A.

On Wed, Nov 4, 2009 at 6:00 AM, Eric Pyle <[hidden email]> wrote:

We have a Java program (not Hudson) which we have successfully run with –Xmx 1600M on some 32-bit Windows machines with 2 GB RAM. Other machines with the same or more RAM could not allocate more than 1200M or so. A key factor I found is that if you use the Sun JVM, it attempts to allocate the entire heap as a contiguous block of memory. So you may have plenty of free virtual memory but not be able to use all of it. We experimented a bit with the JRockit JVM from Oracle and found it was able to make more memory available – however in the end we did not want to support a second JVM and instead reduced the memory usage of our program. But it may be an option if you are stuck with needing a larger Java heap.

 

-Eric

 

---

From: Joshua McKinnon [mailto:[hidden email]]
Sent: Tuesday, November 03, 2009 5:53 PM
To: [hidden email]
Subject: Re: memory issue with Hudson server version 1.316+

 

1234m is not the max heap value for java on 32-bit windows, it is just an example of a number that isn't an increment of any value in particular.

If 1234m, or 1200m, or some number works - I would stick with it until it doesn't. The max is generally 1400-1500m, but if you set it to that...the next time the memory footprint grows and that is no longer enough, you'll have a problem you can't fix immediately, and your builds will be on hold until you diagnose why memory usage grew, or you go 64-bit. If you set it to something lower, you can try to identify the problem the next time this happens before you run out of address space to use.

The only way I know of to identify the max heap you can set is by trial and error, though like I said a heap settiting in the 1400-1500m range commonly causes the limit to be hit, since the heap is only part of Java's memory usage. If you really want to find out, you can keep adjusting by 50m or so until you hit the limit, or go under the limit.


Joshua

On Tue, Nov 3, 2009 at 4:43 PM, baz themail <[hidden email]> wrote:

The value of -Xmx1234m seems to work. Thank you so much.

Just curious, how can i determine on the valid values? Is 1234 the max within 32-bit?

 

On Tue, Nov 3, 2009 at 2:22 PM, Joshua McKinnon <[hidden email]> wrote:

You should be able to set any value in MB that you want, until you exceed the limit for a 32-bit process (just guessing, since 2048 did not work for you)

i.e. -Xmx1100m, -Xmx1150m, -Xmx1234m should all be legal.


From memory, on windows anyway, Java usually tops out by 1400-1500m max heap size as a 32-bit process, due to other memory usage outside of the heap itself adding up to the 2 gig limit imposed per process in 32-bit windows. I'm not sure what OS you're on, I'm just going by personal experience.

That said - I am also interested in how much memory usage may have increased in Hudson (or more likely, certain plugins I use). I currently run 1.316 and previously was on 1.280ish and did see a jump as well. In my case, I upped my heap for Hudson from 500m to 800m...


Joshua

On Tue, Nov 3, 2009 at 3:39 PM, baz themail <[hidden email]> wrote:

When i set to 2048, then the error is "Could not create the Java virtual machine.". So i wonder what should i set above 1024? Is there a limit or convention?

 

On Wed, Oct 28, 2009 at 7:52 PM, baz themail <[hidden email]> wrote:

Can i set Xmx more than 1024? What is the next resasonable value? 2048?

 

On Wed, Oct 21, 2009 at 2:17 PM, baz themail <[hidden email]> wrote:

All,

I upgraded from 1.251 to 1.316 two weeks ago. The server stopped working because of memory usage. Does the version 1.316 use more memory than the 1.251? Any details on this?

This is my current settings:

-Xrs -Xmx1024m -Dhudson.lifecycle=hudson.lifecycle.WindowsServiceLifecycle -jar "%BASE%\hudson.war" --httpPort=80

Followings are the errors:

21-Oct-2009 11:08:41 hudson.triggers.SafeTimerTask run
SEVERE: Timer task hudson.slaves.NodeProvisioner$NodeProvisionerInvoker@11548f3 failed
java.lang.OutOfMemoryError: Java heap space
Could not create the Java virtual machine.
Could not create the Java virtual machine.

Thanks. A.

 

 

 

 

 

[baz themail]

Is it possible to monitor how much memory the Hudson server is actually using? From TaskManager?

On Thu, Nov 5, 2009 at 11:31 AM, baz themail <[hidden email]> wrote:

Eric, thanks for the information on Sun JVM.

I am now restart the Hudson server with "-Xmx 1234M" and see if I am going to see the problem again.

Meanwhile, for "-Xmx 1024M", I saw the following error from hudson error log:

Before the Heap error, I saw the followings, could they be the cause? Anyone can give me some advice? May be memory leak somewhere in version 1.316?

[Winstone 2009/11/05 04:59:35] - Untrapped Error in Servlet
org.apache.commons.jelly.JellyTagException: jar:file:/E:/pl/Hudson_master/war/WEB-INF/lib/hudson-core-1.316.jar!/hudson/widgets/HistoryWidget/entry.jelly:32:174: <st:nbsp> Failed to write to client
    at org.apache.commons.jelly.impl.TagScript.handleException(TagScript.java:694)
    at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:272)
...

javax.servlet.ServletException: jar:file:/E:/pl/Hudson_master/war/WEB-INF/lib/hudson-core-1.316.jar!/hudson/widgets/HistoryWidget/entry.jelly:32:174: <st:nbsp> Failed to write to client
...

[Winstone 2009/11/05 05:09:56] - Untrapped Error in Servlet

java.lang.OutOfMemoryError: Java heap space

    at java.nio.HeapByteBuffer.<init>(Unknown Source)

...

A.

On Wed, Nov 4, 2009 at 6:00 AM, Eric Pyle <[hidden email]> wrote:

We have a Java program (not Hudson) which we have successfully run with –Xmx 1600M on some 32-bit Windows machines with 2 GB RAM. Other machines with the same or more RAM could not allocate more than 1200M or so. A key factor I found is that if you use the Sun JVM, it attempts to allocate the entire heap as a contiguous block of memory. So you may have plenty of free virtual memory but not be able to use all of it. We experimented a bit with the JRockit JVM from Oracle and found it was able to make more memory available – however in the end we did not want to support a second JVM and instead reduced the memory usage of our program. But it may be an option if you are stuck with needing a larger Java heap.

 

-Eric

 

---

From: Joshua McKinnon [mailto:[hidden email]]
Sent: Tuesday, November 03, 2009 5:53 PM
To: [hidden email]
Subject: Re: memory issue with Hudson server version 1.316+

 

1234m is not the max heap value for java on 32-bit windows, it is just an example of a number that isn't an increment of any value in particular.

If 1234m, or 1200m, or some number works - I would stick with it until it doesn't. The max is generally 1400-1500m, but if you set it to that...the next time the memory footprint grows and that is no longer enough, you'll have a problem you can't fix immediately, and your builds will be on hold until you diagnose why memory usage grew, or you go 64-bit. If you set it to something lower, you can try to identify the problem the next time this happens before you run out of address space to use.

The only way I know of to identify the max heap you can set is by trial and error, though like I said a heap settiting in the 1400-1500m range commonly causes the limit to be hit, since the heap is only part of Java's memory usage. If you really want to find out, you can keep adjusting by 50m or so until you hit the limit, or go under the limit.


Joshua

On Tue, Nov 3, 2009 at 4:43 PM, baz themail <[hidden email]> wrote:

The value of -Xmx1234m seems to work. Thank you so much.

Just curious, how can i determine on the valid values? Is 1234 the max within 32-bit?

 

On Tue, Nov 3, 2009 at 2:22 PM, Joshua McKinnon <[hidden email]> wrote:

You should be able to set any value in MB that you want, until you exceed the limit for a 32-bit process (just guessing, since 2048 did not work for you)

i.e. -Xmx1100m, -Xmx1150m, -Xmx1234m should all be legal.


From memory, on windows anyway, Java usually tops out by 1400-1500m max heap size as a 32-bit process, due to other memory usage outside of the heap itself adding up to the 2 gig limit imposed per process in 32-bit windows. I'm not sure what OS you're on, I'm just going by personal experience.

That said - I am also interested in how much memory usage may have increased in Hudson (or more likely, certain plugins I use). I currently run 1.316 and previously was on 1.280ish and did see a jump as well. In my case, I upped my heap for Hudson from 500m to 800m...


Joshua

On Tue, Nov 3, 2009 at 3:39 PM, baz themail <[hidden email]> wrote:

When i set to 2048, then the error is "Could not create the Java virtual machine.". So i wonder what should i set above 1024? Is there a limit or convention?

 

On Wed, Oct 28, 2009 at 7:52 PM, baz themail <[hidden email]> wrote:

Can i set Xmx more than 1024? What is the next resasonable value? 2048?

 

On Wed, Oct 21, 2009 at 2:17 PM, baz themail <[hidden email]> wrote:

All,

I upgraded from 1.251 to 1.316 two weeks ago. The server stopped working because of memory usage. Does the version 1.316 use more memory than the 1.251? Any details on this?

This is my current settings:

-Xrs -Xmx1024m -Dhudson.lifecycle=hudson.lifecycle.WindowsServiceLifecycle -jar "%BASE%\hudson.war" --httpPort=80

Followings are the errors:

21-Oct-2009 11:08:41 hudson.triggers.SafeTimerTask run
SEVERE: Timer task hudson.slaves.NodeProvisioner$NodeProvisionerInvoker@11548f3 failed
java.lang.OutOfMemoryError: Java heap space
Could not create the Java virtual machine.
Could not create the Java virtual machine.

Thanks. A.

 

 

 

 

 

[baz themail]

After increasing to "1234", I still have the memory heap issue but this time, it seems to take more than one week to get there. I have never seen this issue when i was using 1.251. Any idea why I have this issue with 1.316?

On Sat, Nov 7, 2009 at 7:24 PM, baz themail <[hidden email]> wrote:

Is it possible to monitor how much memory the Hudson server is actually using? From TaskManager?

On Thu, Nov 5, 2009 at 11:31 AM, baz themail <[hidden email]> wrote:

Eric, thanks for the information on Sun JVM.

I am now restart the Hudson server with "-Xmx 1234M" and see if I am going to see the problem again.

Meanwhile, for "-Xmx 1024M", I saw the following error from hudson error log:

Before the Heap error, I saw the followings, could they be the cause? Anyone can give me some advice? May be memory leak somewhere in version 1.316?

[Winstone 2009/11/05 04:59:35] - Untrapped Error in Servlet
org.apache.commons.jelly.JellyTagException: jar:file:/E:/pl/Hudson_master/war/WEB-INF/lib/hudson-core-1.316.jar!/hudson/widgets/HistoryWidget/entry.jelly:32:174: <st:nbsp> Failed to write to client
    at org.apache.commons.jelly.impl.TagScript.handleException(TagScript.java:694)
    at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:272)
...

javax.servlet.ServletException: jar:file:/E:/pl/Hudson_master/war/WEB-INF/lib/hudson-core-1.316.jar!/hudson/widgets/HistoryWidget/entry.jelly:32:174: <st:nbsp> Failed to write to client
...

[Winstone 2009/11/05 05:09:56] - Untrapped Error in Servlet

java.lang.OutOfMemoryError: Java heap space

    at java.nio.HeapByteBuffer.<init>(Unknown Source)

...

A.

On Wed, Nov 4, 2009 at 6:00 AM, Eric Pyle <[hidden email]> wrote:

We have a Java program (not Hudson) which we have successfully run with –Xmx 1600M on some 32-bit Windows machines with 2 GB RAM. Other machines with the same or more RAM could not allocate more than 1200M or so. A key factor I found is that if you use the Sun JVM, it attempts to allocate the entire heap as a contiguous block of memory. So you may have plenty of free virtual memory but not be able to use all of it. We experimented a bit with the JRockit JVM from Oracle and found it was able to make more memory available – however in the end we did not want to support a second JVM and instead reduced the memory usage of our program. But it may be an option if you are stuck with needing a larger Java heap.

 

-Eric

 

---

From: Joshua McKinnon [mailto:[hidden email]]
Sent: Tuesday, November 03, 2009 5:53 PM
To: [hidden email]
Subject: Re: memory issue with Hudson server version 1.316+

 

1234m is not the max heap value for java on 32-bit windows, it is just an example of a number that isn't an increment of any value in particular.

If 1234m, or 1200m, or some number works - I would stick with it until it doesn't. The max is generally 1400-1500m, but if you set it to that...the next time the memory footprint grows and that is no longer enough, you'll have a problem you can't fix immediately, and your builds will be on hold until you diagnose why memory usage grew, or you go 64-bit. If you set it to something lower, you can try to identify the problem the next time this happens before you run out of address space to use.

The only way I know of to identify the max heap you can set is by trial and error, though like I said a heap settiting in the 1400-1500m range commonly causes the limit to be hit, since the heap is only part of Java's memory usage. If you really want to find out, you can keep adjusting by 50m or so until you hit the limit, or go under the limit.


Joshua

On Tue, Nov 3, 2009 at 4:43 PM, baz themail <[hidden email]> wrote:

The value of -Xmx1234m seems to work. Thank you so much.

Just curious, how can i determine on the valid values? Is 1234 the max within 32-bit?

 

On Tue, Nov 3, 2009 at 2:22 PM, Joshua McKinnon <[hidden email]> wrote:

You should be able to set any value in MB that you want, until you exceed the limit for a 32-bit process (just guessing, since 2048 did not work for you)

i.e. -Xmx1100m, -Xmx1150m, -Xmx1234m should all be legal.


From memory, on windows anyway, Java usually tops out by 1400-1500m max heap size as a 32-bit process, due to other memory usage outside of the heap itself adding up to the 2 gig limit imposed per process in 32-bit windows. I'm not sure what OS you're on, I'm just going by personal experience.

That said - I am also interested in how much memory usage may have increased in Hudson (or more likely, certain plugins I use). I currently run 1.316 and previously was on 1.280ish and did see a jump as well. In my case, I upped my heap for Hudson from 500m to 800m...


Joshua

On Tue, Nov 3, 2009 at 3:39 PM, baz themail <[hidden email]> wrote:

When i set to 2048, then the error is "Could not create the Java virtual machine.". So i wonder what should i set above 1024? Is there a limit or convention?

 

On Wed, Oct 28, 2009 at 7:52 PM, baz themail <[hidden email]> wrote:

Can i set Xmx more than 1024? What is the next resasonable value? 2048?

 

On Wed, Oct 21, 2009 at 2:17 PM, baz themail <[hidden email]> wrote:

All,

I upgraded from 1.251 to 1.316 two weeks ago. The server stopped working because of memory usage. Does the version 1.316 use more memory than the 1.251? Any details on this?

This is my current settings:

-Xrs -Xmx1024m -Dhudson.lifecycle=hudson.lifecycle.WindowsServiceLifecycle -jar "%BASE%\hudson.war" --httpPort=80

Followings are the errors:

21-Oct-2009 11:08:41 hudson.triggers.SafeTimerTask run
SEVERE: Timer task hudson.slaves.NodeProvisioner$NodeProvisionerInvoker@11548f3 failed
java.lang.OutOfMemoryError: Java heap space
Could not create the Java virtual machine.
Could not create the Java virtual machine.

Thanks. A.

 

 

 

 

 

[Scott Tatum-2]

Are you using slaves? Your email trail hints at this. There are some
known outstanding issues with memory leaks and slaves that may be
causing your problem.

https://hudson.dev.java.net/issues/show_bug.cgi?id=4008

Actually, if you are still using 1.316 you may be suffering from an
old bug. There was a major leak fixed in 1.317. Upgrading to 1.317+
should help you a lot. Before I got 1.317 my master was OOME'ing every
other day.

https://hudson.dev.java.net/issues/show_bug.cgi?id=4045

-Scott

On Tue, Nov 17, 2009 at 10:34 AM, baz themail <[hidden email]> wrote:

> After increasing to "1234", I still have the memory heap issue but this
> time, it seems to take more than one week to get there. I have never seen
> this issue when i was using 1.251. Any idea why I have this issue with
> 1.316?
>
> On Sat, Nov 7, 2009 at 7:24 PM, baz themail <[hidden email]> wrote:
>>
>> Is it possible to monitor how much memory the Hudson server is actually
>> using? From TaskManager?
>>
>> On Thu, Nov 5, 2009 at 11:31 AM, baz themail <[hidden email]> wrote:
>>>
>>> Eric, thanks for the information on Sun JVM.
>>>
>>> I am now restart the Hudson server with "-Xmx 1234M" and see if I am
>>> going to see the problem again.
>>>
>>> Meanwhile, for "-Xmx 1024M", I saw the following error from hudson error
>>> log:
>>>
>>> Before the Heap error, I saw the followings, could they be the cause?
>>> Anyone can give me some advice? May be memory leak somewhere in version
>>> 1.316?
>>>
>>> [Winstone 2009/11/05 04:59:35] - Untrapped Error in Servlet
>>> org.apache.commons.jelly.JellyTagException:
>>> jar:file:/E:/pl/Hudson_master/war/WEB-INF/lib/hudson-core-1.316.jar!/hudson/widgets/HistoryWidget/entry.jelly:32:174:
>>> <st:nbsp> Failed to write to client
>>>     at
>>> org.apache.commons.jelly.impl.TagScript.handleException(TagScript.java:694)
>>>     at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:272)
>>> ...
>>>
>>> javax.servlet.ServletException:
>>> jar:file:/E:/pl/Hudson_master/war/WEB-INF/lib/hudson-core-1.316.jar!/hudson/widgets/HistoryWidget/entry.jelly:32:174:
>>> <st:nbsp> Failed to write to client
>>> ...
>>>
>>> [Winstone 2009/11/05 05:09:56] - Untrapped Error in Servlet
>>> java.lang.OutOfMemoryError: Java heap space
>>>     at java.nio.HeapByteBuffer.<init>(Unknown Source)
>>>
>>> ...
>>>
>>> A.
>>>
>>> On Wed, Nov 4, 2009 at 6:00 AM, Eric Pyle <[hidden email]>
>>> wrote:
>>>>
>>>> We have a Java program (not Hudson) which we have successfully run with
>>>> –Xmx 1600M on some 32-bit Windows machines with 2 GB RAM. Other machines
>>>> with the same or more RAM could not allocate more than 1200M or so. A key
>>>> factor I found is that if you use the Sun JVM, it attempts to allocate the
>>>> entire heap as a contiguous block of memory. So you may have plenty of free
>>>> virtual memory but not be able to use all of it. We experimented a bit with
>>>> the JRockit JVM from Oracle and found it was able to make more memory
>>>> available – however in the end we did not want to support a second JVM and
>>>> instead reduced the memory usage of our program. But it may be an option if
>>>> you are stuck with needing a larger Java heap.
>>>>
>>>>
>>>>
>>>> -Eric
>>>>
>>>>
>>>>
>>>> ________________________________
>>>>
>>>> From: Joshua McKinnon [mailto:[hidden email]]
>>>> Sent: Tuesday, November 03, 2009 5:53 PM
>>>> To: [hidden email]
>>>> Subject: Re: memory issue with Hudson server version 1.316+
>>>>
>>>>
>>>>
>>>> 1234m is not the max heap value for java on 32-bit windows, it is just
>>>> an example of a number that isn't an increment of any value in particular.
>>>>
>>>> If 1234m, or 1200m, or some number works - I would stick with it until
>>>> it doesn't. The max is generally 1400-1500m, but if you set it to that...the
>>>> next time the memory footprint grows and that is no longer enough, you'll
>>>> have a problem you can't fix immediately, and your builds will be on hold
>>>> until you diagnose why memory usage grew, or you go 64-bit. If you set it to
>>>> something lower, you can try to identify the problem the next time this
>>>> happens before you run out of address space to use.
>>>>
>>>> The only way I know of to identify the max heap you can set is by trial
>>>> and error, though like I said a heap settiting in the 1400-1500m range
>>>> commonly causes the limit to be hit, since the heap is only part of Java's
>>>> memory usage. If you really want to find out, you can keep adjusting by 50m
>>>> or so until you hit the limit, or go under the limit.
>>>>
>>>>
>>>> Joshua
>>>>
>>>> On Tue, Nov 3, 2009 at 4:43 PM, baz themail <[hidden email]>
>>>> wrote:
>>>>
>>>> The value of -Xmx1234m seems to work. Thank you so much.
>>>>
>>>> Just curious, how can i determine on the valid values? Is 1234 the max
>>>> within 32-bit?
>>>>
>>>>
>>>>
>>>> On Tue, Nov 3, 2009 at 2:22 PM, Joshua McKinnon <[hidden email]>
>>>> wrote:
>>>>
>>>> You should be able to set any value in MB that you want, until you
>>>> exceed the limit for a 32-bit process (just guessing, since 2048 did not
>>>> work for you)
>>>>
>>>> i.e. -Xmx1100m, -Xmx1150m, -Xmx1234m should all be legal.
>>>>
>>>>
>>>> From memory, on windows anyway, Java usually tops out by 1400-1500m max
>>>> heap size as a 32-bit process, due to other memory usage outside of the heap
>>>> itself adding up to the 2 gig limit imposed per process in 32-bit windows.
>>>> I'm not sure what OS you're on, I'm just going by personal experience.
>>>>
>>>> That said - I am also interested in how much memory usage may have
>>>> increased in Hudson (or more likely, certain plugins I use). I currently run
>>>> 1.316 and previously was on 1.280ish and did see a jump as well. In my case,
>>>> I upped my heap for Hudson from 500m to 800m...
>>>>
>>>>
>>>> Joshua
>>>>
>>>> On Tue, Nov 3, 2009 at 3:39 PM, baz themail <[hidden email]>
>>>> wrote:
>>>>
>>>> When i set to 2048, then the error is "Could not create the Java virtual
>>>> machine.". So i wonder what should i set above 1024? Is there a limit or
>>>> convention?
>>>>
>>>>
>>>>
>>>> On Wed, Oct 28, 2009 at 7:52 PM, baz themail <[hidden email]>
>>>> wrote:
>>>>
>>>> Can i set Xmx more than 1024? What is the next resasonable value? 2048?
>>>>
>>>>
>>>>
>>>> On Wed, Oct 21, 2009 at 2:17 PM, baz themail <[hidden email]>
>>>> wrote:
>>>>
>>>> All,
>>>>
>>>> I upgraded from 1.251 to 1.316 two weeks ago. The server stopped working
>>>> because of memory usage. Does the version 1.316 use more memory than the
>>>> 1.251? Any details on this?
>>>>
>>>> This is my current settings:
>>>>
>>>> -Xrs -Xmx1024m
>>>> -Dhudson.lifecycle=hudson.lifecycle.WindowsServiceLifecycle -jar
>>>> "%BASE%\hudson.war" --httpPort=80
>>>>
>>>> Followings are the errors:
>>>>
>>>> 21-Oct-2009 11:08:41 hudson.triggers.SafeTimerTask run
>>>> SEVERE: Timer task
>>>> hudson.slaves.NodeProvisioner$NodeProvisionerInvoker@11548f3 failed
>>>> java.lang.OutOfMemoryError: Java heap space
>>>> Could not create the Java virtual machine.
>>>> Could not create the Java virtual machine.
>>>>
>>>> Thanks. A.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[baz themail]

Scott,

1. Is the upgrade from 1.316 to 1.317+ straight forward?
2. Which version should i upgrade to if I am currently on 1.316?

Thank you. A.

On Tue, Nov 17, 2009 at 11:50 AM, Scott Tatum <[hidden email]> wrote:

Are you using slaves? Your email trail hints at this. There are some
known outstanding issues with memory leaks and slaves that may be
causing your problem.

https://hudson.dev.java.net/issues/show_bug.cgi?id=4008

Actually, if you are still using 1.316 you may be suffering from an
old bug. There was a major leak fixed in 1.317. Upgrading to 1.317+
should help you a lot. Before I got 1.317 my master was OOME'ing every
other day.

https://hudson.dev.java.net/issues/show_bug.cgi?id=4045

-Scott

On Tue, Nov 17, 2009 at 10:34 AM, baz themail <[hidden email]> wrote:
> After increasing to "1234", I still have the memory heap issue but this
> time, it seems to take more than one week to get there. I have never seen
> this issue when i was using 1.251. Any idea why I have this issue with
> 1.316?
>
> On Sat, Nov 7, 2009 at 7:24 PM, baz themail <[hidden email]> wrote:
>>
>> Is it possible to monitor how much memory the Hudson server is actually
>> using? From TaskManager?
>>
>> On Thu, Nov 5, 2009 at 11:31 AM, baz themail <[hidden email]> wrote:
>>>
>>> Eric, thanks for the information on Sun JVM.
>>>
>>> I am now restart the Hudson server with "-Xmx 1234M" and see if I am
>>> going to see the problem again.
>>>
>>> Meanwhile, for "-Xmx 1024M", I saw the following error from hudson error
>>> log:
>>>
>>> Before the Heap error, I saw the followings, could they be the cause?
>>> Anyone can give me some advice? May be memory leak somewhere in version
>>> 1.316?
>>>
>>> [Winstone 2009/11/05 04:59:35] - Untrapped Error in Servlet
>>> org.apache.commons.jelly.JellyTagException:
>>> jar:file:/E:/pl/Hudson_master/war/WEB-INF/lib/hudson-core-1.316.jar!/hudson/widgets/HistoryWidget/entry.jelly:32:174:
>>> <st:nbsp> Failed to write to client
>>>     at
>>> org.apache.commons.jelly.impl.TagScript.handleException(TagScript.java:694)
>>>     at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:272)
>>> ...
>>>
>>> javax.servlet.ServletException:
>>> jar:file:/E:/pl/Hudson_master/war/WEB-INF/lib/hudson-core-1.316.jar!/hudson/widgets/HistoryWidget/entry.jelly:32:174:
>>> <st:nbsp> Failed to write to client
>>> ...
>>>
>>> [Winstone 2009/11/05 05:09:56] - Untrapped Error in Servlet
>>> java.lang.OutOfMemoryError: Java heap space
>>>     at java.nio.HeapByteBuffer.<init>(Unknown Source)
>>>
>>> ...
>>>
>>> A.
>>>
>>> On Wed, Nov 4, 2009 at 6:00 AM, Eric Pyle <[hidden email]>
>>> wrote:
>>>>
>>>> We have a Java program (not Hudson) which we have successfully run with
>>>> –Xmx 1600M on some 32-bit Windows machines with 2 GB RAM. Other machines
>>>> with the same or more RAM could not allocate more than 1200M or so. A key
>>>> factor I found is that if you use the Sun JVM, it attempts to allocate the
>>>> entire heap as a contiguous block of memory. So you may have plenty of free
>>>> virtual memory but not be able to use all of it. We experimented a bit with
>>>> the JRockit JVM from Oracle and found it was able to make more memory
>>>> available – however in the end we did not want to support a second JVM and
>>>> instead reduced the memory usage of our program. But it may be an option if
>>>> you are stuck with needing a larger Java heap.
>>>>
>>>>
>>>>
>>>> -Eric
>>>>
>>>>
>>>>
>>>> ________________________________
>>>>
>>>> From: Joshua McKinnon [mailto:[hidden email]]
>>>> Sent: Tuesday, November 03, 2009 5:53 PM
>>>> To: [hidden email]
>>>> Subject: Re: memory issue with Hudson server version 1.316+
>>>>
>>>>
>>>>
>>>> 1234m is not the max heap value for java on 32-bit windows, it is just
>>>> an example of a number that isn't an increment of any value in particular.
>>>>
>>>> If 1234m, or 1200m, or some number works - I would stick with it until
>>>> it doesn't. The max is generally 1400-1500m, but if you set it to that...the
>>>> next time the memory footprint grows and that is no longer enough, you'll
>>>> have a problem you can't fix immediately, and your builds will be on hold
>>>> until you diagnose why memory usage grew, or you go 64-bit. If you set it to
>>>> something lower, you can try to identify the problem the next time this
>>>> happens before you run out of address space to use.
>>>>
>>>> The only way I know of to identify the max heap you can set is by trial
>>>> and error, though like I said a heap settiting in the 1400-1500m range
>>>> commonly causes the limit to be hit, since the heap is only part of Java's
>>>> memory usage. If you really want to find out, you can keep adjusting by 50m
>>>> or so until you hit the limit, or go under the limit.
>>>>
>>>>
>>>> Joshua
>>>>
>>>> On Tue, Nov 3, 2009 at 4:43 PM, baz themail <[hidden email]>
>>>> wrote:
>>>>
>>>> The value of -Xmx1234m seems to work. Thank you so much.
>>>>
>>>> Just curious, how can i determine on the valid values? Is 1234 the max
>>>> within 32-bit?
>>>>
>>>>
>>>>
>>>> On Tue, Nov 3, 2009 at 2:22 PM, Joshua McKinnon <[hidden email]>
>>>> wrote:
>>>>
>>>> You should be able to set any value in MB that you want, until you
>>>> exceed the limit for a 32-bit process (just guessing, since 2048 did not
>>>> work for you)
>>>>
>>>> i.e. -Xmx1100m, -Xmx1150m, -Xmx1234m should all be legal.
>>>>
>>>>
>>>> From memory, on windows anyway, Java usually tops out by 1400-1500m max
>>>> heap size as a 32-bit process, due to other memory usage outside of the heap
>>>> itself adding up to the 2 gig limit imposed per process in 32-bit windows.
>>>> I'm not sure what OS you're on, I'm just going by personal experience.
>>>>
>>>> That said - I am also interested in how much memory usage may have
>>>> increased in Hudson (or more likely, certain plugins I use). I currently run
>>>> 1.316 and previously was on 1.280ish and did see a jump as well. In my case,
>>>> I upped my heap for Hudson from 500m to 800m...
>>>>
>>>>
>>>> Joshua
>>>>
>>>> On Tue, Nov 3, 2009 at 3:39 PM, baz themail <[hidden email]>
>>>> wrote:
>>>>
>>>> When i set to 2048, then the error is "Could not create the Java virtual
>>>> machine.". So i wonder what should i set above 1024? Is there a limit or
>>>> convention?
>>>>
>>>>
>>>>
>>>> On Wed, Oct 28, 2009 at 7:52 PM, baz themail <[hidden email]>
>>>> wrote:
>>>>
>>>> Can i set Xmx more than 1024? What is the next resasonable value? 2048?
>>>>
>>>>
>>>>
>>>> On Wed, Oct 21, 2009 at 2:17 PM, baz themail <[hidden email]>
>>>> wrote:
>>>>
>>>> All,
>>>>
>>>> I upgraded from 1.251 to 1.316 two weeks ago. The server stopped working
>>>> because of memory usage. Does the version 1.316 use more memory than the
>>>> 1.251? Any details on this?
>>>>
>>>> This is my current settings:
>>>>
>>>> -Xrs -Xmx1024m
>>>> -Dhudson.lifecycle=hudson.lifecycle.WindowsServiceLifecycle -jar
>>>> "%BASE%\hudson.war" --httpPort=80
>>>>
>>>> Followings are the errors:
>>>>
>>>> 21-Oct-2009 11:08:41 hudson.triggers.SafeTimerTask run
>>>> SEVERE: Timer task
>>>> hudson.slaves.NodeProvisioner$NodeProvisionerInvoker@11548f3 failed
>>>> java.lang.OutOfMemoryError: Java heap space
>>>> Could not create the Java virtual machine.
>>>> Could not create the Java virtual machine.
>>>>
>>>> Thanks. A.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[baz themail]

Scott,

The second link may be an issue, cause the first link seems to be about the slave and I had memory heap issue in hudson server and causing it to be non-responsive.

A.

On Tue, Nov 17, 2009 at 11:50 AM, Scott Tatum <[hidden email]> wrote:

Are you using slaves? Your email trail hints at this. There are some
known outstanding issues with memory leaks and slaves that may be
causing your problem.

https://hudson.dev.java.net/issues/show_bug.cgi?id=4008

Actually, if you are still using 1.316 you may be suffering from an
old bug. There was a major leak fixed in 1.317. Upgrading to 1.317+
should help you a lot. Before I got 1.317 my master was OOME'ing every
other day.

https://hudson.dev.java.net/issues/show_bug.cgi?id=4045

-Scott

On Tue, Nov 17, 2009 at 10:34 AM, baz themail <[hidden email]> wrote:
> After increasing to "1234", I still have the memory heap issue but this
> time, it seems to take more than one week to get there. I have never seen
> this issue when i was using 1.251. Any idea why I have this issue with
> 1.316?
>
> On Sat, Nov 7, 2009 at 7:24 PM, baz themail <[hidden email]> wrote:
>>
>> Is it possible to monitor how much memory the Hudson server is actually
>> using? From TaskManager?
>>
>> On Thu, Nov 5, 2009 at 11:31 AM, baz themail <[hidden email]> wrote:
>>>
>>> Eric, thanks for the information on Sun JVM.
>>>
>>> I am now restart the Hudson server with "-Xmx 1234M" and see if I am
>>> going to see the problem again.
>>>
>>> Meanwhile, for "-Xmx 1024M", I saw the following error from hudson error
>>> log:
>>>
>>> Before the Heap error, I saw the followings, could they be the cause?
>>> Anyone can give me some advice? May be memory leak somewhere in version
>>> 1.316?
>>>
>>> [Winstone 2009/11/05 04:59:35] - Untrapped Error in Servlet
>>> org.apache.commons.jelly.JellyTagException:
>>> jar:file:/E:/pl/Hudson_master/war/WEB-INF/lib/hudson-core-1.316.jar!/hudson/widgets/HistoryWidget/entry.jelly:32:174:
>>> <st:nbsp> Failed to write to client
>>>     at
>>> org.apache.commons.jelly.impl.TagScript.handleException(TagScript.java:694)
>>>     at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:272)
>>> ...
>>>
>>> javax.servlet.ServletException:
>>> jar:file:/E:/pl/Hudson_master/war/WEB-INF/lib/hudson-core-1.316.jar!/hudson/widgets/HistoryWidget/entry.jelly:32:174:
>>> <st:nbsp> Failed to write to client
>>> ...
>>>
>>> [Winstone 2009/11/05 05:09:56] - Untrapped Error in Servlet
>>> java.lang.OutOfMemoryError: Java heap space
>>>     at java.nio.HeapByteBuffer.<init>(Unknown Source)
>>>
>>> ...
>>>
>>> A.
>>>
>>> On Wed, Nov 4, 2009 at 6:00 AM, Eric Pyle <[hidden email]>
>>> wrote:
>>>>
>>>> We have a Java program (not Hudson) which we have successfully run with
>>>> –Xmx 1600M on some 32-bit Windows machines with 2 GB RAM. Other machines
>>>> with the same or more RAM could not allocate more than 1200M or so. A key
>>>> factor I found is that if you use the Sun JVM, it attempts to allocate the
>>>> entire heap as a contiguous block of memory. So you may have plenty of free
>>>> virtual memory but not be able to use all of it. We experimented a bit with
>>>> the JRockit JVM from Oracle and found it was able to make more memory
>>>> available – however in the end we did not want to support a second JVM and
>>>> instead reduced the memory usage of our program. But it may be an option if
>>>> you are stuck with needing a larger Java heap.
>>>>
>>>>
>>>>
>>>> -Eric
>>>>
>>>>
>>>>
>>>> ________________________________
>>>>
>>>> From: Joshua McKinnon [mailto:[hidden email]]
>>>> Sent: Tuesday, November 03, 2009 5:53 PM
>>>> To: [hidden email]
>>>> Subject: Re: memory issue with Hudson server version 1.316+
>>>>
>>>>
>>>>
>>>> 1234m is not the max heap value for java on 32-bit windows, it is just
>>>> an example of a number that isn't an increment of any value in particular.
>>>>
>>>> If 1234m, or 1200m, or some number works - I would stick with it until
>>>> it doesn't. The max is generally 1400-1500m, but if you set it to that...the
>>>> next time the memory footprint grows and that is no longer enough, you'll
>>>> have a problem you can't fix immediately, and your builds will be on hold
>>>> until you diagnose why memory usage grew, or you go 64-bit. If you set it to
>>>> something lower, you can try to identify the problem the next time this
>>>> happens before you run out of address space to use.
>>>>
>>>> The only way I know of to identify the max heap you can set is by trial
>>>> and error, though like I said a heap settiting in the 1400-1500m range
>>>> commonly causes the limit to be hit, since the heap is only part of Java's
>>>> memory usage. If you really want to find out, you can keep adjusting by 50m
>>>> or so until you hit the limit, or go under the limit.
>>>>
>>>>
>>>> Joshua
>>>>
>>>> On Tue, Nov 3, 2009 at 4:43 PM, baz themail <[hidden email]>
>>>> wrote:
>>>>
>>>> The value of -Xmx1234m seems to work. Thank you so much.
>>>>
>>>> Just curious, how can i determine on the valid values? Is 1234 the max
>>>> within 32-bit?
>>>>
>>>>
>>>>
>>>> On Tue, Nov 3, 2009 at 2:22 PM, Joshua McKinnon <[hidden email]>
>>>> wrote:
>>>>
>>>> You should be able to set any value in MB that you want, until you
>>>> exceed the limit for a 32-bit process (just guessing, since 2048 did not
>>>> work for you)
>>>>
>>>> i.e. -Xmx1100m, -Xmx1150m, -Xmx1234m should all be legal.
>>>>
>>>>
>>>> From memory, on windows anyway, Java usually tops out by 1400-1500m max
>>>> heap size as a 32-bit process, due to other memory usage outside of the heap
>>>> itself adding up to the 2 gig limit imposed per process in 32-bit windows.
>>>> I'm not sure what OS you're on, I'm just going by personal experience.
>>>>
>>>> That said - I am also interested in how much memory usage may have
>>>> increased in Hudson (or more likely, certain plugins I use). I currently run
>>>> 1.316 and previously was on 1.280ish and did see a jump as well. In my case,
>>>> I upped my heap for Hudson from 500m to 800m...
>>>>
>>>>
>>>> Joshua
>>>>
>>>> On Tue, Nov 3, 2009 at 3:39 PM, baz themail <[hidden email]>
>>>> wrote:
>>>>
>>>> When i set to 2048, then the error is "Could not create the Java virtual
>>>> machine.". So i wonder what should i set above 1024? Is there a limit or
>>>> convention?
>>>>
>>>>
>>>>
>>>> On Wed, Oct 28, 2009 at 7:52 PM, baz themail <[hidden email]>
>>>> wrote:
>>>>
>>>> Can i set Xmx more than 1024? What is the next resasonable value? 2048?
>>>>
>>>>
>>>>
>>>> On Wed, Oct 21, 2009 at 2:17 PM, baz themail <[hidden email]>
>>>> wrote:
>>>>
>>>> All,
>>>>
>>>> I upgraded from 1.251 to 1.316 two weeks ago. The server stopped working
>>>> because of memory usage. Does the version 1.316 use more memory than the
>>>> 1.251? Any details on this?
>>>>
>>>> This is my current settings:
>>>>
>>>> -Xrs -Xmx1024m
>>>> -Dhudson.lifecycle=hudson.lifecycle.WindowsServiceLifecycle -jar
>>>> "%BASE%\hudson.war" --httpPort=80
>>>>
>>>> Followings are the errors:
>>>>
>>>> 21-Oct-2009 11:08:41 hudson.triggers.SafeTimerTask run
>>>> SEVERE: Timer task
>>>> hudson.slaves.NodeProvisioner$NodeProvisionerInvoker@11548f3 failed
>>>> java.lang.OutOfMemoryError: Java heap space
>>>> Could not create the Java virtual machine.
>>>> Could not create the Java virtual machine.
>>>>
>>>> Thanks. A.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[Scott Tatum-2]

Well, I would go straight to 1.334, mainly to fix the issue with
Winstone and form submission (if you are running winstone).

Upgrading is easy - if you're running standalone via. Winstone, you
should get a notification upon logging in that a new version is
available, with an "Upgrade Automatically" link. If you don't get
this, you can just download the war, copy it over the existing war,
and restart your container. You may have to go this route if your
update center is broken (which is the issue fixed in 1.334 I am
talking about).

-Scott

On Tue, Nov 17, 2009 at 3:36 PM, baz themail <[hidden email]> wrote:

> Scott,
>
> The second link may be an issue, cause the first link seems to be about the
> slave and I had memory heap issue in hudson server and causing it to be
> non-responsive.
>
> A.
>
> On Tue, Nov 17, 2009 at 11:50 AM, Scott Tatum <[hidden email]> wrote:
>>
>> Are you using slaves? Your email trail hints at this. There are some
>> known outstanding issues with memory leaks and slaves that may be
>> causing your problem.
>>
>> https://hudson.dev.java.net/issues/show_bug.cgi?id=4008
>>
>> Actually, if you are still using 1.316 you may be suffering from an
>> old bug. There was a major leak fixed in 1.317. Upgrading to 1.317+
>> should help you a lot. Before I got 1.317 my master was OOME'ing every
>> other day.
>>
>> https://hudson.dev.java.net/issues/show_bug.cgi?id=4045
>>
>> -Scott
>>
>> On Tue, Nov 17, 2009 at 10:34 AM, baz themail <[hidden email]>
>> wrote:
>> > After increasing to "1234", I still have the memory heap issue but this
>> > time, it seems to take more than one week to get there. I have never
>> > seen
>> > this issue when i was using 1.251. Any idea why I have this issue with
>> > 1.316?
>> >
>> > On Sat, Nov 7, 2009 at 7:24 PM, baz themail <[hidden email]>
>> > wrote:
>> >>
>> >> Is it possible to monitor how much memory the Hudson server is actually
>> >> using? From TaskManager?
>> >>
>> >> On Thu, Nov 5, 2009 at 11:31 AM, baz themail <[hidden email]>
>> >> wrote:
>> >>>
>> >>> Eric, thanks for the information on Sun JVM.
>> >>>
>> >>> I am now restart the Hudson server with "-Xmx 1234M" and see if I am
>> >>> going to see the problem again.
>> >>>
>> >>> Meanwhile, for "-Xmx 1024M", I saw the following error from hudson
>> >>> error
>> >>> log:
>> >>>
>> >>> Before the Heap error, I saw the followings, could they be the cause?
>> >>> Anyone can give me some advice? May be memory leak somewhere in
>> >>> version
>> >>> 1.316?
>> >>>
>> >>> [Winstone 2009/11/05 04:59:35] - Untrapped Error in Servlet
>> >>> org.apache.commons.jelly.JellyTagException:
>> >>>
>> >>> jar:file:/E:/pl/Hudson_master/war/WEB-INF/lib/hudson-core-1.316.jar!/hudson/widgets/HistoryWidget/entry.jelly:32:174:
>> >>> <st:nbsp> Failed to write to client
>> >>>     at
>> >>>
>> >>> org.apache.commons.jelly.impl.TagScript.handleException(TagScript.java:694)
>> >>>     at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:272)
>> >>> ...
>> >>>
>> >>> javax.servlet.ServletException:
>> >>>
>> >>> jar:file:/E:/pl/Hudson_master/war/WEB-INF/lib/hudson-core-1.316.jar!/hudson/widgets/HistoryWidget/entry.jelly:32:174:
>> >>> <st:nbsp> Failed to write to client
>> >>> ...
>> >>>
>> >>> [Winstone 2009/11/05 05:09:56] - Untrapped Error in Servlet
>> >>> java.lang.OutOfMemoryError: Java heap space
>> >>>     at java.nio.HeapByteBuffer.<init>(Unknown Source)
>> >>>
>> >>> ...
>> >>>
>> >>> A.
>> >>>
>> >>> On Wed, Nov 4, 2009 at 6:00 AM, Eric Pyle <[hidden email]>
>> >>> wrote:
>> >>>>
>> >>>> We have a Java program (not Hudson) which we have successfully run
>> >>>> with
>> >>>> –Xmx 1600M on some 32-bit Windows machines with 2 GB RAM. Other
>> >>>> machines
>> >>>> with the same or more RAM could not allocate more than 1200M or so. A
>> >>>> key
>> >>>> factor I found is that if you use the Sun JVM, it attempts to
>> >>>> allocate the
>> >>>> entire heap as a contiguous block of memory. So you may have plenty
>> >>>> of free
>> >>>> virtual memory but not be able to use all of it. We experimented a
>> >>>> bit with
>> >>>> the JRockit JVM from Oracle and found it was able to make more memory
>> >>>> available – however in the end we did not want to support a second
>> >>>> JVM and
>> >>>> instead reduced the memory usage of our program. But it may be an
>> >>>> option if
>> >>>> you are stuck with needing a larger Java heap.
>> >>>>
>> >>>>
>> >>>>
>> >>>> -Eric
>> >>>>
>> >>>>
>> >>>>
>> >>>> ________________________________
>> >>>>
>> >>>> From: Joshua McKinnon [mailto:[hidden email]]
>> >>>> Sent: Tuesday, November 03, 2009 5:53 PM
>> >>>> To: [hidden email]
>> >>>> Subject: Re: memory issue with Hudson server version 1.316+
>> >>>>
>> >>>>
>> >>>>
>> >>>> 1234m is not the max heap value for java on 32-bit windows, it is
>> >>>> just
>> >>>> an example of a number that isn't an increment of any value in
>> >>>> particular.
>> >>>>
>> >>>> If 1234m, or 1200m, or some number works - I would stick with it
>> >>>> until
>> >>>> it doesn't. The max is generally 1400-1500m, but if you set it to
>> >>>> that...the
>> >>>> next time the memory footprint grows and that is no longer enough,
>> >>>> you'll
>> >>>> have a problem you can't fix immediately, and your builds will be on
>> >>>> hold
>> >>>> until you diagnose why memory usage grew, or you go 64-bit. If you
>> >>>> set it to
>> >>>> something lower, you can try to identify the problem the next time
>> >>>> this
>> >>>> happens before you run out of address space to use.
>> >>>>
>> >>>> The only way I know of to identify the max heap you can set is by
>> >>>> trial
>> >>>> and error, though like I said a heap settiting in the 1400-1500m
>> >>>> range
>> >>>> commonly causes the limit to be hit, since the heap is only part of
>> >>>> Java's
>> >>>> memory usage. If you really want to find out, you can keep adjusting
>> >>>> by 50m
>> >>>> or so until you hit the limit, or go under the limit.
>> >>>>
>> >>>>
>> >>>> Joshua
>> >>>>
>> >>>> On Tue, Nov 3, 2009 at 4:43 PM, baz themail <[hidden email]>
>> >>>> wrote:
>> >>>>
>> >>>> The value of -Xmx1234m seems to work. Thank you so much.
>> >>>>
>> >>>> Just curious, how can i determine on the valid values? Is 1234 the
>> >>>> max
>> >>>> within 32-bit?
>> >>>>
>> >>>>
>> >>>>
>> >>>> On Tue, Nov 3, 2009 at 2:22 PM, Joshua McKinnon
>> >>>> <[hidden email]>
>> >>>> wrote:
>> >>>>
>> >>>> You should be able to set any value in MB that you want, until you
>> >>>> exceed the limit for a 32-bit process (just guessing, since 2048 did
>> >>>> not
>> >>>> work for you)
>> >>>>
>> >>>> i.e. -Xmx1100m, -Xmx1150m, -Xmx1234m should all be legal.
>> >>>>
>> >>>>
>> >>>> From memory, on windows anyway, Java usually tops out by 1400-1500m
>> >>>> max
>> >>>> heap size as a 32-bit process, due to other memory usage outside of
>> >>>> the heap
>> >>>> itself adding up to the 2 gig limit imposed per process in 32-bit
>> >>>> windows.
>> >>>> I'm not sure what OS you're on, I'm just going by personal
>> >>>> experience.
>> >>>>
>> >>>> That said - I am also interested in how much memory usage may have
>> >>>> increased in Hudson (or more likely, certain plugins I use). I
>> >>>> currently run
>> >>>> 1.316 and previously was on 1.280ish and did see a jump as well. In
>> >>>> my case,
>> >>>> I upped my heap for Hudson from 500m to 800m...
>> >>>>
>> >>>>
>> >>>> Joshua
>> >>>>
>> >>>> On Tue, Nov 3, 2009 at 3:39 PM, baz themail <[hidden email]>
>> >>>> wrote:
>> >>>>
>> >>>> When i set to 2048, then the error is "Could not create the Java
>> >>>> virtual
>> >>>> machine.". So i wonder what should i set above 1024? Is there a limit
>> >>>> or
>> >>>> convention?
>> >>>>
>> >>>>
>> >>>>
>> >>>> On Wed, Oct 28, 2009 at 7:52 PM, baz themail <[hidden email]>
>> >>>> wrote:
>> >>>>
>> >>>> Can i set Xmx more than 1024? What is the next resasonable value?
>> >>>> 2048?
>> >>>>
>> >>>>
>> >>>>
>> >>>> On Wed, Oct 21, 2009 at 2:17 PM, baz themail <[hidden email]>
>> >>>> wrote:
>> >>>>
>> >>>> All,
>> >>>>
>> >>>> I upgraded from 1.251 to 1.316 two weeks ago. The server stopped
>> >>>> working
>> >>>> because of memory usage. Does the version 1.316 use more memory than
>> >>>> the
>> >>>> 1.251? Any details on this?
>> >>>>
>> >>>> This is my current settings:
>> >>>>
>> >>>> -Xrs -Xmx1024m
>> >>>> -Dhudson.lifecycle=hudson.lifecycle.WindowsServiceLifecycle -jar
>> >>>> "%BASE%\hudson.war" --httpPort=80
>> >>>>
>> >>>> Followings are the errors:
>> >>>>
>> >>>> 21-Oct-2009 11:08:41 hudson.triggers.SafeTimerTask run
>> >>>> SEVERE: Timer task
>> >>>> hudson.slaves.NodeProvisioner$NodeProvisionerInvoker@11548f3 failed
>> >>>> java.lang.OutOfMemoryError: Java heap space
>> >>>> Could not create the Java virtual machine.
>> >>>> Could not create the Java virtual machine.
>> >>>>
>> >>>> Thanks. A.
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]