Incredible memory usage on public server, normal on dev server

19 messages Options
Embed this post
Permalink
jollyr0ger

Incredible memory usage on public server, normal on dev server

Reply Threaded More More options
Print post
Permalink
Hi,
my ZF application runs perfectely with 16mb of memory (into php.ini) on my dev machine.

The same code on the public hosting (limit to 64mb of memory) returns error like:
"Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to allocate 532655056 bytes) in Unknown on line 0"

Try to allocate 500mb of memory! Can't be, it is simply loading the login page: http://beta.hinii.com/en/auth/login

Try to open the same page many times. One times is loaded (with the error at the bottom) and one time return "Internal Server Error". An so, again and again.

I've inserted memory_get_peak_usage() to the end of the request cycle (visible now at the end of each page) and it isn't higher than 9-10mb.

How can I resolve the problem?
Thanks

PS: a friend zf developer, said to me that can be a bug of zend_db (mysql adapter) that doesn't do the mysql_free_resource(). Can be? How to solve it?
fab2008

Re: Incredible memory usage on public server, normal on dev server

Reply Threaded More More options
Print post
Permalink
jollyr0ger wrote:
Hi,
my ZF application runs perfectely with 16mb of memory (into php.ini) on my dev machine.

The same code on the public hosting (limit to 64mb of memory) returns error like:
"Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to allocate 532655056 bytes) in Unknown on line 0"

Try to allocate 500mb of memory! Can't be, it is simply loading the login page: http://beta.hinii.com/en/auth/login

I've inserted memory_get_peak_usage() to the end of the request cycle (visible now at the end of each page) and it isn't higher than 9-10mb.
I have a similar problem using Zend_Mail, I found a couple of these messages in my php logfile:

PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 115901301 bytes) in /home/fabio/ZendFramework-1.9.4/library/Zend/Mail/Protocol/Abstract.php on line 284
[04-Nov-2009 14:55:41] PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted

Now what is strange is that enormous request of memory, how is possible to ask system for 115MB in a single line of code? Maybe a php bug? Or more likely a bug in php error reporting function. I can't believe that the indicated line asks for 115MB of memory to the system.

My php version is:
# php -v
PHP 5.2.6-1+lenny3 with Suhosin-Patch 0.9.6.2 (cli) (built: Apr 26 2009 22:16:23)
Copyright (c) 1997-2008 The PHP Group
Zend Engine v2.2.0, Copyright (c) 1998-2008 Zend Technologies

P.s. I tried to trace the error using memory_get_peak_usage() at the end of my script. When it ends with no errors that function reports only 20-30MB of ram used.
jollyr0ger

Re: Incredible memory usage on public server, normal on dev server

Reply Threaded More More options
Print post
Permalink
Yeah, my process just to show a page (without doing some logic operation, just showing the view) sometimes request 490mb of memory...

Can be a bug... But I can't get help from zend developers.



fab2008 wrote:
I have a similar problem using Zend_Mail, I found a couple of these messages in my php logfile:

PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 115901301 bytes) in /home/fabio/ZendFramework-1.9.4/library/Zend/Mail/Protocol/Abstract.php on line 284
[04-Nov-2009 14:55:41] PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted

Now what is strange is that enormous request of memory, how is possible to ask system for 115MB in a single line of code? Maybe a php bug? Or more likely a bug in php error reporting function. I can't believe that the indicated line asks for 115MB of memory to the system.

My php version is:
# php -v
PHP 5.2.6-1+lenny3 with Suhosin-Patch 0.9.6.2 (cli) (built: Apr 26 2009 22:16:23)
Copyright (c) 1997-2008 The PHP Group
Zend Engine v2.2.0, Copyright (c) 1998-2008 Zend Technologies

P.s. I tried to trace the error using memory_get_peak_usage() at the end of my script. When it ends with no errors that function reports only 20-30MB of ram used.
vRandom

RE: Incredible memory usage on public server, normal on dev server

Reply Threaded More More options
Print post
Permalink
In reply to this post by fab2008

I could see how the code at line 284 on
/home/fabio/ZendFramework-1.9.4/library/Zend/Mail/Protocol/Abstract.php
could attempt to take a lot of memory, if the $request was large or the
function was called multiple times.

// Save request to internal log
$this->_log .= $request . self::EOL;

You could try adding a syslog debug code before line 284 and see if the size
of the $request is the problem.

syslog(LOG_NOTICE, "{$_SERVER['REMOTE_ADDR']}: debug: Abstract.php - ".
Strlen($request));

Maybe to make it cleaner wrap a try around line 284, and then syslog the
size when an error occurs, or even dump the request string to a file and see
what it was. Though I'd start with checking the size.

Just be warned if your site is busy this could flood the syslog.

As for the similar error in a login, http://beta.hinii.com/en/auth/login

As a suggestion, I'd put in a limit on the number of characters they can
enter for a username/password. The form doesn't have any limit set
currently.

Not sure on the memory error, there should be a line before the "Unknown on
line 0" that might give more clues.

I've seen the "Unknown on line 0" error with db calls, file uploads and a
few other times in loops...

Terre

-----Original Message-----
From: fab2008 [mailto:[hidden email]]
Sent: Thursday, November 05, 2009 9:48 PM
To: [hidden email]
Subject: Re: [fw-general] Incredible memory usage on public server, normal
on dev server



jollyr0ger wrote:

>
> Hi,
> my ZF application runs perfectely with 16mb of memory (into php.ini)
> on my dev machine.
>
> The same code on the public hosting (limit to 64mb of memory) returns
> error like:
> "Fatal error: Allowed memory size of 67108864 bytes exhausted (tried
> to allocate 532655056 bytes) in Unknown on line 0"
>
> Try to allocate 500mb of memory! Can't be, it is simply loading the
> login
> page: http://beta.hinii.com/en/auth/login
>
> I've inserted memory_get_peak_usage() to the end of the request cycle
> (visible now at the end of each page) and it isn't higher than 9-10mb.
>

I have a similar problem using Zend_Mail, I found a couple of these messages
in my php logfile:

PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to
allocate 115901301 bytes) in
/home/fabio/ZendFramework-1.9.4/library/Zend/Mail/Protocol/Abstract.php on
line 284
[04-Nov-2009 14:55:41] PHP Fatal error:  Allowed memory size of 134217728
bytes exhausted

Now what is strange is that enormous request of memory, how is possible to
ask system for 115MB in a single line of code? Maybe a php bug? Or more
likely a bug in php error reporting function. I can't believe that the
indicated line asks for 115MB of memory to the system.

My php version is:
# php -v
PHP 5.2.6-1+lenny3 with Suhosin-Patch 0.9.6.2 (cli) (built: Apr 26 2009
22:16:23)
Copyright (c) 1997-2008 The PHP Group
Zend Engine v2.2.0, Copyright (c) 1998-2008 Zend Technologies

P.s. I tried to trace the error using memory_get_peak_usage() at the end of
my script. When it ends with no errors that function reports only 20-30MB of
ram used.

--
View this message in context:
http://old.nabble.com/Incredible-memory-usage-on-public-server%2C-normal-on-
dev-server-tp26215767p26225656.html
Sent from the Zend Framework mailing list archive at Nabble.com.

Ralph Schindler-2

Re: Incredible memory usage on public server, normal on dev server

Reply Threaded More More options
Print post
Permalink
In reply to this post by fab2008
One thing to keep in mind... Just because the request is running out of
memory on that line does not mean that it is that file/line/execution
that is the culprit.  It only means that the like mentioned is where you
have run out of memory.

This basically means that any process before that line could be the
problematic code.  I would try to profile your applicaion, or even
perhaps see if you are attempting to retrieve a large dataset before
spinning off a mail?  Perhaps you have Zend_Pdf in play creating a large
document?

There are alot of variables here.  I would try to find out what part of
the process is consuming all the memory.

-ralph

fab2008 wrote:

>
> jollyr0ger wrote:
>> Hi,
>> my ZF application runs perfectely with 16mb of memory (into php.ini) on my
>> dev machine.
>>
>> The same code on the public hosting (limit to 64mb of memory) returns
>> error like:
>> "Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to
>> allocate 532655056 bytes) in Unknown on line 0"
>>
>> Try to allocate 500mb of memory! Can't be, it is simply loading the login
>> page: http://beta.hinii.com/en/auth/login
>>
>> I've inserted memory_get_peak_usage() to the end of the request cycle
>> (visible now at the end of each page) and it isn't higher than 9-10mb.
>>
>
> I have a similar problem using Zend_Mail, I found a couple of these messages
> in my php logfile:
>
> PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to
> allocate 115901301 bytes) in
> /home/fabio/ZendFramework-1.9.4/library/Zend/Mail/Protocol/Abstract.php on
> line 284
> [04-Nov-2009 14:55:41] PHP Fatal error:  Allowed memory size of 134217728
> bytes exhausted
>
> Now what is strange is that enormous request of memory, how is possible to
> ask system for 115MB in a single line of code? Maybe a php bug? Or more
> likely a bug in php error reporting function. I can't believe that the
> indicated line asks for 115MB of memory to the system.
>
> My php version is:
> # php -v
> PHP 5.2.6-1+lenny3 with Suhosin-Patch 0.9.6.2 (cli) (built: Apr 26 2009
> 22:16:23)
> Copyright (c) 1997-2008 The PHP Group
> Zend Engine v2.2.0, Copyright (c) 1998-2008 Zend Technologies
>
> P.s. I tried to trace the error using memory_get_peak_usage() at the end of
> my script. When it ends with no errors that function reports only 20-30MB of
> ram used.
>
Kevin McArthur-2

Re: Incredible memory usage on public server, normal on dev server

Reply Threaded More More options
Print post
Permalink
Ralph, normally you'd be right, but since that allocation is 500 megs in
one-shot. It should be near the code that is the problem. Would have to
see what he's doing but im guessing a massive attachment?

If it was a pecked-to-death-by-ducks thing, you'd see a message saying
"Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to
allocate 16 bytes) in Unknown on line 0" or something similar, with a
low byte count.

Looking at the file at line ~284 it appears thats where the email
finally gets sent, and logged. To debug, check the size of $request at
that point. If its huge, you have your error.

K

Ralph Schindler wrote:

> One thing to keep in mind... Just because the request is running out
> of memory on that line does not mean that it is that
> file/line/execution that is the culprit.  It only means that the like
> mentioned is where you have run out of memory.
>
> This basically means that any process before that line could be the
> problematic code.  I would try to profile your applicaion, or even
> perhaps see if you are attempting to retrieve a large dataset before
> spinning off a mail?  Perhaps you have Zend_Pdf in play creating a
> large document?
>
> There are alot of variables here.  I would try to find out what part
> of the process is consuming all the memory.
>
> -ralph
>
> fab2008 wrote:
>>
>> jollyr0ger wrote:
>>> Hi,
>>> my ZF application runs perfectely with 16mb of memory (into php.ini)
>>> on my
>>> dev machine.
>>>
>>> The same code on the public hosting (limit to 64mb of memory) returns
>>> error like:
>>> "Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to
>>> allocate 532655056 bytes) in Unknown on line 0"
>>>
>>> Try to allocate 500mb of memory! Can't be, it is simply loading the
>>> login
>>> page: http://beta.hinii.com/en/auth/login
>>>
>>> I've inserted memory_get_peak_usage() to the end of the request cycle
>>> (visible now at the end of each page) and it isn't higher than 9-10mb.
>>>
>>
>> I have a similar problem using Zend_Mail, I found a couple of these
>> messages
>> in my php logfile:
>>
>> PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted
>> (tried to
>> allocate 115901301 bytes) in
>> /home/fabio/ZendFramework-1.9.4/library/Zend/Mail/Protocol/Abstract.php
>> on
>> line 284
>> [04-Nov-2009 14:55:41] PHP Fatal error:  Allowed memory size of
>> 134217728
>> bytes exhausted
>> Now what is strange is that enormous request of memory, how is
>> possible to
>> ask system for 115MB in a single line of code? Maybe a php bug? Or more
>> likely a bug in php error reporting function. I can't believe that the
>> indicated line asks for 115MB of memory to the system.
>>
>> My php version is: # php -v
>> PHP 5.2.6-1+lenny3 with Suhosin-Patch 0.9.6.2 (cli) (built: Apr 26 2009
>> 22:16:23) Copyright (c) 1997-2008 The PHP Group
>> Zend Engine v2.2.0, Copyright (c) 1998-2008 Zend Technologies
>>
>> P.s. I tried to trace the error using memory_get_peak_usage() at the
>> end of
>> my script. When it ends with no errors that function reports only
>> 20-30MB of
>> ram used.
>>

fab2008

Re: Incredible memory usage on public server, normal on dev server

Reply Threaded More More options
Print post
Permalink

Il giorno 06/nov/2009, alle ore 22.59, Kevin McArthur ha scritto:

> Ralph, normally you'd be right, but since that allocation is 500  
> megs in one-shot. It should be near the code that is the problem.  
> Would have to see what he's doing but im guessing a massive  
> attachment?

There is no attachment in my emails, it is a cli script invoked by a  
cronjob that sends a newsletter to about 800 users, each message is  
~80KB. The algorithm is this:

1) Fetch users from db
2) Instantiate a Zend_View object
3) Instantiate a Zend_Mail object
4) For each users as u
        4.1) Render the view with u as model
        4.2) Set to address to u.email
        4.3) Set html body to the rendered view
        4.4) clear the addresses in the email
end for.

> If it was a pecked-to-death-by-ducks thing, you'd see a message  
> saying "Fatal error: Allowed memory size of 67108864 bytes exhausted  
> (tried to
> allocate 16 bytes) in Unknown on line 0" or something similar, with  
> a low byte count.

In fact IMHO that's the actual problem, the one-shot allocation (in my  
case 130MB, in the other case 500MB).

> Looking at the file at line ~284 it appears thats where the email  
> finally gets sent, and logged. To debug, check the size of $request  
> at that point. If its huge, you have your error.

I read some code, emails are sent by _sendMail function in  
Zend_Mail_Transport_Smtp which calls data method in  
Zend_Mail_Protocol_Smtp. This method calls the _send function but it  
breaks lines (line 315) so the _send argument should never be huge,  
because it's encoded by Zend_Mail and it should be broken in small  
lines.

--
Fabio Napoleoni
[hidden email]

"Computer Science is no more about computers than astronomy is
about telescopes"

                                                     Edsger W. Dijkstra

jollyr0ger

Re: Incredible memory usage on public server, normal on dev server

Reply Threaded More More options
Print post
Permalink
In reply to this post by Kevin McArthur-2
In HiNii app, the problems are not the data passed by the input fields, because the error appears just showing the page (1 time yes, 1 time no). But afther the sending of the form there are serverside validations.

There is a way to determine where the error appears? The file and the real line of code?
fab2008

Re: Incredible memory usage on public server, normal on dev server

Reply Threaded More More options
Print post
Permalink
In reply to this post by Kevin McArthur-2

Il giorno 06/nov/2009, alle ore 22.59, Kevin McArthur ha scritto:

> Ralph, normally you'd be right, but since that allocation is 500  
> megs in one-shot. It should be near the code that is the problem.  
> Would have to see what he's doing but im guessing a massive  
> attachment?
>
> If it was a pecked-to-death-by-ducks thing, you'd see a message  
> saying "Fatal error: Allowed memory size of 67108864 bytes exhausted  
> (tried to
> allocate 16 bytes) in Unknown on line 0" or something similar, with  
> a low byte count.
>
> Looking at the file at line ~284 it appears thats where the email  
> finally gets sent, and logged. To debug, check the size of $request  
> at that point. If its huge, you have your error.

For the records, I run my script with a debugger and I set a  
conditional breakpoint in the _send method with strlen($request) > 75  
as condition and the script was never halted...

--
Fabio Napoleoni
[hidden email]

"Computer Science is no more about computers than astronomy is
about telescopes"

                                                     Edsger W. Dijkstra

Ralph Schindler-2

Re: Incredible memory usage on public server, normal on dev server

Reply Threaded More More options
Print post
Permalink
I would inspect whats going on inside that user loop.  It seems like
perhaps in that loop either some objects are growing, or memory is being
lost somehow, OR your database result set when iterating is actually
larger than 500M.

Are you using Zend_Db_Table for this?  Are the rows being stored in a
container or rowset or something?

I would try putting memory_get_usage() and set a conditional breakpoint
inside your loop on that, just to see if your memory is growing
consistently per iteration.  Ideally, you can keep memory growth inside
this loop to a minimal number by cleaning up at the end of each iteration.

-ralph

Fabio Napoleoni wrote:

>
> Il giorno 06/nov/2009, alle ore 22.59, Kevin McArthur ha scritto:
>
>> Ralph, normally you'd be right, but since that allocation is 500 megs
>> in one-shot. It should be near the code that is the problem. Would
>> have to see what he's doing but im guessing a massive attachment?
>>
>> If it was a pecked-to-death-by-ducks thing, you'd see a message saying
>> "Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to
>> allocate 16 bytes) in Unknown on line 0" or something similar, with a
>> low byte count.
>>
>> Looking at the file at line ~284 it appears thats where the email
>> finally gets sent, and logged. To debug, check the size of $request at
>> that point. If its huge, you have your error.
>
> For the records, I run my script with a debugger and I set a conditional
> breakpoint in the _send method with strlen($request) > 75 as condition
> and the script was never halted...
>
> --
> Fabio Napoleoni
> [hidden email]
>
> "Computer Science is no more about computers than astronomy is
> about telescopes"
>
>                                                     Edsger W. Dijkstra
>
>
drm-4

Re: Incredible memory usage on public server, normal on dev server

Reply Threaded More More options
Print post
Permalink
In reply to this post by fab2008
Did you ever find out what was going on?
fab2008

Re: Incredible memory usage on public server, normal on dev server

Reply Threaded More More options
Print post
Permalink
drm-4 wrote:
Did you ever find out what was going on?
No, I haven't. I upgraded to 1.9.5 (from 1.9.4) and I raised memory limit to 512M. I've also added some log messages in my application, this is the result in the last few days:

2009-11-08T15:24:06-05:00 INFO (6): Peak memory usage during script: 18MB
2009-11-08T15:36:19-05:00 INFO (6): Peak memory usage during script: 149MB
2009-11-08T18:22:45-05:00 INFO (6): Peak memory usage during script: 27MB
2009-11-08T19:20:10-05:00 INFO (6): Peak memory usage during script: 24MB
2009-11-08T19:48:51-05:00 INFO (6): Peak memory usage during script: 15MB
2009-11-08T21:56:19-05:00 INFO (6): Peak memory usage during script: 17MB
2009-11-09T12:52:57-05:00 INFO (6): Peak memory usage during script: 12MB
2009-11-09T13:48:55-05:00 INFO (6): Peak memory usage during script: 21MB
2009-11-09T14:41:24-05:00 INFO (6): Peak memory usage during script: 187MB
2009-11-09T15:01:07-05:00 INFO (6): Peak memory usage during script: 29MB
2009-11-09T18:42:00-05:00 INFO (6): Peak memory usage during script: 35MB
2009-11-09T19:42:17-05:00 INFO (6): Peak memory usage during script: 24MB

In this way the script doesn't crash anymore, I'll try to investigate further to find out why some days the memory usage is higher than other.
keith Pope-4

Re: Incredible memory usage on public server, normal on dev server

Reply Threaded More More options
Print post
Permalink
2009/11/10 fab2008 <[hidden email]>:

>
>
> drm-4 wrote:
>>
>> Did you ever find out what was going on?
>>
>>
>
> No, I haven't. I upgraded to 1.9.5 (from 1.9.4) and I raised memory limit to
> 512M. I've also added some log messages in my application, this is the
> result in the last few days:
>
> 2009-11-08T15:24:06-05:00 INFO (6): Peak memory usage during script: 18MB
> 2009-11-08T15:36:19-05:00 INFO (6): Peak memory usage during script: 149MB
> 2009-11-08T18:22:45-05:00 INFO (6): Peak memory usage during script: 27MB
> 2009-11-08T19:20:10-05:00 INFO (6): Peak memory usage during script: 24MB
> 2009-11-08T19:48:51-05:00 INFO (6): Peak memory usage during script: 15MB
> 2009-11-08T21:56:19-05:00 INFO (6): Peak memory usage during script: 17MB
> 2009-11-09T12:52:57-05:00 INFO (6): Peak memory usage during script: 12MB
> 2009-11-09T13:48:55-05:00 INFO (6): Peak memory usage during script: 21MB
> 2009-11-09T14:41:24-05:00 INFO (6): Peak memory usage during script: 187MB
> 2009-11-09T15:01:07-05:00 INFO (6): Peak memory usage during script: 29MB
> 2009-11-09T18:42:00-05:00 INFO (6): Peak memory usage during script: 35MB
> 2009-11-09T19:42:17-05:00 INFO (6): Peak memory usage during script: 24MB
>
> In this way the script doesn't crash anymore, I'll try to investigate
> further to find out why some days the memory usage is higher than other.
>
> --
> View this message in context: http://old.nabble.com/Incredible-memory-usage-on-public-server%2C-normal-on-dev-server-tp26215767p26283691.html
> Sent from the Zend Framework mailing list archive at Nabble.com.
>
>

Have you tried using clearstatcache() ?

--
------------
http://www.thepopeisdead.com
fab2008

Re: Incredible memory usage on public server, normal on dev server

Reply Threaded More More options
Print post
Permalink
keith Pope-4 wrote:
2009/11/10 fab2008 <f.napoleoni@gmail.com>:
>
>
> drm-4 wrote:
>>
>> Did you ever find out what was going on?

Have you tried using clearstatcache() ?
In which manner it could be related to my problem? How to use it, at the end of the loop?
Luke Richards

Re: Incredible memory usage on public server, normal on dev server

Reply Threaded More More options
Print post
Permalink
In reply to this post by drm-4
Some javascript/style in this post has been disabled (why?)
Hi,

We use the Zend_Mail classes where I work and we're sending a lot of emails. It was a while ago but I think we had the same problem running out of memory. We tracked the problem down to the ZF SMTP transport. If you are using that try overriding it with the following code. We found the problem to be a log being kept on the abstract protocol class. Each time the protected send method is called is attaches the request to the end of the log.

class Yourlibrary_Mail_Transport_Smtp extends Zend_Mail_Transport_Smtp
{
    /**
     * Send a mail using this transport
     *
     * @param  Zend_Mail $mail
     * @access public
     * @return void
     * @throws Zend_Mail_Transport_Exception if mail is empty
     */
    public function send(Zend_Mail $mail)
    {
        $connection = $this->getConnection();
        if ($connection instanceof Zend_Mail_Protocol_Smtp) {
            $connection->resetLog();
        }
        return parent::send($mail);
    }
}

Hope that helps someone.

drm wrote:
Did you ever find out what was going on?

weierophinney

Re: Incredible memory usage on public server, normal on dev server

Reply Threaded More More options
Print post
Permalink
-- Luke Richards <[hidden email]> wrote
(on Tuesday, 10 November 2009, 02:28 PM +0000):
> We use the Zend_Mail classes where I work and we're sending a lot of emails. It
> was a while ago but I think we had the same problem running out of memory. We
> tracked the problem down to the ZF SMTP transport. If you are using that try
> overriding it with the following code. We found the problem to be a log being
> kept on the abstract protocol class. Each time the protected send method is
> called is attaches the request to the end of the log.

Could you post an issue in the tracker indicating this, as well as your
fix? It's probably something we should resolve.

> class Yourlibrary_Mail_Transport_Smtp extends Zend_Mail_Transport_Smtp
> {
>     /**
>      * Send a mail using this transport
>      *
>      * @param  Zend_Mail $mail
>      * @access public
>      * @return void
>      * @throws Zend_Mail_Transport_Exception if mail is empty
>      */
>     public function send(Zend_Mail $mail)
>     {
>         $connection = $this->getConnection();
>         if ($connection instanceof Zend_Mail_Protocol_Smtp) {
>             $connection->resetLog();
>         }
>         return parent::send($mail);
>     }
> }

--
Matthew Weier O'Phinney
Project Lead            | [hidden email]
Zend Framework          | http://framework.zend.com/
Luke Richards

Re: Incredible memory usage on public server, normal on dev server

Reply Threaded More More options
Print post
Permalink
Some javascript/style in this post has been disabled (why?)
Done

http://framework.zend.com/issues/browse/ZF-8278

Matthew Weier O'Phinney wrote:
-- Luke Richards [hidden email] wrote
(on Tuesday, 10 November 2009, 02:28 PM +0000):
  
We use the Zend_Mail classes where I work and we're sending a lot of emails. It
was a while ago but I think we had the same problem running out of memory. We
tracked the problem down to the ZF SMTP transport. If you are using that try
overriding it with the following code. We found the problem to be a log being
kept on the abstract protocol class. Each time the protected send method is
called is attaches the request to the end of the log.
    

Could you post an issue in the tracker indicating this, as well as your
fix? It's probably something we should resolve.

  
class Yourlibrary_Mail_Transport_Smtp extends Zend_Mail_Transport_Smtp
{
    /**
     * Send a mail using this transport
     *
     * @param  Zend_Mail $mail
     * @access public
     * @return void
     * @throws Zend_Mail_Transport_Exception if mail is empty
     */
    public function send(Zend_Mail $mail)
    {
        $connection = $this->getConnection();
        if ($connection instanceof Zend_Mail_Protocol_Smtp) {
            $connection->resetLog();
        }
        return parent::send($mail);
    }
}
    

  

-- 
Luke Richards

Emailcenter UK Limited
Kingthorn Park
Bradden Road
Greens Norton
Towcester
Northamptonshire
NN12 8BS

e: [hidden email]
t: 01327 350921
f: 01327 359502
www.emailcenteruk.com

UK Leaders in permission email marketing solutions and services
Privileged/Confidential information may be contained in this message. If you are not the addressee indicated in this message (or responsible for delivery of the message to such person), you may not copy or deliver this message to anyone. In such cases, you should destroy this message, and please notify us immediately.

Please advise immediately if you or your employer does not consent to Internet e-mail for messages of this kind. Opinions, conclusions and other information expressed in this message are those of the author and not necessarily endorsed by Emailcenter UK Limited unless indicated by an authorised representative of the company independent of this message. If you need to contact Emailcenter please call ++44(0) 1327 350921.

Emailcenter UK Limited is registered in the UK. Reg No.04254916 
fab2008

Re: Incredible memory usage on public server, normal on dev server

Reply Threaded More More options
Print post
Permalink
In reply to this post by Luke Richards
Luke Richards wrote:
Hi,

We use the Zend_Mail classes where I work and we're sending a lot of
emails. It was a while ago but I think we had the same problem running
out of memory. We tracked the problem down to the ZF SMTP transport. If
you are using that try overriding it with the following code. We found
the problem to be a log being kept on the abstract protocol class. Each
time the protected send method is called is attaches the request to the
end of the log.

class Yourlibrary_Mail_Transport_Smtp extends Zend_Mail_Transport_Smtp
{
    /**
     * Send a mail using this transport
     *
     * @param  Zend_Mail $mail
     * @access public
     * @return void
     * @throws Zend_Mail_Transport_Exception if mail is empty
     */
    public function send(Zend_Mail $mail)
    {
        $connection = $this->getConnection();
        if ($connection instanceof Zend_Mail_Protocol_Smtp) {
            $connection->resetLog();
        }
        return parent::send($mail);
    }
}

Hope that helps someone.
It has helped me :-) It works...

When I send a lot of mails the log grows and grows without limit. The big memory request was not for the $request var, but for reallocation of $_log field.

Thank you very much for your hint.





fab2008

Re: Incredible memory usage on public server, normal on dev server

Reply Threaded More More options
Print post
Permalink
fab2008 wrote:
Luke Richards wrote:
class Yourlibrary_Mail_Transport_Smtp extends Zend_Mail_Transport_Smtp
{
    /**
     * Send a mail using this transport
     *
     * @param  Zend_Mail $mail
     * @access public
     * @return void
     * @throws Zend_Mail_Transport_Exception if mail is empty
     */
    public function send(Zend_Mail $mail)
    {
        $connection = $this->getConnection();
        if ($connection instanceof Zend_Mail_Protocol_Smtp) {
            $connection->resetLog();
        }
        return parent::send($mail);
    }
}

Hope that helps someone.
It has helped me :-) It works...

When I send a lot of mails the log grows and grows without limit. The big memory request was not for the $request var, but for reallocation of $_log field.

Thank you very much for your hint.
It definitely works, here are the results (sending about 1200 mails):

Yesterday: 2009-11-10T15:34:28-05:00 INFO (6): Peak memory usage during script: 129MB
Today: 2009-11-11T15:13:28-05:00 INFO (6): Peak memory usage during script: 17MB

Thank you again :-)