Memory leak with CAsyncSocket::Create

I have traced a memory leak problem to CAsyncSocket::Create().  Is
this a known problem?  Is there a workaround/solution/fix?  Here is
sample code:

  for (int i=0; i<m_nReopenCount; ++i) {
    CAsyncSocket *pAS = new CAsyncSocket;
    pAS->Create();
    pAS->Close();
    delete pAS;
}

Running this 1000 times uses up 1200 KBytes of memory, or just over 1
KByte per call.  Commenting out the Create() leaves memory clean. (And
please don't complain about my bracketing style -- I like it.)

I have Visual Studio 2005 Professional version 8.0.

Incidentally, I also discovered that the call to Create() is not
re-entrant.  My application involves connecting to some 10 to 20
external devices and my normal code creates a CWinThread to support
each socket, where the socket is created and destroyed only  within
the thread routine.  Creating all the threads and starting them up
simultaneously meant having multiple instances of
CAsyncSocket::Create() being called at the same time, crashing my
system (memory access faults).    That one I found and fixed with
sentries.  Now I am left with the memory leak.

The problem is that I have an rather intricate communication protocol
system all self contained so that adding a new hardware device simply
means creating a new instance of the whole works.  It runs fine until
the external hardware goes haywire, in which case I destruct the whole
instance and start a new one which breaks and reconnects the socket
with a clean start and, most of the time, results in a good
connection; the external device resets itself through the disconnect.
One faulty device, though, generated thousand upon thousand of
disconnects over a number of days and, after a few hundred thousand of
these I my own system crashed due, I have now found out, to a lack of
memory caused by this leak.

My application must run essentially as an embedded system, unattended
week after week, month after month so I cannot tolerate a memory leak.
Does anybody know about this?  Is there a simple clean way to force a
socket disconnection on a CAsyncSocket and then reconnect?  My
application is the connect() end of the socket, not the listen() end.




0
r
7/9/2007 9:38:55 PM
vc.mfc 33608 articles. 0 followers. Follow

27 Replies
1009 Views

Similar Articles

[PageSpeed] 33

I couldn't find a memory leak.  What you are most likely seeing is windows 
memory managment doing its work.  The Create method is creating a socket 
object, when when it's freed the memory is not given back to the system 
right away.


(I used this method to detect a leak)

// Declare the variables needed
#ifdef _DEBUG
   CMemoryState oldMemState, newMemState, diffMemState;
   oldMemState.Checkpoint();
#endif

   for (int i=0; i<10; ++i)
   {
      CAsyncSocket *pAS = new CAsyncSocket;
      pAS->Create();
      pAS->Close();
      delete pAS;
   }

#ifdef _DEBUG
   newMemState.Checkpoint();
   if( diffMemState.Difference( oldMemState, newMemState ) )
   {
      TRACE( "Memory leaked!\n" );
   }
#endif

AliR.


"r norman" <r_s_norman@_comcast.net> wrote in message 
news:2895939efidggi556s7fbje0euhm2jd2d0@4ax.com...
>I have traced a memory leak problem to CAsyncSocket::Create().  Is
> this a known problem?  Is there a workaround/solution/fix?  Here is
> sample code:
>
>  for (int i=0; i<m_nReopenCount; ++i) {
>    CAsyncSocket *pAS = new CAsyncSocket;
>    pAS->Create();
>    pAS->Close();
>    delete pAS;
> }
>
> Running this 1000 times uses up 1200 KBytes of memory, or just over 1
> KByte per call.  Commenting out the Create() leaves memory clean. (And
> please don't complain about my bracketing style -- I like it.)
>
> I have Visual Studio 2005 Professional version 8.0.
>
> Incidentally, I also discovered that the call to Create() is not
> re-entrant.  My application involves connecting to some 10 to 20
> external devices and my normal code creates a CWinThread to support
> each socket, where the socket is created and destroyed only  within
> the thread routine.  Creating all the threads and starting them up
> simultaneously meant having multiple instances of
> CAsyncSocket::Create() being called at the same time, crashing my
> system (memory access faults).    That one I found and fixed with
> sentries.  Now I am left with the memory leak.
>
> The problem is that I have an rather intricate communication protocol
> system all self contained so that adding a new hardware device simply
> means creating a new instance of the whole works.  It runs fine until
> the external hardware goes haywire, in which case I destruct the whole
> instance and start a new one which breaks and reconnects the socket
> with a clean start and, most of the time, results in a good
> connection; the external device resets itself through the disconnect.
> One faulty device, though, generated thousand upon thousand of
> disconnects over a number of days and, after a few hundred thousand of
> these I my own system crashed due, I have now found out, to a lack of
> memory caused by this leak.
>
> My application must run essentially as an embedded system, unattended
> week after week, month after month so I cannot tolerate a memory leak.
> Does anybody know about this?  Is there a simple clean way to force a
> socket disconnection on a CAsyncSocket and then reconnect?  My
> application is the connect() end of the socket, not the listen() end.
>
>
>
> 


0
AliR3470 (3236)
7/10/2007 2:45:46 PM
I would have thought it *would* be returned.

I'm also concerned about the concurrency problem, because I've not seen that particular
problem in MFC before.  I'm wondering if there is some storage damage that is causing both
apparent problems.
					joe

On Tue, 10 Jul 2007 09:45:46 -0500, "AliR \(VC++ MVP\)" <AliR@online.nospam> wrote:

>I couldn't find a memory leak.  What you are most likely seeing is windows 
>memory managment doing its work.  The Create method is creating a socket 
>object, when when it's freed the memory is not given back to the system 
>right away.
>
>
>(I used this method to detect a leak)
>
>// Declare the variables needed
>#ifdef _DEBUG
>   CMemoryState oldMemState, newMemState, diffMemState;
>   oldMemState.Checkpoint();
>#endif
>
>   for (int i=0; i<10; ++i)
>   {
>      CAsyncSocket *pAS = new CAsyncSocket;
>      pAS->Create();
>      pAS->Close();
>      delete pAS;
>   }
>
>#ifdef _DEBUG
>   newMemState.Checkpoint();
>   if( diffMemState.Difference( oldMemState, newMemState ) )
>   {
>      TRACE( "Memory leaked!\n" );
>   }
>#endif
>
>AliR.
>
>
>"r norman" <r_s_norman@_comcast.net> wrote in message 
>news:2895939efidggi556s7fbje0euhm2jd2d0@4ax.com...
>>I have traced a memory leak problem to CAsyncSocket::Create().  Is
>> this a known problem?  Is there a workaround/solution/fix?  Here is
>> sample code:
>>
>>  for (int i=0; i<m_nReopenCount; ++i) {
>>    CAsyncSocket *pAS = new CAsyncSocket;
>>    pAS->Create();
>>    pAS->Close();
>>    delete pAS;
>> }
>>
>> Running this 1000 times uses up 1200 KBytes of memory, or just over 1
>> KByte per call.  Commenting out the Create() leaves memory clean. (And
>> please don't complain about my bracketing style -- I like it.)
>>
>> I have Visual Studio 2005 Professional version 8.0.
>>
>> Incidentally, I also discovered that the call to Create() is not
>> re-entrant.  My application involves connecting to some 10 to 20
>> external devices and my normal code creates a CWinThread to support
>> each socket, where the socket is created and destroyed only  within
>> the thread routine.  Creating all the threads and starting them up
>> simultaneously meant having multiple instances of
>> CAsyncSocket::Create() being called at the same time, crashing my
>> system (memory access faults).    That one I found and fixed with
>> sentries.  Now I am left with the memory leak.
>>
>> The problem is that I have an rather intricate communication protocol
>> system all self contained so that adding a new hardware device simply
>> means creating a new instance of the whole works.  It runs fine until
>> the external hardware goes haywire, in which case I destruct the whole
>> instance and start a new one which breaks and reconnects the socket
>> with a clean start and, most of the time, results in a good
>> connection; the external device resets itself through the disconnect.
>> One faulty device, though, generated thousand upon thousand of
>> disconnects over a number of days and, after a few hundred thousand of
>> these I my own system crashed due, I have now found out, to a lack of
>> memory caused by this leak.
>>
>> My application must run essentially as an embedded system, unattended
>> week after week, month after month so I cannot tolerate a memory leak.
>> Does anybody know about this?  Is there a simple clean way to force a
>> socket disconnection on a CAsyncSocket and then reconnect?  My
>> application is the connect() end of the socket, not the listen() end.
>>
>>
>>
>> 
>
Joseph M. Newcomer [MVP]
email: newcomer@flounder.com
Web: http://www.flounder.com
MVP Tips: http://www.flounder.com/mvp_tips.htm
0
newcomer (15974)
7/10/2007 3:12:12 PM
On Tue, 10 Jul 2007 09:45:46 -0500, "AliR \(VC++ MVP\)"
<AliR@online.nospam> wrote:

Thank you for you help.  Perhaps "leak" is the wrong word because my
programs in debug mode never show a problem on exit.  However the
Create method is consuming system memory that is not released back to
the system until the program closes.

The memory consumption is either shown as "Mem Usage" on the Task
Manager Processes page or by the following code:

  PROCESS_MEMORY_COUNTERS pmc;
  HANDLE hProcess = OpenProcess(PROCESS_QUERY_INFORMATION 
       | PROCESS_VM_READ, FALSE, _getpid());
  size_t retval = 0;
  if (GetProcessMemoryInfo(hProcess, &pmc, sizeof(pmc)))
      retval = pmc.WorkingSetSize;
  CloseHandle(hProcess);

The memory used by my process continues to grow and grow indefinitely
even though I close and delete the socket.  That seems intolerable,
whether or not it officially counts as a leak. 

Maybe the Windows memory manager will eventually discover this and
recoup the memory, but how long will it take?  All I know is that I
have a problem in the field when my program runs month after month and
this is the best clue I can generate on my test bed about what might
be the cause.  I now have a version running in the field that logs its
memory usage once a day, but that will take days and weeks to analyze.
The problem is that the CAsyncSocket delete and recreate only occurs
when there are problems in the field, and they occur only
sporadically.

Incidentally, the memory consumption occurs both in debug and release
compilations and for both Unicode and simple char string (Character
Set not set) options.  I tested that because CAsyncSocket::Create uses
T2A_EX within CAsyncSocket::Bind and I wonder whether that might be
the problem.


>I couldn't find a memory leak.  What you are most likely seeing is windows 
>memory managment doing its work.  The Create method is creating a socket 
>object, when when it's freed the memory is not given back to the system 
>right away.
>
>
>(I used this method to detect a leak)
>
>// Declare the variables needed
>#ifdef _DEBUG
>   CMemoryState oldMemState, newMemState, diffMemState;
>   oldMemState.Checkpoint();
>#endif
>
>   for (int i=0; i<10; ++i)
>   {
>      CAsyncSocket *pAS = new CAsyncSocket;
>      pAS->Create();
>      pAS->Close();
>      delete pAS;
>   }
>
>#ifdef _DEBUG
>   newMemState.Checkpoint();
>   if( diffMemState.Difference( oldMemState, newMemState ) )
>   {
>      TRACE( "Memory leaked!\n" );
>   }
>#endif
>
>AliR.
>
>
>"r norman" <r_s_norman@_comcast.net> wrote in message 
>news:2895939efidggi556s7fbje0euhm2jd2d0@4ax.com...
>>I have traced a memory leak problem to CAsyncSocket::Create().  Is
>> this a known problem?  Is there a workaround/solution/fix?  Here is
>> sample code:
>>
>>  for (int i=0; i<m_nReopenCount; ++i) {
>>    CAsyncSocket *pAS = new CAsyncSocket;
>>    pAS->Create();
>>    pAS->Close();
>>    delete pAS;
>> }
>>
>> Running this 1000 times uses up 1200 KBytes of memory, or just over 1
>> KByte per call.  Commenting out the Create() leaves memory clean. (And
>> please don't complain about my bracketing style -- I like it.)
>>
>> I have Visual Studio 2005 Professional version 8.0.
>>
>> Incidentally, I also discovered that the call to Create() is not
>> re-entrant.  My application involves connecting to some 10 to 20
>> external devices and my normal code creates a CWinThread to support
>> each socket, where the socket is created and destroyed only  within
>> the thread routine.  Creating all the threads and starting them up
>> simultaneously meant having multiple instances of
>> CAsyncSocket::Create() being called at the same time, crashing my
>> system (memory access faults).    That one I found and fixed with
>> sentries.  Now I am left with the memory leak.
>>
>> The problem is that I have an rather intricate communication protocol
>> system all self contained so that adding a new hardware device simply
>> means creating a new instance of the whole works.  It runs fine until
>> the external hardware goes haywire, in which case I destruct the whole
>> instance and start a new one which breaks and reconnects the socket
>> with a clean start and, most of the time, results in a good
>> connection; the external device resets itself through the disconnect.
>> One faulty device, though, generated thousand upon thousand of
>> disconnects over a number of days and, after a few hundred thousand of
>> these I my own system crashed due, I have now found out, to a lack of
>> memory caused by this leak.
>>
>> My application must run essentially as an embedded system, unattended
>> week after week, month after month so I cannot tolerate a memory leak.
>> Does anybody know about this?  Is there a simple clean way to force a
>> socket disconnection on a CAsyncSocket and then reconnect?  My
>> application is the connect() end of the socket, not the listen() end.
>>
>>
>>
>> 
>
0
r
7/10/2007 3:32:17 PM
As far as the memory being returned, it will eventually, when the system 
thinks that the program won't need it anymore.
With the example I posted, before the loop my sample app is using 3676K of 
memory after the loop the program is using 4024K of memory but there is no 
memory leak.

As far as Create not being reentrant goes, I am not really sure what could 
be causing that.  There are some many possibilities as to why it doesn't 
work the way he wants it.   Is Close being called before the next call to 
Create?  Is the socket being passed from one thread to another?  Maybe even 
a corrupted install of Visual Studio.

I think that the OP should try and recreate the problem with one socket in 
the main thread.  And go from there.

AliR.


"Joseph M. Newcomer" <newcomer@flounder.com> wrote in message 
news:2c87935kti783rkaf53n5d28th16khebq9@4ax.com...
>I would have thought it *would* be returned.
>
> I'm also concerned about the concurrency problem, because I've not seen 
> that particular
> problem in MFC before.  I'm wondering if there is some storage damage that 
> is causing both
> apparent problems.
> joe
>
> On Tue, 10 Jul 2007 09:45:46 -0500, "AliR \(VC++ MVP\)" 
> <AliR@online.nospam> wrote:
>
>>I couldn't find a memory leak.  What you are most likely seeing is windows
>>memory managment doing its work.  The Create method is creating a socket
>>object, when when it's freed the memory is not given back to the system
>>right away.
>>
>>
>>(I used this method to detect a leak)
>>
>>// Declare the variables needed
>>#ifdef _DEBUG
>>   CMemoryState oldMemState, newMemState, diffMemState;
>>   oldMemState.Checkpoint();
>>#endif
>>
>>   for (int i=0; i<10; ++i)
>>   {
>>      CAsyncSocket *pAS = new CAsyncSocket;
>>      pAS->Create();
>>      pAS->Close();
>>      delete pAS;
>>   }
>>
>>#ifdef _DEBUG
>>   newMemState.Checkpoint();
>>   if( diffMemState.Difference( oldMemState, newMemState ) )
>>   {
>>      TRACE( "Memory leaked!\n" );
>>   }
>>#endif
>>
>>AliR.
>>
>>
>>"r norman" <r_s_norman@_comcast.net> wrote in message
>>news:2895939efidggi556s7fbje0euhm2jd2d0@4ax.com...
>>>I have traced a memory leak problem to CAsyncSocket::Create().  Is
>>> this a known problem?  Is there a workaround/solution/fix?  Here is
>>> sample code:
>>>
>>>  for (int i=0; i<m_nReopenCount; ++i) {
>>>    CAsyncSocket *pAS = new CAsyncSocket;
>>>    pAS->Create();
>>>    pAS->Close();
>>>    delete pAS;
>>> }
>>>
>>> Running this 1000 times uses up 1200 KBytes of memory, or just over 1
>>> KByte per call.  Commenting out the Create() leaves memory clean. (And
>>> please don't complain about my bracketing style -- I like it.)
>>>
>>> I have Visual Studio 2005 Professional version 8.0.
>>>
>>> Incidentally, I also discovered that the call to Create() is not
>>> re-entrant.  My application involves connecting to some 10 to 20
>>> external devices and my normal code creates a CWinThread to support
>>> each socket, where the socket is created and destroyed only  within
>>> the thread routine.  Creating all the threads and starting them up
>>> simultaneously meant having multiple instances of
>>> CAsyncSocket::Create() being called at the same time, crashing my
>>> system (memory access faults).    That one I found and fixed with
>>> sentries.  Now I am left with the memory leak.
>>>
>>> The problem is that I have an rather intricate communication protocol
>>> system all self contained so that adding a new hardware device simply
>>> means creating a new instance of the whole works.  It runs fine until
>>> the external hardware goes haywire, in which case I destruct the whole
>>> instance and start a new one which breaks and reconnects the socket
>>> with a clean start and, most of the time, results in a good
>>> connection; the external device resets itself through the disconnect.
>>> One faulty device, though, generated thousand upon thousand of
>>> disconnects over a number of days and, after a few hundred thousand of
>>> these I my own system crashed due, I have now found out, to a lack of
>>> memory caused by this leak.
>>>
>>> My application must run essentially as an embedded system, unattended
>>> week after week, month after month so I cannot tolerate a memory leak.
>>> Does anybody know about this?  Is there a simple clean way to force a
>>> socket disconnection on a CAsyncSocket and then reconnect?  My
>>> application is the connect() end of the socket, not the listen() end.
>>>
>>>
>>>
>>>
>>
> Joseph M. Newcomer [MVP]
> email: newcomer@flounder.com
> Web: http://www.flounder.com
> MVP Tips: http://www.flounder.com/mvp_tips.htm 


0
AliR3470 (3236)
7/10/2007 3:33:26 PM
On Tue, 10 Jul 2007 11:12:12 -0400, Joseph M. Newcomer
<newcomer@flounder.com> wrote:

The concurrency problem was over a year ago so I am speaking only from
memory and my comments I noted in my code at the time.  I created
sixteen instances of a CWinThread derived class in quick succession,
each of which creates its own instance of a CAsyncSocket and calls
Create.  That produces a system "crash", a term which you (Joe)
scolded me about at the time because it carries no diagnostic
information.  It was, as I recall, a memory access error somewhere
within CAsyncSocket::Create but I don't now remember exactly where. It
showed behavior typical of a race or 'simultaneousness'  problem: two
instances seemed to never have a problem, three or four showed a
problem sometimes, five or six almost always did and seven or more
always did.  I  solved the problem by protecting the call to Create so
that only one could execute at a time.  Since the problem was solved,
I didn't concern myself with the far more serious problem of how such
a situation could continue to exist in the MFC code.


>I would have thought it *would* be returned.
>
>I'm also concerned about the concurrency problem, because I've not seen that particular
>problem in MFC before.  I'm wondering if there is some storage damage that is causing both
>apparent problems.
>					joe
>
>On Tue, 10 Jul 2007 09:45:46 -0500, "AliR \(VC++ MVP\)" <AliR@online.nospam> wrote:
>
>>I couldn't find a memory leak.  What you are most likely seeing is windows 
>>memory managment doing its work.  The Create method is creating a socket 
>>object, when when it's freed the memory is not given back to the system 
>>right away.
>>
>>
>>(I used this method to detect a leak)
>>
>>// Declare the variables needed
>>#ifdef _DEBUG
>>   CMemoryState oldMemState, newMemState, diffMemState;
>>   oldMemState.Checkpoint();
>>#endif
>>
>>   for (int i=0; i<10; ++i)
>>   {
>>      CAsyncSocket *pAS = new CAsyncSocket;
>>      pAS->Create();
>>      pAS->Close();
>>      delete pAS;
>>   }
>>
>>#ifdef _DEBUG
>>   newMemState.Checkpoint();
>>   if( diffMemState.Difference( oldMemState, newMemState ) )
>>   {
>>      TRACE( "Memory leaked!\n" );
>>   }
>>#endif
>>
>>AliR.
>>
>>
>>"r norman" <r_s_norman@_comcast.net> wrote in message 
>>news:2895939efidggi556s7fbje0euhm2jd2d0@4ax.com...
>>>I have traced a memory leak problem to CAsyncSocket::Create().  Is
>>> this a known problem?  Is there a workaround/solution/fix?  Here is
>>> sample code:
>>>
>>>  for (int i=0; i<m_nReopenCount; ++i) {
>>>    CAsyncSocket *pAS = new CAsyncSocket;
>>>    pAS->Create();
>>>    pAS->Close();
>>>    delete pAS;
>>> }
>>>
>>> Running this 1000 times uses up 1200 KBytes of memory, or just over 1
>>> KByte per call.  Commenting out the Create() leaves memory clean. (And
>>> please don't complain about my bracketing style -- I like it.)
>>>
>>> I have Visual Studio 2005 Professional version 8.0.
>>>
>>> Incidentally, I also discovered that the call to Create() is not
>>> re-entrant.  My application involves connecting to some 10 to 20
>>> external devices and my normal code creates a CWinThread to support
>>> each socket, where the socket is created and destroyed only  within
>>> the thread routine.  Creating all the threads and starting them up
>>> simultaneously meant having multiple instances of
>>> CAsyncSocket::Create() being called at the same time, crashing my
>>> system (memory access faults).    That one I found and fixed with
>>> sentries.  Now I am left with the memory leak.
>>>
>>> The problem is that I have an rather intricate communication protocol
>>> system all self contained so that adding a new hardware device simply
>>> means creating a new instance of the whole works.  It runs fine until
>>> the external hardware goes haywire, in which case I destruct the whole
>>> instance and start a new one which breaks and reconnects the socket
>>> with a clean start and, most of the time, results in a good
>>> connection; the external device resets itself through the disconnect.
>>> One faulty device, though, generated thousand upon thousand of
>>> disconnects over a number of days and, after a few hundred thousand of
>>> these I my own system crashed due, I have now found out, to a lack of
>>> memory caused by this leak.
>>>
>>> My application must run essentially as an embedded system, unattended
>>> week after week, month after month so I cannot tolerate a memory leak.
>>> Does anybody know about this?  Is there a simple clean way to force a
>>> socket disconnection on a CAsyncSocket and then reconnect?  My
>>> application is the connect() end of the socket, not the listen() end.
>>>
>>>
>>>
>>> 
>>
>Joseph M. Newcomer [MVP]
>email: newcomer@flounder.com
>Web: http://www.flounder.com
>MVP Tips: http://www.flounder.com/mvp_tips.htm
0
r
7/10/2007 4:08:44 PM
On Tue, 10 Jul 2007 10:33:26 -0500, "AliR \(VC++ MVP\)"
<AliR@online.nospam> wrote:

>As far as the memory being returned, it will eventually, when the system 
>thinks that the program won't need it anymore.
>With the example I posted, before the loop my sample app is using 3676K of 
>memory after the loop the program is using 4024K of memory but there is no 
>memory leak.
>
>As far as Create not being reentrant goes, I am not really sure what could 
>be causing that.  There are some many possibilities as to why it doesn't 
>work the way he wants it.   Is Close being called before the next call to 
>Create?  Is the socket being passed from one thread to another?  Maybe even 
>a corrupted install of Visual Studio.
>
>I think that the OP should try and recreate the problem with one socket in 
>the main thread.  And go from there.

I responded to Joe's post about the concurrency problem and explained
what I was doing there.  It was multiple instances of CAsyncSocket
that was the problem, not successive Creates on the same instance or
sockets shared between threads.  And I found a workaround by
serializing the calls to Create so I left it at that.

I still worry about Windows memory management and whether it is solid
enough to support embedded programs that run continuously for month
after month.  I know for a fact that older versions of Visual C
programs (version 3.0 or 4.0 if my memory serves me well) running on
16 bit MS-DOS  failed miserably -- memory became so fragmented that it
was unusable even if the total available byte count was maintained. At
that time, I had to go to a commercial third party memory manager. I
only reluctantly migrated from C to C++ because of my fears of  the
object oriented code's proclivity to create and delete objects at
will, thereby stressing the memory manager to its limits.  My roots
are in 8 bit embedded systems with really severe memory constraints
and I still have a probably unfounded fear of that problem.





>AliR.
>
>
>"Joseph M. Newcomer" <newcomer@flounder.com> wrote in message 
>news:2c87935kti783rkaf53n5d28th16khebq9@4ax.com...
>>I would have thought it *would* be returned.
>>
>> I'm also concerned about the concurrency problem, because I've not seen 
>> that particular
>> problem in MFC before.  I'm wondering if there is some storage damage that 
>> is causing both
>> apparent problems.
>> joe
>>
>> On Tue, 10 Jul 2007 09:45:46 -0500, "AliR \(VC++ MVP\)" 
>> <AliR@online.nospam> wrote:
>>
>>>I couldn't find a memory leak.  What you are most likely seeing is windows
>>>memory managment doing its work.  The Create method is creating a socket
>>>object, when when it's freed the memory is not given back to the system
>>>right away.
>>>
>>>
>>>(I used this method to detect a leak)
>>>
>>>// Declare the variables needed
>>>#ifdef _DEBUG
>>>   CMemoryState oldMemState, newMemState, diffMemState;
>>>   oldMemState.Checkpoint();
>>>#endif
>>>
>>>   for (int i=0; i<10; ++i)
>>>   {
>>>      CAsyncSocket *pAS = new CAsyncSocket;
>>>      pAS->Create();
>>>      pAS->Close();
>>>      delete pAS;
>>>   }
>>>
>>>#ifdef _DEBUG
>>>   newMemState.Checkpoint();
>>>   if( diffMemState.Difference( oldMemState, newMemState ) )
>>>   {
>>>      TRACE( "Memory leaked!\n" );
>>>   }
>>>#endif
>>>
>>>AliR.
>>>
>>>
>>>"r norman" <r_s_norman@_comcast.net> wrote in message
>>>news:2895939efidggi556s7fbje0euhm2jd2d0@4ax.com...
>>>>I have traced a memory leak problem to CAsyncSocket::Create().  Is
>>>> this a known problem?  Is there a workaround/solution/fix?  Here is
>>>> sample code:
>>>>
>>>>  for (int i=0; i<m_nReopenCount; ++i) {
>>>>    CAsyncSocket *pAS = new CAsyncSocket;
>>>>    pAS->Create();
>>>>    pAS->Close();
>>>>    delete pAS;
>>>> }
>>>>
>>>> Running this 1000 times uses up 1200 KBytes of memory, or just over 1
>>>> KByte per call.  Commenting out the Create() leaves memory clean. (And
>>>> please don't complain about my bracketing style -- I like it.)
>>>>
>>>> I have Visual Studio 2005 Professional version 8.0.
>>>>
>>>> Incidentally, I also discovered that the call to Create() is not
>>>> re-entrant.  My application involves connecting to some 10 to 20
>>>> external devices and my normal code creates a CWinThread to support
>>>> each socket, where the socket is created and destroyed only  within
>>>> the thread routine.  Creating all the threads and starting them up
>>>> simultaneously meant having multiple instances of
>>>> CAsyncSocket::Create() being called at the same time, crashing my
>>>> system (memory access faults).    That one I found and fixed with
>>>> sentries.  Now I am left with the memory leak.
>>>>
>>>> The problem is that I have an rather intricate communication protocol
>>>> system all self contained so that adding a new hardware device simply
>>>> means creating a new instance of the whole works.  It runs fine until
>>>> the external hardware goes haywire, in which case I destruct the whole
>>>> instance and start a new one which breaks and reconnects the socket
>>>> with a clean start and, most of the time, results in a good
>>>> connection; the external device resets itself through the disconnect.
>>>> One faulty device, though, generated thousand upon thousand of
>>>> disconnects over a number of days and, after a few hundred thousand of
>>>> these I my own system crashed due, I have now found out, to a lack of
>>>> memory caused by this leak.
>>>>
>>>> My application must run essentially as an embedded system, unattended
>>>> week after week, month after month so I cannot tolerate a memory leak.
>>>> Does anybody know about this?  Is there a simple clean way to force a
>>>> socket disconnection on a CAsyncSocket and then reconnect?  My
>>>> application is the connect() end of the socket, not the listen() end.
>>>>
>>>>
>>>>
>>>>
>>>
>> Joseph M. Newcomer [MVP]
>> email: newcomer@flounder.com
>> Web: http://www.flounder.com
>> MVP Tips: http://www.flounder.com/mvp_tips.htm 
>
0
r
7/10/2007 4:20:08 PM
hummm, I ran a sample that created 1000 sockets and closed them.  The memory 
usage went up 1 Meg, I watched it for an hour to see when the memory would 
be released, it didn't.

I put a call to WSACleanup right after the loop and no more increase in 
memory.  Seems like whatever memory is allocated by the sockets gets cleaned 
up on after WSACleanup is called!

Ok I'm baffled.

AliR.

"r norman" <r_s_norman@_comcast.net> wrote in message 
news:cpb793pdjqar537ms5nctjavvbqmumgo76@4ax.com...
> On Tue, 10 Jul 2007 10:33:26 -0500, "AliR \(VC++ MVP\)"
> <AliR@online.nospam> wrote:
>
>>As far as the memory being returned, it will eventually, when the system
>>thinks that the program won't need it anymore.
>>With the example I posted, before the loop my sample app is using 3676K of
>>memory after the loop the program is using 4024K of memory but there is no
>>memory leak.
>>
>>As far as Create not being reentrant goes, I am not really sure what could
>>be causing that.  There are some many possibilities as to why it doesn't
>>work the way he wants it.   Is Close being called before the next call to
>>Create?  Is the socket being passed from one thread to another?  Maybe 
>>even
>>a corrupted install of Visual Studio.
>>
>>I think that the OP should try and recreate the problem with one socket in
>>the main thread.  And go from there.
>
> I responded to Joe's post about the concurrency problem and explained
> what I was doing there.  It was multiple instances of CAsyncSocket
> that was the problem, not successive Creates on the same instance or
> sockets shared between threads.  And I found a workaround by
> serializing the calls to Create so I left it at that.
>
> I still worry about Windows memory management and whether it is solid
> enough to support embedded programs that run continuously for month
> after month.  I know for a fact that older versions of Visual C
> programs (version 3.0 or 4.0 if my memory serves me well) running on
> 16 bit MS-DOS  failed miserably -- memory became so fragmented that it
> was unusable even if the total available byte count was maintained. At
> that time, I had to go to a commercial third party memory manager. I
> only reluctantly migrated from C to C++ because of my fears of  the
> object oriented code's proclivity to create and delete objects at
> will, thereby stressing the memory manager to its limits.  My roots
> are in 8 bit embedded systems with really severe memory constraints
> and I still have a probably unfounded fear of that problem.
>
>
>
>
>
>>AliR.
>>
>>
>>"Joseph M. Newcomer" <newcomer@flounder.com> wrote in message
>>news:2c87935kti783rkaf53n5d28th16khebq9@4ax.com...
>>>I would have thought it *would* be returned.
>>>
>>> I'm also concerned about the concurrency problem, because I've not seen
>>> that particular
>>> problem in MFC before.  I'm wondering if there is some storage damage 
>>> that
>>> is causing both
>>> apparent problems.
>>> joe
>>>
>>> On Tue, 10 Jul 2007 09:45:46 -0500, "AliR \(VC++ MVP\)"
>>> <AliR@online.nospam> wrote:
>>>
>>>>I couldn't find a memory leak.  What you are most likely seeing is 
>>>>windows
>>>>memory managment doing its work.  The Create method is creating a socket
>>>>object, when when it's freed the memory is not given back to the system
>>>>right away.
>>>>
>>>>
>>>>(I used this method to detect a leak)
>>>>
>>>>// Declare the variables needed
>>>>#ifdef _DEBUG
>>>>   CMemoryState oldMemState, newMemState, diffMemState;
>>>>   oldMemState.Checkpoint();
>>>>#endif
>>>>
>>>>   for (int i=0; i<10; ++i)
>>>>   {
>>>>      CAsyncSocket *pAS = new CAsyncSocket;
>>>>      pAS->Create();
>>>>      pAS->Close();
>>>>      delete pAS;
>>>>   }
>>>>
>>>>#ifdef _DEBUG
>>>>   newMemState.Checkpoint();
>>>>   if( diffMemState.Difference( oldMemState, newMemState ) )
>>>>   {
>>>>      TRACE( "Memory leaked!\n" );
>>>>   }
>>>>#endif
>>>>
>>>>AliR.
>>>>
>>>>
>>>>"r norman" <r_s_norman@_comcast.net> wrote in message
>>>>news:2895939efidggi556s7fbje0euhm2jd2d0@4ax.com...
>>>>>I have traced a memory leak problem to CAsyncSocket::Create().  Is
>>>>> this a known problem?  Is there a workaround/solution/fix?  Here is
>>>>> sample code:
>>>>>
>>>>>  for (int i=0; i<m_nReopenCount; ++i) {
>>>>>    CAsyncSocket *pAS = new CAsyncSocket;
>>>>>    pAS->Create();
>>>>>    pAS->Close();
>>>>>    delete pAS;
>>>>> }
>>>>>
>>>>> Running this 1000 times uses up 1200 KBytes of memory, or just over 1
>>>>> KByte per call.  Commenting out the Create() leaves memory clean. (And
>>>>> please don't complain about my bracketing style -- I like it.)
>>>>>
>>>>> I have Visual Studio 2005 Professional version 8.0.
>>>>>
>>>>> Incidentally, I also discovered that the call to Create() is not
>>>>> re-entrant.  My application involves connecting to some 10 to 20
>>>>> external devices and my normal code creates a CWinThread to support
>>>>> each socket, where the socket is created and destroyed only  within
>>>>> the thread routine.  Creating all the threads and starting them up
>>>>> simultaneously meant having multiple instances of
>>>>> CAsyncSocket::Create() being called at the same time, crashing my
>>>>> system (memory access faults).    That one I found and fixed with
>>>>> sentries.  Now I am left with the memory leak.
>>>>>
>>>>> The problem is that I have an rather intricate communication protocol
>>>>> system all self contained so that adding a new hardware device simply
>>>>> means creating a new instance of the whole works.  It runs fine until
>>>>> the external hardware goes haywire, in which case I destruct the whole
>>>>> instance and start a new one which breaks and reconnects the socket
>>>>> with a clean start and, most of the time, results in a good
>>>>> connection; the external device resets itself through the disconnect.
>>>>> One faulty device, though, generated thousand upon thousand of
>>>>> disconnects over a number of days and, after a few hundred thousand of
>>>>> these I my own system crashed due, I have now found out, to a lack of
>>>>> memory caused by this leak.
>>>>>
>>>>> My application must run essentially as an embedded system, unattended
>>>>> week after week, month after month so I cannot tolerate a memory leak.
>>>>> Does anybody know about this?  Is there a simple clean way to force a
>>>>> socket disconnection on a CAsyncSocket and then reconnect?  My
>>>>> application is the connect() end of the socket, not the listen() end.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>> Joseph M. Newcomer [MVP]
>>> email: newcomer@flounder.com
>>> Web: http://www.flounder.com
>>> MVP Tips: http://www.flounder.com/mvp_tips.htm
>> 


0
AliR3470 (3236)
7/10/2007 4:45:00 PM
On Tue, 10 Jul 2007 10:33:26 -0500, "AliR \(VC++ MVP\)" <AliR@online.nospam> wrote:

>As far as the memory being returned, it will eventually, when the system 
>thinks that the program won't need it anymore.
****
But that's what 'delete' is saying...
****
>With the example I posted, before the loop my sample app is using 3676K of 
>memory after the loop the program is using 4024K of memory but there is no 
>memory leak.
****
Based on what diagnostic tool?  Memory used as memory footprint is not the same as memory
used because it is allocated; in fact, there is no mechanism I am aware of for the program
to release memory to the operating system once it has been granted to the heap.
*****
>
>As far as Create not being reentrant goes, I am not really sure what could 
>be causing that.  There are some many possibilities as to why it doesn't 
>work the way he wants it.   Is Close being called before the next call to 
>Create?  Is the socket being passed from one thread to another?  Maybe even 
>a corrupted install of Visual Studio.
*****
Most of MFC is thread-safe insofar as its own internal data structures; what is not
thread-safe is the user-visible structures.  So if I access a CString or CArray without
synchronization, that is an error.  But if MFC uses a CArray, CMap, etc. internally,
either it is used exclusively by one thread or it is handled with synchronization.  That's
what worries me: that there is some unprotected structure inside MFC that we should not be
depending on without external synchronization.
					joe
*****
>
>I think that the OP should try and recreate the problem with one socket in 
>the main thread.  And go from there.
>
>AliR.
>
>
>"Joseph M. Newcomer" <newcomer@flounder.com> wrote in message 
>news:2c87935kti783rkaf53n5d28th16khebq9@4ax.com...
>>I would have thought it *would* be returned.
>>
>> I'm also concerned about the concurrency problem, because I've not seen 
>> that particular
>> problem in MFC before.  I'm wondering if there is some storage damage that 
>> is causing both
>> apparent problems.
>> joe
>>
>> On Tue, 10 Jul 2007 09:45:46 -0500, "AliR \(VC++ MVP\)" 
>> <AliR@online.nospam> wrote:
>>
>>>I couldn't find a memory leak.  What you are most likely seeing is windows
>>>memory managment doing its work.  The Create method is creating a socket
>>>object, when when it's freed the memory is not given back to the system
>>>right away.
>>>
>>>
>>>(I used this method to detect a leak)
>>>
>>>// Declare the variables needed
>>>#ifdef _DEBUG
>>>   CMemoryState oldMemState, newMemState, diffMemState;
>>>   oldMemState.Checkpoint();
>>>#endif
>>>
>>>   for (int i=0; i<10; ++i)
>>>   {
>>>      CAsyncSocket *pAS = new CAsyncSocket;
>>>      pAS->Create();
>>>      pAS->Close();
>>>      delete pAS;
>>>   }
>>>
>>>#ifdef _DEBUG
>>>   newMemState.Checkpoint();
>>>   if( diffMemState.Difference( oldMemState, newMemState ) )
>>>   {
>>>      TRACE( "Memory leaked!\n" );
>>>   }
>>>#endif
>>>
>>>AliR.
>>>
>>>
>>>"r norman" <r_s_norman@_comcast.net> wrote in message
>>>news:2895939efidggi556s7fbje0euhm2jd2d0@4ax.com...
>>>>I have traced a memory leak problem to CAsyncSocket::Create().  Is
>>>> this a known problem?  Is there a workaround/solution/fix?  Here is
>>>> sample code:
>>>>
>>>>  for (int i=0; i<m_nReopenCount; ++i) {
>>>>    CAsyncSocket *pAS = new CAsyncSocket;
>>>>    pAS->Create();
>>>>    pAS->Close();
>>>>    delete pAS;
>>>> }
>>>>
>>>> Running this 1000 times uses up 1200 KBytes of memory, or just over 1
>>>> KByte per call.  Commenting out the Create() leaves memory clean. (And
>>>> please don't complain about my bracketing style -- I like it.)
>>>>
>>>> I have Visual Studio 2005 Professional version 8.0.
>>>>
>>>> Incidentally, I also discovered that the call to Create() is not
>>>> re-entrant.  My application involves connecting to some 10 to 20
>>>> external devices and my normal code creates a CWinThread to support
>>>> each socket, where the socket is created and destroyed only  within
>>>> the thread routine.  Creating all the threads and starting them up
>>>> simultaneously meant having multiple instances of
>>>> CAsyncSocket::Create() being called at the same time, crashing my
>>>> system (memory access faults).    That one I found and fixed with
>>>> sentries.  Now I am left with the memory leak.
>>>>
>>>> The problem is that I have an rather intricate communication protocol
>>>> system all self contained so that adding a new hardware device simply
>>>> means creating a new instance of the whole works.  It runs fine until
>>>> the external hardware goes haywire, in which case I destruct the whole
>>>> instance and start a new one which breaks and reconnects the socket
>>>> with a clean start and, most of the time, results in a good
>>>> connection; the external device resets itself through the disconnect.
>>>> One faulty device, though, generated thousand upon thousand of
>>>> disconnects over a number of days and, after a few hundred thousand of
>>>> these I my own system crashed due, I have now found out, to a lack of
>>>> memory caused by this leak.
>>>>
>>>> My application must run essentially as an embedded system, unattended
>>>> week after week, month after month so I cannot tolerate a memory leak.
>>>> Does anybody know about this?  Is there a simple clean way to force a
>>>> socket disconnection on a CAsyncSocket and then reconnect?  My
>>>> application is the connect() end of the socket, not the listen() end.
>>>>
>>>>
>>>>
>>>>
>>>
>> Joseph M. Newcomer [MVP]
>> email: newcomer@flounder.com
>> Web: http://www.flounder.com
>> MVP Tips: http://www.flounder.com/mvp_tips.htm 
>
Joseph M. Newcomer [MVP]
email: newcomer@flounder.com
Web: http://www.flounder.com
MVP Tips: http://www.flounder.com/mvp_tips.htm
0
newcomer (15974)
7/10/2007 4:45:25 PM
See below...
On Tue, 10 Jul 2007 12:20:08 -0400, r norman <r_s_norman@_comcast.net> wrote:

>On Tue, 10 Jul 2007 10:33:26 -0500, "AliR \(VC++ MVP\)"
><AliR@online.nospam> wrote:
>
>>As far as the memory being returned, it will eventually, when the system 
>>thinks that the program won't need it anymore.
>>With the example I posted, before the loop my sample app is using 3676K of 
>>memory after the loop the program is using 4024K of memory but there is no 
>>memory leak.
>>
>>As far as Create not being reentrant goes, I am not really sure what could 
>>be causing that.  There are some many possibilities as to why it doesn't 
>>work the way he wants it.   Is Close being called before the next call to 
>>Create?  Is the socket being passed from one thread to another?  Maybe even 
>>a corrupted install of Visual Studio.
>>
>>I think that the OP should try and recreate the problem with one socket in 
>>the main thread.  And go from there.
>
>I responded to Joe's post about the concurrency problem and explained
>what I was doing there.  It was multiple instances of CAsyncSocket
>that was the problem, not successive Creates on the same instance or
>sockets shared between threads.  And I found a workaround by
>serializing the calls to Create so I left it at that.
>
>I still worry about Windows memory management and whether it is solid
>enough to support embedded programs that run continuously for month
>after month.  
****
There are several levels of memory management going on here: the low-level VirtualAlloc
level, the LocalAlloc/GlobalAlloc memory, malloc/free (which are used by new/delete) and
MFC and its internal data structures.  
****
>I know for a fact that older versions of Visual C
>programs (version 3.0 or 4.0 if my memory serves me well) running on
>16 bit MS-DOS  failed miserably -- memory became so fragmented that it
>was unusable even if the total available byte count was maintained. 
*****
Memory fragmentation is the real issue here; it has nothing to do with "robustness", which
is a metric for reliability.  A highly-fragmented memory can still be robust, but there is
nothing can be done at the simplistic level to change fragmentation.  It is true that the
current high-level allocator (malloc/free) tends to be lower-fragmentation, but one of the
reasons I try to avoid using MFC in services is the fact that there is no control at all
over fragmentation problems.  However, VS.NET and beyond allow you to replug the MFC
allocator mechanism, or so I've read, but I've not written any services since its release
and therefore haven't investigated it.
*****
>At
>that time, I had to go to a commercial third party memory manager. I
>only reluctantly migrated from C to C++ because of my fears of  the
>object oriented code's proclivity to create and delete objects at
>will, thereby stressing the memory manager to its limits.  My roots
>are in 8 bit embedded systems with really severe memory constraints
>and I still have a probably unfounded fear of that problem.
****
It is a real problem with C++ because there is no layer you can introduce transparently to
put your own manager that would allow compaction underneath it.  But if you eliminate
compaction, you can do a number of things to reduce fragmentation, but they involve either
replacing the basic allocator (the rumored capability I have not investigated) or live
with MFC allocation for its internal objects but use your own allocator for the objects
you manipulate.  As long as they are your own objects, it's a lot easier.  For example,
you can keep a table of indirection pointers to locate the objects, but the objects
themselves can use either only anchored pointers or the pointers to your own indirection
table.

The simplest solution is to do all your allocations using your own allocator strategy. Use
only one size of object, the max of all objects.  The effect here is that every object is
the same size so all holes are the same size, so there is no fragmentation.

Note that these are not related to the basic OS strategy, but to higher-level strategies,
and "robustness" is not at all the same as "non-fragmenting".

Memory management is one of the nightmares of long-term services, and fragmentation is the
worst problem you can have after leaks.  I've done a number of long-term services, and I
would say that the major effort I invested was in avoiding memory fragmentation.  The
issue is not the nature of the basic allocator, but the fact that allocators are based on
certain models that do not necessarily provide the best support for long-term
applications.
				joe

*****
>
>
>
>
>
>>AliR.
>>
>>
>>"Joseph M. Newcomer" <newcomer@flounder.com> wrote in message 
>>news:2c87935kti783rkaf53n5d28th16khebq9@4ax.com...
>>>I would have thought it *would* be returned.
>>>
>>> I'm also concerned about the concurrency problem, because I've not seen 
>>> that particular
>>> problem in MFC before.  I'm wondering if there is some storage damage that 
>>> is causing both
>>> apparent problems.
>>> joe
>>>
>>> On Tue, 10 Jul 2007 09:45:46 -0500, "AliR \(VC++ MVP\)" 
>>> <AliR@online.nospam> wrote:
>>>
>>>>I couldn't find a memory leak.  What you are most likely seeing is windows
>>>>memory managment doing its work.  The Create method is creating a socket
>>>>object, when when it's freed the memory is not given back to the system
>>>>right away.
>>>>
>>>>
>>>>(I used this method to detect a leak)
>>>>
>>>>// Declare the variables needed
>>>>#ifdef _DEBUG
>>>>   CMemoryState oldMemState, newMemState, diffMemState;
>>>>   oldMemState.Checkpoint();
>>>>#endif
>>>>
>>>>   for (int i=0; i<10; ++i)
>>>>   {
>>>>      CAsyncSocket *pAS = new CAsyncSocket;
>>>>      pAS->Create();
>>>>      pAS->Close();
>>>>      delete pAS;
>>>>   }
>>>>
>>>>#ifdef _DEBUG
>>>>   newMemState.Checkpoint();
>>>>   if( diffMemState.Difference( oldMemState, newMemState ) )
>>>>   {
>>>>      TRACE( "Memory leaked!\n" );
>>>>   }
>>>>#endif
>>>>
>>>>AliR.
>>>>
>>>>
>>>>"r norman" <r_s_norman@_comcast.net> wrote in message
>>>>news:2895939efidggi556s7fbje0euhm2jd2d0@4ax.com...
>>>>>I have traced a memory leak problem to CAsyncSocket::Create().  Is
>>>>> this a known problem?  Is there a workaround/solution/fix?  Here is
>>>>> sample code:
>>>>>
>>>>>  for (int i=0; i<m_nReopenCount; ++i) {
>>>>>    CAsyncSocket *pAS = new CAsyncSocket;
>>>>>    pAS->Create();
>>>>>    pAS->Close();
>>>>>    delete pAS;
>>>>> }
>>>>>
>>>>> Running this 1000 times uses up 1200 KBytes of memory, or just over 1
>>>>> KByte per call.  Commenting out the Create() leaves memory clean. (And
>>>>> please don't complain about my bracketing style -- I like it.)
>>>>>
>>>>> I have Visual Studio 2005 Professional version 8.0.
>>>>>
>>>>> Incidentally, I also discovered that the call to Create() is not
>>>>> re-entrant.  My application involves connecting to some 10 to 20
>>>>> external devices and my normal code creates a CWinThread to support
>>>>> each socket, where the socket is created and destroyed only  within
>>>>> the thread routine.  Creating all the threads and starting them up
>>>>> simultaneously meant having multiple instances of
>>>>> CAsyncSocket::Create() being called at the same time, crashing my
>>>>> system (memory access faults).    That one I found and fixed with
>>>>> sentries.  Now I am left with the memory leak.
>>>>>
>>>>> The problem is that I have an rather intricate communication protocol
>>>>> system all self contained so that adding a new hardware device simply
>>>>> means creating a new instance of the whole works.  It runs fine until
>>>>> the external hardware goes haywire, in which case I destruct the whole
>>>>> instance and start a new one which breaks and reconnects the socket
>>>>> with a clean start and, most of the time, results in a good
>>>>> connection; the external device resets itself through the disconnect.
>>>>> One faulty device, though, generated thousand upon thousand of
>>>>> disconnects over a number of days and, after a few hundred thousand of
>>>>> these I my own system crashed due, I have now found out, to a lack of
>>>>> memory caused by this leak.
>>>>>
>>>>> My application must run essentially as an embedded system, unattended
>>>>> week after week, month after month so I cannot tolerate a memory leak.
>>>>> Does anybody know about this?  Is there a simple clean way to force a
>>>>> socket disconnection on a CAsyncSocket and then reconnect?  My
>>>>> application is the connect() end of the socket, not the listen() end.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>> Joseph M. Newcomer [MVP]
>>> email: newcomer@flounder.com
>>> Web: http://www.flounder.com
>>> MVP Tips: http://www.flounder.com/mvp_tips.htm 
>>
Joseph M. Newcomer [MVP]
email: newcomer@flounder.com
Web: http://www.flounder.com
MVP Tips: http://www.flounder.com/mvp_tips.htm
0
newcomer (15974)
7/10/2007 5:01:33 PM
On Tue, 10 Jul 2007 12:45:25 -0400, Joseph M. Newcomer
<newcomer@flounder.com> wrote:

I can accept that memory allocated to my process won't be returned to
the system.  But when my process is done with it and needs more,
shouldn't the memory manager take from my own internal unused heap and
not ask the system for more?  Where does it end?  When the system is
exhausted?

>On Tue, 10 Jul 2007 10:33:26 -0500, "AliR \(VC++ MVP\)" <AliR@online.nospam> wrote:
>
>>As far as the memory being returned, it will eventually, when the system 
>>thinks that the program won't need it anymore.
>****
>But that's what 'delete' is saying...
>****
>>With the example I posted, before the loop my sample app is using 3676K of 
>>memory after the loop the program is using 4024K of memory but there is no 
>>memory leak.
>****
>Based on what diagnostic tool?  Memory used as memory footprint is not the same as memory
>used because it is allocated; in fact, there is no mechanism I am aware of for the program
>to release memory to the operating system once it has been granted to the heap.
>*****
>>
>>As far as Create not being reentrant goes, I am not really sure what could 
>>be causing that.  There are some many possibilities as to why it doesn't 
>>work the way he wants it.   Is Close being called before the next call to 
>>Create?  Is the socket being passed from one thread to another?  Maybe even 
>>a corrupted install of Visual Studio.
>*****
>Most of MFC is thread-safe insofar as its own internal data structures; what is not
>thread-safe is the user-visible structures.  So if I access a CString or CArray without
>synchronization, that is an error.  But if MFC uses a CArray, CMap, etc. internally,
>either it is used exclusively by one thread or it is handled with synchronization.  That's
>what worries me: that there is some unprotected structure inside MFC that we should not be
>depending on without external synchronization.
>					joe
>*****
>>
>>I think that the OP should try and recreate the problem with one socket in 
>>the main thread.  And go from there.
>>
>>AliR.
>>
>>
>>"Joseph M. Newcomer" <newcomer@flounder.com> wrote in message 
>>news:2c87935kti783rkaf53n5d28th16khebq9@4ax.com...
>>>I would have thought it *would* be returned.
>>>
>>> I'm also concerned about the concurrency problem, because I've not seen 
>>> that particular
>>> problem in MFC before.  I'm wondering if there is some storage damage that 
>>> is causing both
>>> apparent problems.
>>> joe
>>>
>>> On Tue, 10 Jul 2007 09:45:46 -0500, "AliR \(VC++ MVP\)" 
>>> <AliR@online.nospam> wrote:
>>>
>>>>I couldn't find a memory leak.  What you are most likely seeing is windows
>>>>memory managment doing its work.  The Create method is creating a socket
>>>>object, when when it's freed the memory is not given back to the system
>>>>right away.
>>>>
>>>>
>>>>(I used this method to detect a leak)
>>>>
>>>>// Declare the variables needed
>>>>#ifdef _DEBUG
>>>>   CMemoryState oldMemState, newMemState, diffMemState;
>>>>   oldMemState.Checkpoint();
>>>>#endif
>>>>
>>>>   for (int i=0; i<10; ++i)
>>>>   {
>>>>      CAsyncSocket *pAS = new CAsyncSocket;
>>>>      pAS->Create();
>>>>      pAS->Close();
>>>>      delete pAS;
>>>>   }
>>>>
>>>>#ifdef _DEBUG
>>>>   newMemState.Checkpoint();
>>>>   if( diffMemState.Difference( oldMemState, newMemState ) )
>>>>   {
>>>>      TRACE( "Memory leaked!\n" );
>>>>   }
>>>>#endif
>>>>
>>>>AliR.
>>>>
>>>>
>>>>"r norman" <r_s_norman@_comcast.net> wrote in message
>>>>news:2895939efidggi556s7fbje0euhm2jd2d0@4ax.com...
>>>>>I have traced a memory leak problem to CAsyncSocket::Create().  Is
>>>>> this a known problem?  Is there a workaround/solution/fix?  Here is
>>>>> sample code:
>>>>>
>>>>>  for (int i=0; i<m_nReopenCount; ++i) {
>>>>>    CAsyncSocket *pAS = new CAsyncSocket;
>>>>>    pAS->Create();
>>>>>    pAS->Close();
>>>>>    delete pAS;
>>>>> }
>>>>>
>>>>> Running this 1000 times uses up 1200 KBytes of memory, or just over 1
>>>>> KByte per call.  Commenting out the Create() leaves memory clean. (And
>>>>> please don't complain about my bracketing style -- I like it.)
>>>>>
>>>>> I have Visual Studio 2005 Professional version 8.0.
>>>>>
>>>>> Incidentally, I also discovered that the call to Create() is not
>>>>> re-entrant.  My application involves connecting to some 10 to 20
>>>>> external devices and my normal code creates a CWinThread to support
>>>>> each socket, where the socket is created and destroyed only  within
>>>>> the thread routine.  Creating all the threads and starting them up
>>>>> simultaneously meant having multiple instances of
>>>>> CAsyncSocket::Create() being called at the same time, crashing my
>>>>> system (memory access faults).    That one I found and fixed with
>>>>> sentries.  Now I am left with the memory leak.
>>>>>
>>>>> The problem is that I have an rather intricate communication protocol
>>>>> system all self contained so that adding a new hardware device simply
>>>>> means creating a new instance of the whole works.  It runs fine until
>>>>> the external hardware goes haywire, in which case I destruct the whole
>>>>> instance and start a new one which breaks and reconnects the socket
>>>>> with a clean start and, most of the time, results in a good
>>>>> connection; the external device resets itself through the disconnect.
>>>>> One faulty device, though, generated thousand upon thousand of
>>>>> disconnects over a number of days and, after a few hundred thousand of
>>>>> these I my own system crashed due, I have now found out, to a lack of
>>>>> memory caused by this leak.
>>>>>
>>>>> My application must run essentially as an embedded system, unattended
>>>>> week after week, month after month so I cannot tolerate a memory leak.
>>>>> Does anybody know about this?  Is there a simple clean way to force a
>>>>> socket disconnection on a CAsyncSocket and then reconnect?  My
>>>>> application is the connect() end of the socket, not the listen() end.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>> Joseph M. Newcomer [MVP]
>>> email: newcomer@flounder.com
>>> Web: http://www.flounder.com
>>> MVP Tips: http://www.flounder.com/mvp_tips.htm 
>>
>Joseph M. Newcomer [MVP]
>email: newcomer@flounder.com
>Web: http://www.flounder.com
>MVP Tips: http://www.flounder.com/mvp_tips.htm
0
r
7/10/2007 5:07:32 PM
On Tue, 10 Jul 2007 11:45:00 -0500, "AliR \(VC++ MVP\)"
<AliR@online.nospam> wrote:

Incidentally, the first time I create and delete CAsyncSocket, it uses
312K of system memory.  After that it uses only 1K each time.  I am
not concerned about one-time memory usage, only about continuous
nibbles eating away system memory.

>hummm, I ran a sample that created 1000 sockets and closed them.  The memory 
>usage went up 1 Meg, I watched it for an hour to see when the memory would 
>be released, it didn't.
>
>I put a call to WSACleanup right after the loop and no more increase in 
>memory.  Seems like whatever memory is allocated by the sockets gets cleaned 
>up on after WSACleanup is called!
>
>Ok I'm baffled.
>
>AliR.
>
>"r norman" <r_s_norman@_comcast.net> wrote in message 
>news:cpb793pdjqar537ms5nctjavvbqmumgo76@4ax.com...
>> On Tue, 10 Jul 2007 10:33:26 -0500, "AliR \(VC++ MVP\)"
>> <AliR@online.nospam> wrote:
>>
>>>As far as the memory being returned, it will eventually, when the system
>>>thinks that the program won't need it anymore.
>>>With the example I posted, before the loop my sample app is using 3676K of
>>>memory after the loop the program is using 4024K of memory but there is no
>>>memory leak.
>>>
>>>As far as Create not being reentrant goes, I am not really sure what could
>>>be causing that.  There are some many possibilities as to why it doesn't
>>>work the way he wants it.   Is Close being called before the next call to
>>>Create?  Is the socket being passed from one thread to another?  Maybe 
>>>even
>>>a corrupted install of Visual Studio.
>>>
>>>I think that the OP should try and recreate the problem with one socket in
>>>the main thread.  And go from there.
>>
>> I responded to Joe's post about the concurrency problem and explained
>> what I was doing there.  It was multiple instances of CAsyncSocket
>> that was the problem, not successive Creates on the same instance or
>> sockets shared between threads.  And I found a workaround by
>> serializing the calls to Create so I left it at that.
>>
>> I still worry about Windows memory management and whether it is solid
>> enough to support embedded programs that run continuously for month
>> after month.  I know for a fact that older versions of Visual C
>> programs (version 3.0 or 4.0 if my memory serves me well) running on
>> 16 bit MS-DOS  failed miserably -- memory became so fragmented that it
>> was unusable even if the total available byte count was maintained. At
>> that time, I had to go to a commercial third party memory manager. I
>> only reluctantly migrated from C to C++ because of my fears of  the
>> object oriented code's proclivity to create and delete objects at
>> will, thereby stressing the memory manager to its limits.  My roots
>> are in 8 bit embedded systems with really severe memory constraints
>> and I still have a probably unfounded fear of that problem.
>>
>>
>>
>>
>>
>>>AliR.
>>>
>>>
>>>"Joseph M. Newcomer" <newcomer@flounder.com> wrote in message
>>>news:2c87935kti783rkaf53n5d28th16khebq9@4ax.com...
>>>>I would have thought it *would* be returned.
>>>>
>>>> I'm also concerned about the concurrency problem, because I've not seen
>>>> that particular
>>>> problem in MFC before.  I'm wondering if there is some storage damage 
>>>> that
>>>> is causing both
>>>> apparent problems.
>>>> joe
>>>>
>>>> On Tue, 10 Jul 2007 09:45:46 -0500, "AliR \(VC++ MVP\)"
>>>> <AliR@online.nospam> wrote:
>>>>
>>>>>I couldn't find a memory leak.  What you are most likely seeing is 
>>>>>windows
>>>>>memory managment doing its work.  The Create method is creating a socket
>>>>>object, when when it's freed the memory is not given back to the system
>>>>>right away.
>>>>>
>>>>>
>>>>>(I used this method to detect a leak)
>>>>>
>>>>>// Declare the variables needed
>>>>>#ifdef _DEBUG
>>>>>   CMemoryState oldMemState, newMemState, diffMemState;
>>>>>   oldMemState.Checkpoint();
>>>>>#endif
>>>>>
>>>>>   for (int i=0; i<10; ++i)
>>>>>   {
>>>>>      CAsyncSocket *pAS = new CAsyncSocket;
>>>>>      pAS->Create();
>>>>>      pAS->Close();
>>>>>      delete pAS;
>>>>>   }
>>>>>
>>>>>#ifdef _DEBUG
>>>>>   newMemState.Checkpoint();
>>>>>   if( diffMemState.Difference( oldMemState, newMemState ) )
>>>>>   {
>>>>>      TRACE( "Memory leaked!\n" );
>>>>>   }
>>>>>#endif
>>>>>
>>>>>AliR.
>>>>>
>>>>>
>>>>>"r norman" <r_s_norman@_comcast.net> wrote in message
>>>>>news:2895939efidggi556s7fbje0euhm2jd2d0@4ax.com...
>>>>>>I have traced a memory leak problem to CAsyncSocket::Create().  Is
>>>>>> this a known problem?  Is there a workaround/solution/fix?  Here is
>>>>>> sample code:
>>>>>>
>>>>>>  for (int i=0; i<m_nReopenCount; ++i) {
>>>>>>    CAsyncSocket *pAS = new CAsyncSocket;
>>>>>>    pAS->Create();
>>>>>>    pAS->Close();
>>>>>>    delete pAS;
>>>>>> }
>>>>>>
>>>>>> Running this 1000 times uses up 1200 KBytes of memory, or just over 1
>>>>>> KByte per call.  Commenting out the Create() leaves memory clean. (And
>>>>>> please don't complain about my bracketing style -- I like it.)
>>>>>>
>>>>>> I have Visual Studio 2005 Professional version 8.0.
>>>>>>
>>>>>> Incidentally, I also discovered that the call to Create() is not
>>>>>> re-entrant.  My application involves connecting to some 10 to 20
>>>>>> external devices and my normal code creates a CWinThread to support
>>>>>> each socket, where the socket is created and destroyed only  within
>>>>>> the thread routine.  Creating all the threads and starting them up
>>>>>> simultaneously meant having multiple instances of
>>>>>> CAsyncSocket::Create() being called at the same time, crashing my
>>>>>> system (memory access faults).    That one I found and fixed with
>>>>>> sentries.  Now I am left with the memory leak.
>>>>>>
>>>>>> The problem is that I have an rather intricate communication protocol
>>>>>> system all self contained so that adding a new hardware device simply
>>>>>> means creating a new instance of the whole works.  It runs fine until
>>>>>> the external hardware goes haywire, in which case I destruct the whole
>>>>>> instance and start a new one which breaks and reconnects the socket
>>>>>> with a clean start and, most of the time, results in a good
>>>>>> connection; the external device resets itself through the disconnect.
>>>>>> One faulty device, though, generated thousand upon thousand of
>>>>>> disconnects over a number of days and, after a few hundred thousand of
>>>>>> these I my own system crashed due, I have now found out, to a lack of
>>>>>> memory caused by this leak.
>>>>>>
>>>>>> My application must run essentially as an embedded system, unattended
>>>>>> week after week, month after month so I cannot tolerate a memory leak.
>>>>>> Does anybody know about this?  Is there a simple clean way to force a
>>>>>> socket disconnection on a CAsyncSocket and then reconnect?  My
>>>>>> application is the connect() end of the socket, not the listen() end.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>> Joseph M. Newcomer [MVP]
>>>> email: newcomer@flounder.com
>>>> Web: http://www.flounder.com
>>>> MVP Tips: http://www.flounder.com/mvp_tips.htm
>>> 
>
0
r
7/10/2007 5:08:46 PM
Part of this worries me, because a lot of us depend on the internal integrity of MFC in
such cases.  So you may have located a serious bug in MFC; concurrency at this level
should not cause any malfunction.  If it does, it would be a bug.
				joe

On Tue, 10 Jul 2007 12:08:44 -0400, r norman <r_s_norman@_comcast.net> wrote:

>On Tue, 10 Jul 2007 11:12:12 -0400, Joseph M. Newcomer
><newcomer@flounder.com> wrote:
>
>The concurrency problem was over a year ago so I am speaking only from
>memory and my comments I noted in my code at the time.  I created
>sixteen instances of a CWinThread derived class in quick succession,
>each of which creates its own instance of a CAsyncSocket and calls
>Create.  That produces a system "crash", a term which you (Joe)
>scolded me about at the time because it carries no diagnostic
>information.  It was, as I recall, a memory access error somewhere
>within CAsyncSocket::Create but I don't now remember exactly where. It
>showed behavior typical of a race or 'simultaneousness'  problem: two
>instances seemed to never have a problem, three or four showed a
>problem sometimes, five or six almost always did and seven or more
>always did.  I  solved the problem by protecting the call to Create so
>that only one could execute at a time.  Since the problem was solved,
>I didn't concern myself with the far more serious problem of how such
>a situation could continue to exist in the MFC code.
>
>
>>I would have thought it *would* be returned.
>>
>>I'm also concerned about the concurrency problem, because I've not seen that particular
>>problem in MFC before.  I'm wondering if there is some storage damage that is causing both
>>apparent problems.
>>					joe
>>
>>On Tue, 10 Jul 2007 09:45:46 -0500, "AliR \(VC++ MVP\)" <AliR@online.nospam> wrote:
>>
>>>I couldn't find a memory leak.  What you are most likely seeing is windows 
>>>memory managment doing its work.  The Create method is creating a socket 
>>>object, when when it's freed the memory is not given back to the system 
>>>right away.
>>>
>>>
>>>(I used this method to detect a leak)
>>>
>>>// Declare the variables needed
>>>#ifdef _DEBUG
>>>   CMemoryState oldMemState, newMemState, diffMemState;
>>>   oldMemState.Checkpoint();
>>>#endif
>>>
>>>   for (int i=0; i<10; ++i)
>>>   {
>>>      CAsyncSocket *pAS = new CAsyncSocket;
>>>      pAS->Create();
>>>      pAS->Close();
>>>      delete pAS;
>>>   }
>>>
>>>#ifdef _DEBUG
>>>   newMemState.Checkpoint();
>>>   if( diffMemState.Difference( oldMemState, newMemState ) )
>>>   {
>>>      TRACE( "Memory leaked!\n" );
>>>   }
>>>#endif
>>>
>>>AliR.
>>>
>>>
>>>"r norman" <r_s_norman@_comcast.net> wrote in message 
>>>news:2895939efidggi556s7fbje0euhm2jd2d0@4ax.com...
>>>>I have traced a memory leak problem to CAsyncSocket::Create().  Is
>>>> this a known problem?  Is there a workaround/solution/fix?  Here is
>>>> sample code:
>>>>
>>>>  for (int i=0; i<m_nReopenCount; ++i) {
>>>>    CAsyncSocket *pAS = new CAsyncSocket;
>>>>    pAS->Create();
>>>>    pAS->Close();
>>>>    delete pAS;
>>>> }
>>>>
>>>> Running this 1000 times uses up 1200 KBytes of memory, or just over 1
>>>> KByte per call.  Commenting out the Create() leaves memory clean. (And
>>>> please don't complain about my bracketing style -- I like it.)
>>>>
>>>> I have Visual Studio 2005 Professional version 8.0.
>>>>
>>>> Incidentally, I also discovered that the call to Create() is not
>>>> re-entrant.  My application involves connecting to some 10 to 20
>>>> external devices and my normal code creates a CWinThread to support
>>>> each socket, where the socket is created and destroyed only  within
>>>> the thread routine.  Creating all the threads and starting them up
>>>> simultaneously meant having multiple instances of
>>>> CAsyncSocket::Create() being called at the same time, crashing my
>>>> system (memory access faults).    That one I found and fixed with
>>>> sentries.  Now I am left with the memory leak.
>>>>
>>>> The problem is that I have an rather intricate communication protocol
>>>> system all self contained so that adding a new hardware device simply
>>>> means creating a new instance of the whole works.  It runs fine until
>>>> the external hardware goes haywire, in which case I destruct the whole
>>>> instance and start a new one which breaks and reconnects the socket
>>>> with a clean start and, most of the time, results in a good
>>>> connection; the external device resets itself through the disconnect.
>>>> One faulty device, though, generated thousand upon thousand of
>>>> disconnects over a number of days and, after a few hundred thousand of
>>>> these I my own system crashed due, I have now found out, to a lack of
>>>> memory caused by this leak.
>>>>
>>>> My application must run essentially as an embedded system, unattended
>>>> week after week, month after month so I cannot tolerate a memory leak.
>>>> Does anybody know about this?  Is there a simple clean way to force a
>>>> socket disconnection on a CAsyncSocket and then reconnect?  My
>>>> application is the connect() end of the socket, not the listen() end.
>>>>
>>>>
>>>>
>>>> 
>>>
>>Joseph M. Newcomer [MVP]
>>email: newcomer@flounder.com
>>Web: http://www.flounder.com
>>MVP Tips: http://www.flounder.com/mvp_tips.htm
Joseph M. Newcomer [MVP]
email: newcomer@flounder.com
Web: http://www.flounder.com
MVP Tips: http://www.flounder.com/mvp_tips.htm
0
newcomer (15974)
7/10/2007 5:12:46 PM
Accoding to this post from 2005:
http://www.codeguru.com/forum/archive/index.php/t-353944.html

Microsoft has confirmed the memory leak.

08-25-2005, 11:40 AM
I contacted Microsoft via their MSDN forums and notified them of the memory 
problems I am having with CAsyncSockets and CSockets. One of the Microsoft 
people who monitor the forums wrote that he has verified the memory 
allocation problems using C++ 6.0 and MFC 4.2 on a Windows XP
Pro system (that in addition to it occurring on XP Home Edition).

So, the problem is now verified by Microsoft.

It is NOT something that I was doing wrong. There actually is a memory 
problem with CAsyncSockets and CSockets (beyond the other problems that have 
been identified by others).

AliR.


"Joseph M. Newcomer" <newcomer@flounder.com> wrote in message 
news:lid793dh8nso6a07fpmql60g30gp5bmb81@4ax.com...
>
> On Tue, 10 Jul 2007 10:33:26 -0500, "AliR \(VC++ MVP\)" 
> <AliR@online.nospam> wrote:
>
>>As far as the memory being returned, it will eventually, when the system
>>thinks that the program won't need it anymore.
> ****
> But that's what 'delete' is saying...
> ****
>>With the example I posted, before the loop my sample app is using 3676K of
>>memory after the loop the program is using 4024K of memory but there is no
>>memory leak.
> ****
> Based on what diagnostic tool?  Memory used as memory footprint is not the 
> same as memory
> used because it is allocated; in fact, there is no mechanism I am aware of 
> for the program
> to release memory to the operating system once it has been granted to the 
> heap.
> *****
>>
>>As far as Create not being reentrant goes, I am not really sure what could
>>be causing that.  There are some many possibilities as to why it doesn't
>>work the way he wants it.   Is Close being called before the next call to
>>Create?  Is the socket being passed from one thread to another?  Maybe 
>>even
>>a corrupted install of Visual Studio.
> *****
> Most of MFC is thread-safe insofar as its own internal data structures; 
> what is not
> thread-safe is the user-visible structures.  So if I access a CString or 
> CArray without
> synchronization, that is an error.  But if MFC uses a CArray, CMap, etc. 
> internally,
> either it is used exclusively by one thread or it is handled with 
> synchronization.  That's
> what worries me: that there is some unprotected structure inside MFC that 
> we should not be
> depending on without external synchronization.
> joe
> *****
>>
>>I think that the OP should try and recreate the problem with one socket in
>>the main thread.  And go from there.
>>
>>AliR.
>>
>>
>>"Joseph M. Newcomer" <newcomer@flounder.com> wrote in message
>>news:2c87935kti783rkaf53n5d28th16khebq9@4ax.com...
>>>I would have thought it *would* be returned.
>>>
>>> I'm also concerned about the concurrency problem, because I've not seen
>>> that particular
>>> problem in MFC before.  I'm wondering if there is some storage damage 
>>> that
>>> is causing both
>>> apparent problems.
>>> joe
>>>
>>> On Tue, 10 Jul 2007 09:45:46 -0500, "AliR \(VC++ MVP\)"
>>> <AliR@online.nospam> wrote:
>>>
>>>>I couldn't find a memory leak.  What you are most likely seeing is 
>>>>windows
>>>>memory managment doing its work.  The Create method is creating a socket
>>>>object, when when it's freed the memory is not given back to the system
>>>>right away.
>>>>
>>>>
>>>>(I used this method to detect a leak)
>>>>
>>>>// Declare the variables needed
>>>>#ifdef _DEBUG
>>>>   CMemoryState oldMemState, newMemState, diffMemState;
>>>>   oldMemState.Checkpoint();
>>>>#endif
>>>>
>>>>   for (int i=0; i<10; ++i)
>>>>   {
>>>>      CAsyncSocket *pAS = new CAsyncSocket;
>>>>      pAS->Create();
>>>>      pAS->Close();
>>>>      delete pAS;
>>>>   }
>>>>
>>>>#ifdef _DEBUG
>>>>   newMemState.Checkpoint();
>>>>   if( diffMemState.Difference( oldMemState, newMemState ) )
>>>>   {
>>>>      TRACE( "Memory leaked!\n" );
>>>>   }
>>>>#endif
>>>>
>>>>AliR.
>>>>
>>>>
>>>>"r norman" <r_s_norman@_comcast.net> wrote in message
>>>>news:2895939efidggi556s7fbje0euhm2jd2d0@4ax.com...
>>>>>I have traced a memory leak problem to CAsyncSocket::Create().  Is
>>>>> this a known problem?  Is there a workaround/solution/fix?  Here is
>>>>> sample code:
>>>>>
>>>>>  for (int i=0; i<m_nReopenCount; ++i) {
>>>>>    CAsyncSocket *pAS = new CAsyncSocket;
>>>>>    pAS->Create();
>>>>>    pAS->Close();
>>>>>    delete pAS;
>>>>> }
>>>>>
>>>>> Running this 1000 times uses up 1200 KBytes of memory, or just over 1
>>>>> KByte per call.  Commenting out the Create() leaves memory clean. (And
>>>>> please don't complain about my bracketing style -- I like it.)
>>>>>
>>>>> I have Visual Studio 2005 Professional version 8.0.
>>>>>
>>>>> Incidentally, I also discovered that the call to Create() is not
>>>>> re-entrant.  My application involves connecting to some 10 to 20
>>>>> external devices and my normal code creates a CWinThread to support
>>>>> each socket, where the socket is created and destroyed only  within
>>>>> the thread routine.  Creating all the threads and starting them up
>>>>> simultaneously meant having multiple instances of
>>>>> CAsyncSocket::Create() being called at the same time, crashing my
>>>>> system (memory access faults).    That one I found and fixed with
>>>>> sentries.  Now I am left with the memory leak.
>>>>>
>>>>> The problem is that I have an rather intricate communication protocol
>>>>> system all self contained so that adding a new hardware device simply
>>>>> means creating a new instance of the whole works.  It runs fine until
>>>>> the external hardware goes haywire, in which case I destruct the whole
>>>>> instance and start a new one which breaks and reconnects the socket
>>>>> with a clean start and, most of the time, results in a good
>>>>> connection; the external device resets itself through the disconnect.
>>>>> One faulty device, though, generated thousand upon thousand of
>>>>> disconnects over a number of days and, after a few hundred thousand of
>>>>> these I my own system crashed due, I have now found out, to a lack of
>>>>> memory caused by this leak.
>>>>>
>>>>> My application must run essentially as an embedded system, unattended
>>>>> week after week, month after month so I cannot tolerate a memory leak.
>>>>> Does anybody know about this?  Is there a simple clean way to force a
>>>>> socket disconnection on a CAsyncSocket and then reconnect?  My
>>>>> application is the connect() end of the socket, not the listen() end.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>> Joseph M. Newcomer [MVP]
>>> email: newcomer@flounder.com
>>> Web: http://www.flounder.com
>>> MVP Tips: http://www.flounder.com/mvp_tips.htm
>>
> Joseph M. Newcomer [MVP]
> email: newcomer@flounder.com
> Web: http://www.flounder.com
> MVP Tips: http://www.flounder.com/mvp_tips.htm 


0
AliR3470 (3236)
7/10/2007 5:14:13 PM
I don't have access or Orcas right now.  Joe can you test the example in 
orcas to see if the problem is there too?

AliR.

"AliR (VC++ MVP)" <AliR@online.nospam> wrote in message 
news:2bPki.4747$rL1.2877@newssvr19.news.prodigy.net...
> Accoding to this post from 2005:
> http://www.codeguru.com/forum/archive/index.php/t-353944.html
>
> Microsoft has confirmed the memory leak.
>
> 08-25-2005, 11:40 AM
> I contacted Microsoft via their MSDN forums and notified them of the 
> memory problems I am having with CAsyncSockets and CSockets. One of the 
> Microsoft people who monitor the forums wrote that he has verified the 
> memory allocation problems using C++ 6.0 and MFC 4.2 on a Windows XP
> Pro system (that in addition to it occurring on XP Home Edition).
>
> So, the problem is now verified by Microsoft.
>
> It is NOT something that I was doing wrong. There actually is a memory 
> problem with CAsyncSockets and CSockets (beyond the other problems that 
> have been identified by others).
>
> AliR.
>
>
> "Joseph M. Newcomer" <newcomer@flounder.com> wrote in message 
> news:lid793dh8nso6a07fpmql60g30gp5bmb81@4ax.com...
>>
>> On Tue, 10 Jul 2007 10:33:26 -0500, "AliR \(VC++ MVP\)" 
>> <AliR@online.nospam> wrote:
>>
>>>As far as the memory being returned, it will eventually, when the system
>>>thinks that the program won't need it anymore.
>> ****
>> But that's what 'delete' is saying...
>> ****
>>>With the example I posted, before the loop my sample app is using 3676K 
>>>of
>>>memory after the loop the program is using 4024K of memory but there is 
>>>no
>>>memory leak.
>> ****
>> Based on what diagnostic tool?  Memory used as memory footprint is not 
>> the same as memory
>> used because it is allocated; in fact, there is no mechanism I am aware 
>> of for the program
>> to release memory to the operating system once it has been granted to the 
>> heap.
>> *****
>>>
>>>As far as Create not being reentrant goes, I am not really sure what 
>>>could
>>>be causing that.  There are some many possibilities as to why it doesn't
>>>work the way he wants it.   Is Close being called before the next call to
>>>Create?  Is the socket being passed from one thread to another?  Maybe 
>>>even
>>>a corrupted install of Visual Studio.
>> *****
>> Most of MFC is thread-safe insofar as its own internal data structures; 
>> what is not
>> thread-safe is the user-visible structures.  So if I access a CString or 
>> CArray without
>> synchronization, that is an error.  But if MFC uses a CArray, CMap, etc. 
>> internally,
>> either it is used exclusively by one thread or it is handled with 
>> synchronization.  That's
>> what worries me: that there is some unprotected structure inside MFC that 
>> we should not be
>> depending on without external synchronization.
>> joe
>> *****
>>>
>>>I think that the OP should try and recreate the problem with one socket 
>>>in
>>>the main thread.  And go from there.
>>>
>>>AliR.
>>>
>>>
>>>"Joseph M. Newcomer" <newcomer@flounder.com> wrote in message
>>>news:2c87935kti783rkaf53n5d28th16khebq9@4ax.com...
>>>>I would have thought it *would* be returned.
>>>>
>>>> I'm also concerned about the concurrency problem, because I've not seen
>>>> that particular
>>>> problem in MFC before.  I'm wondering if there is some storage damage 
>>>> that
>>>> is causing both
>>>> apparent problems.
>>>> joe
>>>>
>>>> On Tue, 10 Jul 2007 09:45:46 -0500, "AliR \(VC++ MVP\)"
>>>> <AliR@online.nospam> wrote:
>>>>
>>>>>I couldn't find a memory leak.  What you are most likely seeing is 
>>>>>windows
>>>>>memory managment doing its work.  The Create method is creating a 
>>>>>socket
>>>>>object, when when it's freed the memory is not given back to the system
>>>>>right away.
>>>>>
>>>>>
>>>>>(I used this method to detect a leak)
>>>>>
>>>>>// Declare the variables needed
>>>>>#ifdef _DEBUG
>>>>>   CMemoryState oldMemState, newMemState, diffMemState;
>>>>>   oldMemState.Checkpoint();
>>>>>#endif
>>>>>
>>>>>   for (int i=0; i<10; ++i)
>>>>>   {
>>>>>      CAsyncSocket *pAS = new CAsyncSocket;
>>>>>      pAS->Create();
>>>>>      pAS->Close();
>>>>>      delete pAS;
>>>>>   }
>>>>>
>>>>>#ifdef _DEBUG
>>>>>   newMemState.Checkpoint();
>>>>>   if( diffMemState.Difference( oldMemState, newMemState ) )
>>>>>   {
>>>>>      TRACE( "Memory leaked!\n" );
>>>>>   }
>>>>>#endif
>>>>>
>>>>>AliR.
>>>>>
>>>>>
>>>>>"r norman" <r_s_norman@_comcast.net> wrote in message
>>>>>news:2895939efidggi556s7fbje0euhm2jd2d0@4ax.com...
>>>>>>I have traced a memory leak problem to CAsyncSocket::Create().  Is
>>>>>> this a known problem?  Is there a workaround/solution/fix?  Here is
>>>>>> sample code:
>>>>>>
>>>>>>  for (int i=0; i<m_nReopenCount; ++i) {
>>>>>>    CAsyncSocket *pAS = new CAsyncSocket;
>>>>>>    pAS->Create();
>>>>>>    pAS->Close();
>>>>>>    delete pAS;
>>>>>> }
>>>>>>
>>>>>> Running this 1000 times uses up 1200 KBytes of memory, or just over 1
>>>>>> KByte per call.  Commenting out the Create() leaves memory clean. 
>>>>>> (And
>>>>>> please don't complain about my bracketing style -- I like it.)
>>>>>>
>>>>>> I have Visual Studio 2005 Professional version 8.0.
>>>>>>
>>>>>> Incidentally, I also discovered that the call to Create() is not
>>>>>> re-entrant.  My application involves connecting to some 10 to 20
>>>>>> external devices and my normal code creates a CWinThread to support
>>>>>> each socket, where the socket is created and destroyed only  within
>>>>>> the thread routine.  Creating all the threads and starting them up
>>>>>> simultaneously meant having multiple instances of
>>>>>> CAsyncSocket::Create() being called at the same time, crashing my
>>>>>> system (memory access faults).    That one I found and fixed with
>>>>>> sentries.  Now I am left with the memory leak.
>>>>>>
>>>>>> The problem is that I have an rather intricate communication protocol
>>>>>> system all self contained so that adding a new hardware device simply
>>>>>> means creating a new instance of the whole works.  It runs fine until
>>>>>> the external hardware goes haywire, in which case I destruct the 
>>>>>> whole
>>>>>> instance and start a new one which breaks and reconnects the socket
>>>>>> with a clean start and, most of the time, results in a good
>>>>>> connection; the external device resets itself through the disconnect.
>>>>>> One faulty device, though, generated thousand upon thousand of
>>>>>> disconnects over a number of days and, after a few hundred thousand 
>>>>>> of
>>>>>> these I my own system crashed due, I have now found out, to a lack of
>>>>>> memory caused by this leak.
>>>>>>
>>>>>> My application must run essentially as an embedded system, unattended
>>>>>> week after week, month after month so I cannot tolerate a memory 
>>>>>> leak.
>>>>>> Does anybody know about this?  Is there a simple clean way to force a
>>>>>> socket disconnection on a CAsyncSocket and then reconnect?  My
>>>>>> application is the connect() end of the socket, not the listen() end.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>> Joseph M. Newcomer [MVP]
>>>> email: newcomer@flounder.com
>>>> Web: http://www.flounder.com
>>>> MVP Tips: http://www.flounder.com/mvp_tips.htm
>>>
>> Joseph M. Newcomer [MVP]
>> email: newcomer@flounder.com
>> Web: http://www.flounder.com
>> MVP Tips: http://www.flounder.com/mvp_tips.htm
>
> 


0
AliR3470 (3236)
7/10/2007 5:24:37 PM
That's a quite different problem; read my essay on how storage allocators work.

Mem Usage is relatively useless most of the time, because it is not actually a metric of
how much memory is being *used*, only about how much memory is *allocated* to the process
(which is quite different from the amount of memory that is actually in use at any given
instant!).  Yes, if it monotonically increases, you have either a leak or serious
fragmentation issues, but that's about the only value it has: the first derivative, not
the actual number, is all that matters.  If you are seeing an increasing footprint without
actual leakage, then you have a fragmentation problem, and that is a completely different
problem that what you appeared to be describing.  

See below...
On Tue, 10 Jul 2007 11:32:17 -0400, r norman <r_s_norman@_comcast.net> wrote:

>On Tue, 10 Jul 2007 09:45:46 -0500, "AliR \(VC++ MVP\)"
><AliR@online.nospam> wrote:
>
>Thank you for you help.  Perhaps "leak" is the wrong word because my
>programs in debug mode never show a problem on exit.  However the
>Create method is consuming system memory that is not released back to
>the system until the program closes.
*****
Yes.  That is what is supposed to happen.  If memory is allocated to your PROCESS then it
is nominally allocated forever.  The REAL question is why there appears to be a need to
keep allocating more memory to your process when you are freeing storage up.  That is
actually a very important question to discover the answer to.
*****
>
>The memory consumption is either shown as "Mem Usage" on the Task
>Manager Processes page or by the following code:
>
>  PROCESS_MEMORY_COUNTERS pmc;
>  HANDLE hProcess = OpenProcess(PROCESS_QUERY_INFORMATION 
>       | PROCESS_VM_READ, FALSE, _getpid());
>  size_t retval = 0;
>  if (GetProcessMemoryInfo(hProcess, &pmc, sizeof(pmc)))
>      retval = pmc.WorkingSetSize;
>  CloseHandle(hProcess);
>
>The memory used by my process continues to grow and grow indefinitely
>even though I close and delete the socket.  That seems intolerable,
>whether or not it officially counts as a leak. 
>
>Maybe the Windows memory manager will eventually discover this and
>recoup the memory, but how long will it take? 
****
Forever.  There is "Windows Memory Manager" that will make such a discovery.  Not in
application space, and not in the kernel.  There are many, many levels of allocation going
on here and it is essential to distinguish them.  There is the allocation of physical
memory, which is handled by the only component I would refer to as the "Windows Memory
Manager".  Physical memory is allocated on demand and pages which have not been used in a
long time (that is, are not part of the working set) may be removed from physical memory
and left instead on the paging file or in the executable image so that other pages which
are more in demand may be moved in.  This deals exclusively with how the operating system
manages the physical memory of the computer, and has nothing to do with how applications
manage their own memory.

Application memory is allocated by VirtualAlloc and freed by VirtualFree.  There is no
other way to get memory allocated to your application.  However, these allocations are
fairly coarse-grained (64K on a typical installation) and would be unsuitable for ordinary
allocation problems.  Therefore, the heap allocators manage this memory by allowing it to
be split into smaller pieces (HeapAlloc).  Various strategies such as modified-quickfit
and such are used to try to minimize fragmentation, but they cannot prevent fragmentation.
They merely reduce its effects somewhat.  But there is no mechanism for returning this
heap memory to the operating system in the default heap (and it is not clear what
HeapDestroy on custom heaps actually does, but I suspect it is the only situation in which
it *would* be possible to return memory to the operating system).  So what you are left
with is the issue of how memory is allocated internally in your app.  You can use
LocalAlloc/LocalFree, although those are not particularly good choices; you typically use
malloc/free, or in C++ these are typically wrapped in new/delete.  These functions
manipulate the blocks of storage allocated to your app via VirtualAlloc, breaking them
into small pieces and handing those pieces out and eventually returning them.  There are
various strategies here: quickfit, modified quickfit, first-fit, best-fit,
split-reduction, and combined with strategies such as agressive-coalescing and
lazy-coalescing, these give different performance and fragmentation profiles.  

Generally, tuning performance for memory-intensive applications takes about as long as it
took to develop the application in the first place; I've literally spent months taking
code that came to me and making it perform well by reducing the costs of memory
allocation, and most commonly reducing the effects of memory fragmentation.  In some
cases, major rewrites were required because the performance issues had nothing to do with
memory footprint at all, but on working set size, and paging was killing them.   It isn't
a simple task; you can spend several weeks just gathering data and analyzing it.  What is
your percentage of fragmentation?  (Ratio of unused small blocks to total allocation).
What is the size distribution of small blocks?  What is the size distribution of all
blocks?  What is the memory span?  (This refers to how many pages you would have to touch
to process data in one object, and/or the number of pages you would have to touch to
iterate across a set of objects).  When does it make sense to preallocate buffers inline
(TCHAR something[FIXED_SIZE]) vs. having pointers to buffers (LPTSTR, CString,
std::string)?  

_heapwalk will become overly familiar to you.  I tend to write files that can be imported
to Excel and used to graph what I need to look at.  Or which can be processed by some
scripting language into some new set of data to be shown in Excel.  You can spend a couple
weeks gathering data, just so you can spend a couple weeks staring at it, just so you can
start to form insights in what you need to *actually* gather to do the job, then gathering
*that* data, then staring at it, and finally seeing where the memory is going. Pre-MFC/C++
I actually wrote my own allocation interfaces that took __FILE__ and __LINE__ and a memory
tag, and tagged each block of memory (you get this free in the debug allocator now!) so I
knew at any given point exactly how much memory was in use, and why. I would do regular
snapshots and watch the balance change over the execution lifetime, and say "OK, how can I
change this...".  Back in the days of MS-DOS, I wrote out data which I processed with a
scripting language called "PostScript" and it plotted memory usage on my laser printer (I
have been an expert PostScript coder in the past), and the pictures were quite
interesting.  I am tempted at times to see if I can recover that code from old backup
tapes and apply it to my new color laser printer to get REALLY interesting shots, although
these days I could probably write the algorithm faster in MFC.  Or use Origin (a graphics
package) to do 3-D plots of data allocation over time (it produces VERY sexy graphs).  

None of this is really straightforward, all of it requires a fair amount of understanding
of the application and its storage usage patterns, and it all requires significant effort
to instrument and study (I haven't looked at the performance analysis tools of VS2005 yet,
some of this may already be there, but gathering numbers and looking at numbers are
separate tasks, and deciding the best strategy to act upon the numbers is yet another
phase).  None of this relates to the "Windows Storage Allocator" or HeapAlloc, malloc, or
new directly; they may be sound, robust, thread-safe, etc., and STILL fragment you like
crazy if you hit them with the wrong patterns.

This is one of the reasons I rarely worry about optimizing code these days; real
applications need data optimizations, and they are far harder.  

Note that a third-party allocator is still not going to solve the basic problem, although
by using variants of quickfit and tuning it for either lazy or agressive coalescing may
give you a result more tuned to your application's needs, so I wouldn't rule it out.  But
it won't change the fact that in C/C++, there can be no compaction, nor can there exist
any mechanism to "return storage to the operating system" that could be really effective.
In general, any mechanism that could be invented to do this is likely to actually reduce
performance by a larger factor than it contributes to improving performance (since the
smallest unit that can be freed is 64K, it would require a contiguous 64K-aligned chunk
have no objects in it, a situation unlikely to arise in common practice).

There are no easy answers.  They all take time.  If you want to do such optimizations,
budget yourself a couple months to do them.  Not counting the time taken to retest
everything after you've rewritten the algorithms to improve the performance.
					joe
*****

> All I know is that I
>have a problem in the field when my program runs month after month and
>this is the best clue I can generate on my test bed about what might
>be the cause.  I now have a version running in the field that logs its
>memory usage once a day, but that will take days and weeks to analyze.
>The problem is that the CAsyncSocket delete and recreate only occurs
>when there are problems in the field, and they occur only
>sporadically.
>
>Incidentally, the memory consumption occurs both in debug and release
>compilations and for both Unicode and simple char string (Character
>Set not set) options.  I tested that because CAsyncSocket::Create uses
>T2A_EX within CAsyncSocket::Bind and I wonder whether that might be
>the problem.
>
>
>>I couldn't find a memory leak.  What you are most likely seeing is windows 
>>memory managment doing its work.  The Create method is creating a socket 
>>object, when when it's freed the memory is not given back to the system 
>>right away.
>>
>>
>>(I used this method to detect a leak)
>>
>>// Declare the variables needed
>>#ifdef _DEBUG
>>   CMemoryState oldMemState, newMemState, diffMemState;
>>   oldMemState.Checkpoint();
>>#endif
>>
>>   for (int i=0; i<10; ++i)
>>   {
>>      CAsyncSocket *pAS = new CAsyncSocket;
>>      pAS->Create();
>>      pAS->Close();
>>      delete pAS;
>>   }
>>
>>#ifdef _DEBUG
>>   newMemState.Checkpoint();
>>   if( diffMemState.Difference( oldMemState, newMemState ) )
>>   {
>>      TRACE( "Memory leaked!\n" );
>>   }
>>#endif
>>
>>AliR.
>>
>>
>>"r norman" <r_s_norman@_comcast.net> wrote in message 
>>news:2895939efidggi556s7fbje0euhm2jd2d0@4ax.com...
>>>I have traced a memory leak problem to CAsyncSocket::Create().  Is
>>> this a known problem?  Is there a workaround/solution/fix?  Here is
>>> sample code:
>>>
>>>  for (int i=0; i<m_nReopenCount; ++i) {
>>>    CAsyncSocket *pAS = new CAsyncSocket;
>>>    pAS->Create();
>>>    pAS->Close();
>>>    delete pAS;
>>> }
>>>
>>> Running this 1000 times uses up 1200 KBytes of memory, or just over 1
>>> KByte per call.  Commenting out the Create() leaves memory clean. (And
>>> please don't complain about my bracketing style -- I like it.)
>>>
>>> I have Visual Studio 2005 Professional version 8.0.
>>>
>>> Incidentally, I also discovered that the call to Create() is not
>>> re-entrant.  My application involves connecting to some 10 to 20
>>> external devices and my normal code creates a CWinThread to support
>>> each socket, where the socket is created and destroyed only  within
>>> the thread routine.  Creating all the threads and starting them up
>>> simultaneously meant having multiple instances of
>>> CAsyncSocket::Create() being called at the same time, crashing my
>>> system (memory access faults).    That one I found and fixed with
>>> sentries.  Now I am left with the memory leak.
>>>
>>> The problem is that I have an rather intricate communication protocol
>>> system all self contained so that adding a new hardware device simply
>>> means creating a new instance of the whole works.  It runs fine until
>>> the external hardware goes haywire, in which case I destruct the whole
>>> instance and start a new one which breaks and reconnects the socket
>>> with a clean start and, most of the time, results in a good
>>> connection; the external device resets itself through the disconnect.
>>> One faulty device, though, generated thousand upon thousand of
>>> disconnects over a number of days and, after a few hundred thousand of
>>> these I my own system crashed due, I have now found out, to a lack of
>>> memory caused by this leak.
>>>
>>> My application must run essentially as an embedded system, unattended
>>> week after week, month after month so I cannot tolerate a memory leak.
>>> Does anybody know about this?  Is there a simple clean way to force a
>>> socket disconnection on a CAsyncSocket and then reconnect?  My
>>> application is the connect() end of the socket, not the listen() end.
>>>
>>>
>>>
>>> 
>>
Joseph M. Newcomer [MVP]
email: newcomer@flounder.com
Web: http://www.flounder.com
MVP Tips: http://www.flounder.com/mvp_tips.htm
0
newcomer (15974)
7/10/2007 5:48:46 PM
On Tue, 10 Jul 2007 13:12:46 -0400, Joseph M. Newcomer
<newcomer@flounder.com> wrote:

What is to be done?  Ali R's citation shows Microsoft has known about
the memory issue for some two years, now, at least in C++ 6.0 and MFC
4.2.  Why is it still present in C++ 8.0 and MFC -- what is it now,
8?.  Not only is it present, there is no warning about it in the
documentation as a "feature." So how would reporting this as a bug
make any difference?

>Part of this worries me, because a lot of us depend on the internal integrity of MFC in
>such cases.  So you may have located a serious bug in MFC; concurrency at this level
>should not cause any malfunction.  If it does, it would be a bug.
>				joe
>
>On Tue, 10 Jul 2007 12:08:44 -0400, r norman <r_s_norman@_comcast.net> wrote:
>
>>On Tue, 10 Jul 2007 11:12:12 -0400, Joseph M. Newcomer
>><newcomer@flounder.com> wrote:
>>
>>The concurrency problem was over a year ago so I am speaking only from
>>memory and my comments I noted in my code at the time.  I created
>>sixteen instances of a CWinThread derived class in quick succession,
>>each of which creates its own instance of a CAsyncSocket and calls
>>Create.  That produces a system "crash", a term which you (Joe)
>>scolded me about at the time because it carries no diagnostic
>>information.  It was, as I recall, a memory access error somewhere
>>within CAsyncSocket::Create but I don't now remember exactly where. It
>>showed behavior typical of a race or 'simultaneousness'  problem: two
>>instances seemed to never have a problem, three or four showed a
>>problem sometimes, five or six almost always did and seven or more
>>always did.  I  solved the problem by protecting the call to Create so
>>that only one could execute at a time.  Since the problem was solved,
>>I didn't concern myself with the far more serious problem of how such
>>a situation could continue to exist in the MFC code.
>>
>>
>>>I would have thought it *would* be returned.
>>>
>>>I'm also concerned about the concurrency problem, because I've not seen that particular
>>>problem in MFC before.  I'm wondering if there is some storage damage that is causing both
>>>apparent problems.
>>>					joe
>>>
>>>On Tue, 10 Jul 2007 09:45:46 -0500, "AliR \(VC++ MVP\)" <AliR@online.nospam> wrote:
>>>
>>>>I couldn't find a memory leak.  What you are most likely seeing is windows 
>>>>memory managment doing its work.  The Create method is creating a socket 
>>>>object, when when it's freed the memory is not given back to the system 
>>>>right away.
>>>>
>>>>
>>>>(I used this method to detect a leak)
>>>>
>>>>// Declare the variables needed
>>>>#ifdef _DEBUG
>>>>   CMemoryState oldMemState, newMemState, diffMemState;
>>>>   oldMemState.Checkpoint();
>>>>#endif
>>>>
>>>>   for (int i=0; i<10; ++i)
>>>>   {
>>>>      CAsyncSocket *pAS = new CAsyncSocket;
>>>>      pAS->Create();
>>>>      pAS->Close();
>>>>      delete pAS;
>>>>   }
>>>>
>>>>#ifdef _DEBUG
>>>>   newMemState.Checkpoint();
>>>>   if( diffMemState.Difference( oldMemState, newMemState ) )
>>>>   {
>>>>      TRACE( "Memory leaked!\n" );
>>>>   }
>>>>#endif
>>>>
>>>>AliR.
>>>>
>>>>
>>>>"r norman" <r_s_norman@_comcast.net> wrote in message 
>>>>news:2895939efidggi556s7fbje0euhm2jd2d0@4ax.com...
>>>>>I have traced a memory leak problem to CAsyncSocket::Create().  Is
>>>>> this a known problem?  Is there a workaround/solution/fix?  Here is
>>>>> sample code:
>>>>>
>>>>>  for (int i=0; i<m_nReopenCount; ++i) {
>>>>>    CAsyncSocket *pAS = new CAsyncSocket;
>>>>>    pAS->Create();
>>>>>    pAS->Close();
>>>>>    delete pAS;
>>>>> }
>>>>>
>>>>> Running this 1000 times uses up 1200 KBytes of memory, or just over 1
>>>>> KByte per call.  Commenting out the Create() leaves memory clean. (And
>>>>> please don't complain about my bracketing style -- I like it.)
>>>>>
>>>>> I have Visual Studio 2005 Professional version 8.0.
>>>>>
>>>>> Incidentally, I also discovered that the call to Create() is not
>>>>> re-entrant.  My application involves connecting to some 10 to 20
>>>>> external devices and my normal code creates a CWinThread to support
>>>>> each socket, where the socket is created and destroyed only  within
>>>>> the thread routine.  Creating all the threads and starting them up
>>>>> simultaneously meant having multiple instances of
>>>>> CAsyncSocket::Create() being called at the same time, crashing my
>>>>> system (memory access faults).    That one I found and fixed with
>>>>> sentries.  Now I am left with the memory leak.
>>>>>
>>>>> The problem is that I have an rather intricate communication protocol
>>>>> system all self contained so that adding a new hardware device simply
>>>>> means creating a new instance of the whole works.  It runs fine until
>>>>> the external hardware goes haywire, in which case I destruct the whole
>>>>> instance and start a new one which breaks and reconnects the socket
>>>>> with a clean start and, most of the time, results in a good
>>>>> connection; the external device resets itself through the disconnect.
>>>>> One faulty device, though, generated thousand upon thousand of
>>>>> disconnects over a number of days and, after a few hundred thousand of
>>>>> these I my own system crashed due, I have now found out, to a lack of
>>>>> memory caused by this leak.
>>>>>
>>>>> My application must run essentially as an embedded system, unattended
>>>>> week after week, month after month so I cannot tolerate a memory leak.
>>>>> Does anybody know about this?  Is there a simple clean way to force a
>>>>> socket disconnection on a CAsyncSocket and then reconnect?  My
>>>>> application is the connect() end of the socket, not the listen() end.
>>>>>
>>>>>
>>>>>
>>>>> 
>>>>
>>>Joseph M. Newcomer [MVP]
>>>email: newcomer@flounder.com
>>>Web: http://www.flounder.com
>>>MVP Tips: http://www.flounder.com/mvp_tips.htm
>Joseph M. Newcomer [MVP]
>email: newcomer@flounder.com
>Web: http://www.flounder.com
>MVP Tips: http://www.flounder.com/mvp_tips.htm
0
r
7/10/2007 6:25:05 PM
On Tue, 10 Jul 2007 13:48:46 -0400, Joseph M. Newcomer
<newcomer@flounder.com> wrote:

I really do read, learn from, and appreciate your long answers on this
forum.  Thank you very much for your continual tutelage over the
years.

I recognize that I use the wrong words to describe my problem.  As you
have seen both in your own answer and other exchanges on this thread,
there does seem to be a continual drain on system memory into my
process with repeated calls to CAsyncSocket::Create even though the
socket is properly closed and deallocated between calls.  Whether it
results from leaks or fragmentation is irrelevant if they both cause
my program to fail after months of operation.   Incidentally, I have
other instances of constant allocation with new and delete, for
example, for data passed between modules and threads in my system.  I
have tested this with hundreds of thousands of creations and deletions
of instances of my data management class with different size data and
sequences of construction and destruction and none of these produce
memory creep in my system.

I am exploring solutions, first, to continue to use CAsyncSocket for
the callback functions it supports (and because the code already
exists and is tested in that form) but creating and managing low level
sockets myself.  If that fails, I will explore dropping out of MFC for
socket work completely although that means an awful lot of code to
rework and retest.  I support five different network protocols
simultaneously to five different categories of external devices, each
with variable numbers of instances and only one specific type gives me
problems.  Still they share the same basic set of classes that support
generalized protocols and message formats so changing things at this
low level will require an enormous amount of new testing.

But that is the way the world works!

>That's a quite different problem; read my essay on how storage allocators work.
>
>Mem Usage is relatively useless most of the time, because it is not actually a metric of
>how much memory is being *used*, only about how much memory is *allocated* to the process
>(which is quite different from the amount of memory that is actually in use at any given
>instant!).  Yes, if it monotonically increases, you have either a leak or serious
>fragmentation issues, but that's about the only value it has: the first derivative, not
>the actual number, is all that matters.  If you are seeing an increasing footprint without
>actual leakage, then you have a fragmentation problem, and that is a completely different
>problem that what you appeared to be describing.  
>
>See below...
>On Tue, 10 Jul 2007 11:32:17 -0400, r norman <r_s_norman@_comcast.net> wrote:
>
>>On Tue, 10 Jul 2007 09:45:46 -0500, "AliR \(VC++ MVP\)"
>><AliR@online.nospam> wrote:
>>
>>Thank you for you help.  Perhaps "leak" is the wrong word because my
>>programs in debug mode never show a problem on exit.  However the
>>Create method is consuming system memory that is not released back to
>>the system until the program closes.
>*****
>Yes.  That is what is supposed to happen.  If memory is allocated to your PROCESS then it
>is nominally allocated forever.  The REAL question is why there appears to be a need to
>keep allocating more memory to your process when you are freeing storage up.  That is
>actually a very important question to discover the answer to.
>*****
>>
>>The memory consumption is either shown as "Mem Usage" on the Task
>>Manager Processes page or by the following code:
>>
>>  PROCESS_MEMORY_COUNTERS pmc;
>>  HANDLE hProcess = OpenProcess(PROCESS_QUERY_INFORMATION 
>>       | PROCESS_VM_READ, FALSE, _getpid());
>>  size_t retval = 0;
>>  if (GetProcessMemoryInfo(hProcess, &pmc, sizeof(pmc)))
>>      retval = pmc.WorkingSetSize;
>>  CloseHandle(hProcess);
>>
>>The memory used by my process continues to grow and grow indefinitely
>>even though I close and delete the socket.  That seems intolerable,
>>whether or not it officially counts as a leak. 
>>
>>Maybe the Windows memory manager will eventually discover this and
>>recoup the memory, but how long will it take? 
>****
>Forever.  There is "Windows Memory Manager" that will make such a discovery.  Not in
>application space, and not in the kernel.  There are many, many levels of allocation going
>on here and it is essential to distinguish them.  There is the allocation of physical
>memory, which is handled by the only component I would refer to as the "Windows Memory
>Manager".  Physical memory is allocated on demand and pages which have not been used in a
>long time (that is, are not part of the working set) may be removed from physical memory
>and left instead on the paging file or in the executable image so that other pages which
>are more in demand may be moved in.  This deals exclusively with how the operating system
>manages the physical memory of the computer, and has nothing to do with how applications
>manage their own memory.
>
>Application memory is allocated by VirtualAlloc and freed by VirtualFree.  There is no
>other way to get memory allocated to your application.  However, these allocations are
>fairly coarse-grained (64K on a typical installation) and would be unsuitable for ordinary
>allocation problems.  Therefore, the heap allocators manage this memory by allowing it to
>be split into smaller pieces (HeapAlloc).  Various strategies such as modified-quickfit
>and such are used to try to minimize fragmentation, but they cannot prevent fragmentation.
>They merely reduce its effects somewhat.  But there is no mechanism for returning this
>heap memory to the operating system in the default heap (and it is not clear what
>HeapDestroy on custom heaps actually does, but I suspect it is the only situation in which
>it *would* be possible to return memory to the operating system).  So what you are left
>with is the issue of how memory is allocated internally in your app.  You can use
>LocalAlloc/LocalFree, although those are not particularly good choices; you typically use
>malloc/free, or in C++ these are typically wrapped in new/delete.  These functions
>manipulate the blocks of storage allocated to your app via VirtualAlloc, breaking them
>into small pieces and handing those pieces out and eventually returning them.  There are
>various strategies here: quickfit, modified quickfit, first-fit, best-fit,
>split-reduction, and combined with strategies such as agressive-coalescing and
>lazy-coalescing, these give different performance and fragmentation profiles.  
>
>Generally, tuning performance for memory-intensive applications takes about as long as it
>took to develop the application in the first place; I've literally spent months taking
>code that came to me and making it perform well by reducing the costs of memory
>allocation, and most commonly reducing the effects of memory fragmentation.  In some
>cases, major rewrites were required because the performance issues had nothing to do with
>memory footprint at all, but on working set size, and paging was killing them.   It isn't
>a simple task; you can spend several weeks just gathering data and analyzing it.  What is
>your percentage of fragmentation?  (Ratio of unused small blocks to total allocation).
>What is the size distribution of small blocks?  What is the size distribution of all
>blocks?  What is the memory span?  (This refers to how many pages you would have to touch
>to process data in one object, and/or the number of pages you would have to touch to
>iterate across a set of objects).  When does it make sense to preallocate buffers inline
>(TCHAR something[FIXED_SIZE]) vs. having pointers to buffers (LPTSTR, CString,
>std::string)?  
>
>_heapwalk will become overly familiar to you.  I tend to write files that can be imported
>to Excel and used to graph what I need to look at.  Or which can be processed by some
>scripting language into some new set of data to be shown in Excel.  You can spend a couple
>weeks gathering data, just so you can spend a couple weeks staring at it, just so you can
>start to form insights in what you need to *actually* gather to do the job, then gathering
>*that* data, then staring at it, and finally seeing where the memory is going. Pre-MFC/C++
>I actually wrote my own allocation interfaces that took __FILE__ and __LINE__ and a memory
>tag, and tagged each block of memory (you get this free in the debug allocator now!) so I
>knew at any given point exactly how much memory was in use, and why. I would do regular
>snapshots and watch the balance change over the execution lifetime, and say "OK, how can I
>change this...".  Back in the days of MS-DOS, I wrote out data which I processed with a
>scripting language called "PostScript" and it plotted memory usage on my laser printer (I
>have been an expert PostScript coder in the past), and the pictures were quite
>interesting.  I am tempted at times to see if I can recover that code from old backup
>tapes and apply it to my new color laser printer to get REALLY interesting shots, although
>these days I could probably write the algorithm faster in MFC.  Or use Origin (a graphics
>package) to do 3-D plots of data allocation over time (it produces VERY sexy graphs).  
>
>None of this is really straightforward, all of it requires a fair amount of understanding
>of the application and its storage usage patterns, and it all requires significant effort
>to instrument and study (I haven't looked at the performance analysis tools of VS2005 yet,
>some of this may already be there, but gathering numbers and looking at numbers are
>separate tasks, and deciding the best strategy to act upon the numbers is yet another
>phase).  None of this relates to the "Windows Storage Allocator" or HeapAlloc, malloc, or
>new directly; they may be sound, robust, thread-safe, etc., and STILL fragment you like
>crazy if you hit them with the wrong patterns.
>
>This is one of the reasons I rarely worry about optimizing code these days; real
>applications need data optimizations, and they are far harder.  
>
>Note that a third-party allocator is still not going to solve the basic problem, although
>by using variants of quickfit and tuning it for either lazy or agressive coalescing may
>give you a result more tuned to your application's needs, so I wouldn't rule it out.  But
>it won't change the fact that in C/C++, there can be no compaction, nor can there exist
>any mechanism to "return storage to the operating system" that could be really effective.
>In general, any mechanism that could be invented to do this is likely to actually reduce
>performance by a larger factor than it contributes to improving performance (since the
>smallest unit that can be freed is 64K, it would require a contiguous 64K-aligned chunk
>have no objects in it, a situation unlikely to arise in common practice).
>
>There are no easy answers.  They all take time.  If you want to do such optimizations,
>budget yourself a couple months to do them.  Not counting the time taken to retest
>everything after you've rewritten the algorithms to improve the performance.
>					joe
>*****
>
>> All I know is that I
>>have a problem in the field when my program runs month after month and
>>this is the best clue I can generate on my test bed about what might
>>be the cause.  I now have a version running in the field that logs its
>>memory usage once a day, but that will take days and weeks to analyze.
>>The problem is that the CAsyncSocket delete and recreate only occurs
>>when there are problems in the field, and they occur only
>>sporadically.
>>
>>Incidentally, the memory consumption occurs both in debug and release
>>compilations and for both Unicode and simple char string (Character
>>Set not set) options.  I tested that because CAsyncSocket::Create uses
>>T2A_EX within CAsyncSocket::Bind and I wonder whether that might be
>>the problem.
>>
>>
>>>I couldn't find a memory leak.  What you are most likely seeing is windows 
>>>memory managment doing its work.  The Create method is creating a socket 
>>>object, when when it's freed the memory is not given back to the system 
>>>right away.
>>>
>>>
>>>(I used this method to detect a leak)
>>>
>>>// Declare the variables needed
>>>#ifdef _DEBUG
>>>   CMemoryState oldMemState, newMemState, diffMemState;
>>>   oldMemState.Checkpoint();
>>>#endif
>>>
>>>   for (int i=0; i<10; ++i)
>>>   {
>>>      CAsyncSocket *pAS = new CAsyncSocket;
>>>      pAS->Create();
>>>      pAS->Close();
>>>      delete pAS;
>>>   }
>>>
>>>#ifdef _DEBUG
>>>   newMemState.Checkpoint();
>>>   if( diffMemState.Difference( oldMemState, newMemState ) )
>>>   {
>>>      TRACE( "Memory leaked!\n" );
>>>   }
>>>#endif
>>>
>>>AliR.
>>>
>>>
>>>"r norman" <r_s_norman@_comcast.net> wrote in message 
>>>news:2895939efidggi556s7fbje0euhm2jd2d0@4ax.com...
>>>>I have traced a memory leak problem to CAsyncSocket::Create().  Is
>>>> this a known problem?  Is there a workaround/solution/fix?  Here is
>>>> sample code:
>>>>
>>>>  for (int i=0; i<m_nReopenCount; ++i) {
>>>>    CAsyncSocket *pAS = new CAsyncSocket;
>>>>    pAS->Create();
>>>>    pAS->Close();
>>>>    delete pAS;
>>>> }
>>>>
>>>> Running this 1000 times uses up 1200 KBytes of memory, or just over 1
>>>> KByte per call.  Commenting out the Create() leaves memory clean. (And
>>>> please don't complain about my bracketing style -- I like it.)
>>>>
>>>> I have Visual Studio 2005 Professional version 8.0.
>>>>
>>>> Incidentally, I also discovered that the call to Create() is not
>>>> re-entrant.  My application involves connecting to some 10 to 20
>>>> external devices and my normal code creates a CWinThread to support
>>>> each socket, where the socket is created and destroyed only  within
>>>> the thread routine.  Creating all the threads and starting them up
>>>> simultaneously meant having multiple instances of
>>>> CAsyncSocket::Create() being called at the same time, crashing my
>>>> system (memory access faults).    That one I found and fixed with
>>>> sentries.  Now I am left with the memory leak.
>>>>
>>>> The problem is that I have an rather intricate communication protocol
>>>> system all self contained so that adding a new hardware device simply
>>>> means creating a new instance of the whole works.  It runs fine until
>>>> the external hardware goes haywire, in which case I destruct the whole
>>>> instance and start a new one which breaks and reconnects the socket
>>>> with a clean start and, most of the time, results in a good
>>>> connection; the external device resets itself through the disconnect.
>>>> One faulty device, though, generated thousand upon thousand of
>>>> disconnects over a number of days and, after a few hundred thousand of
>>>> these I my own system crashed due, I have now found out, to a lack of
>>>> memory caused by this leak.
>>>>
>>>> My application must run essentially as an embedded system, unattended
>>>> week after week, month after month so I cannot tolerate a memory leak.
>>>> Does anybody know about this?  Is there a simple clean way to force a
>>>> socket disconnection on a CAsyncSocket and then reconnect?  My
>>>> application is the connect() end of the socket, not the listen() end.
>>>>
>>>>
>>>>
>>>> 
>>>
>Joseph M. Newcomer [MVP]
>email: newcomer@flounder.com
>Web: http://www.flounder.com
>MVP Tips: http://www.flounder.com/mvp_tips.htm
0
r
7/10/2007 6:37:21 PM
See below...
On Tue, 10 Jul 2007 13:07:32 -0400, r norman <r_s_norman@_comcast.net> wrote:

>On Tue, 10 Jul 2007 12:45:25 -0400, Joseph M. Newcomer
><newcomer@flounder.com> wrote:
>
>I can accept that memory allocated to my process won't be returned to
>the system.  But when my process is done with it and needs more,
>shouldn't the memory manager take from my own internal unused heap and
>not ask the system for more?  Where does it end?  When the system is
>exhausted?
****
It should, assuming it can find the space.  But the space you freed up may be broken up
into smaller pieces so that by the time you get back to asking for it, there isn't a piece
big enough to satisfy your request.  This is what the "memory fragmentation" problem is
all about, and which is why you can spend weeks figuring out how to fix it.  This is *not*
an "error" in the allocator; it is doing exactly what it is supposed to do.  It has
nothing to do with the robustness of the algorithms, or their correctness; it has to do
with whether or not your allocation patterns generate situations guaranteed to be pessimal
for a specific design of an allocator (by the way, like paging algorithms, tell me how
your allocator works and I can create a situation that will stress it to beyond its
breaking point).

And yes, it ends when you hit the 2GB limit for your process, or the system runs out of
virtual memory to hold your process, whichever comes first.  
				joe

****
>
>>On Tue, 10 Jul 2007 10:33:26 -0500, "AliR \(VC++ MVP\)" <AliR@online.nospam> wrote:
>>
>>>As far as the memory being returned, it will eventually, when the system 
>>>thinks that the program won't need it anymore.
>>****
>>But that's what 'delete' is saying...
>>****
>>>With the example I posted, before the loop my sample app is using 3676K of 
>>>memory after the loop the program is using 4024K of memory but there is no 
>>>memory leak.
>>****
>>Based on what diagnostic tool?  Memory used as memory footprint is not the same as memory
>>used because it is allocated; in fact, there is no mechanism I am aware of for the program
>>to release memory to the operating system once it has been granted to the heap.
>>*****
>>>
>>>As far as Create not being reentrant goes, I am not really sure what could 
>>>be causing that.  There are some many possibilities as to why it doesn't 
>>>work the way he wants it.   Is Close being called before the next call to 
>>>Create?  Is the socket being passed from one thread to another?  Maybe even 
>>>a corrupted install of Visual Studio.
>>*****
>>Most of MFC is thread-safe insofar as its own internal data structures; what is not
>>thread-safe is the user-visible structures.  So if I access a CString or CArray without
>>synchronization, that is an error.  But if MFC uses a CArray, CMap, etc. internally,
>>either it is used exclusively by one thread or it is handled with synchronization.  That's
>>what worries me: that there is some unprotected structure inside MFC that we should not be
>>depending on without external synchronization.
>>					joe
>>*****
>>>
>>>I think that the OP should try and recreate the problem with one socket in 
>>>the main thread.  And go from there.
>>>
>>>AliR.
>>>
>>>
>>>"Joseph M. Newcomer" <newcomer@flounder.com> wrote in message 
>>>news:2c87935kti783rkaf53n5d28th16khebq9@4ax.com...
>>>>I would have thought it *would* be returned.
>>>>
>>>> I'm also concerned about the concurrency problem, because I've not seen 
>>>> that particular
>>>> problem in MFC before.  I'm wondering if there is some storage damage that 
>>>> is causing both
>>>> apparent problems.
>>>> joe
>>>>
>>>> On Tue, 10 Jul 2007 09:45:46 -0500, "AliR \(VC++ MVP\)" 
>>>> <AliR@online.nospam> wrote:
>>>>
>>>>>I couldn't find a memory leak.  What you are most likely seeing is windows
>>>>>memory managment doing its work.  The Create method is creating a socket
>>>>>object, when when it's freed the memory is not given back to the system
>>>>>right away.
>>>>>
>>>>>
>>>>>(I used this method to detect a leak)
>>>>>
>>>>>// Declare the variables needed
>>>>>#ifdef _DEBUG
>>>>>   CMemoryState oldMemState, newMemState, diffMemState;
>>>>>   oldMemState.Checkpoint();
>>>>>#endif
>>>>>
>>>>>   for (int i=0; i<10; ++i)
>>>>>   {
>>>>>      CAsyncSocket *pAS = new CAsyncSocket;
>>>>>      pAS->Create();
>>>>>      pAS->Close();
>>>>>      delete pAS;
>>>>>   }
>>>>>
>>>>>#ifdef _DEBUG
>>>>>   newMemState.Checkpoint();
>>>>>   if( diffMemState.Difference( oldMemState, newMemState ) )
>>>>>   {
>>>>>      TRACE( "Memory leaked!\n" );
>>>>>   }
>>>>>#endif
>>>>>
>>>>>AliR.
>>>>>
>>>>>
>>>>>"r norman" <r_s_norman@_comcast.net> wrote in message
>>>>>news:2895939efidggi556s7fbje0euhm2jd2d0@4ax.com...
>>>>>>I have traced a memory leak problem to CAsyncSocket::Create().  Is
>>>>>> this a known problem?  Is there a workaround/solution/fix?  Here is
>>>>>> sample code:
>>>>>>
>>>>>>  for (int i=0; i<m_nReopenCount; ++i) {
>>>>>>    CAsyncSocket *pAS = new CAsyncSocket;
>>>>>>    pAS->Create();
>>>>>>    pAS->Close();
>>>>>>    delete pAS;
>>>>>> }
>>>>>>
>>>>>> Running this 1000 times uses up 1200 KBytes of memory, or just over 1
>>>>>> KByte per call.  Commenting out the Create() leaves memory clean. (And
>>>>>> please don't complain about my bracketing style -- I like it.)
>>>>>>
>>>>>> I have Visual Studio 2005 Professional version 8.0.
>>>>>>
>>>>>> Incidentally, I also discovered that the call to Create() is not
>>>>>> re-entrant.  My application involves connecting to some 10 to 20
>>>>>> external devices and my normal code creates a CWinThread to support
>>>>>> each socket, where the socket is created and destroyed only  within
>>>>>> the thread routine.  Creating all the threads and starting them up
>>>>>> simultaneously meant having multiple instances of
>>>>>> CAsyncSocket::Create() being called at the same time, crashing my
>>>>>> system (memory access faults).    That one I found and fixed with
>>>>>> sentries.  Now I am left with the memory leak.
>>>>>>
>>>>>> The problem is that I have an rather intricate communication protocol
>>>>>> system all self contained so that adding a new hardware device simply
>>>>>> means creating a new instance of the whole works.  It runs fine until
>>>>>> the external hardware goes haywire, in which case I destruct the whole
>>>>>> instance and start a new one which breaks and reconnects the socket
>>>>>> with a clean start and, most of the time, results in a good
>>>>>> connection; the external device resets itself through the disconnect.
>>>>>> One faulty device, though, generated thousand upon thousand of
>>>>>> disconnects over a number of days and, after a few hundred thousand of
>>>>>> these I my own system crashed due, I have now found out, to a lack of
>>>>>> memory caused by this leak.
>>>>>>
>>>>>> My application must run essentially as an embedded system, unattended
>>>>>> week after week, month after month so I cannot tolerate a memory leak.
>>>>>> Does anybody know about this?  Is there a simple clean way to force a
>>>>>> socket disconnection on a CAsyncSocket and then reconnect?  My
>>>>>> application is the connect() end of the socket, not the listen() end.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>> Joseph M. Newcomer [MVP]
>>>> email: newcomer@flounder.com
>>>> Web: http://www.flounder.com
>>>> MVP Tips: http://www.flounder.com/mvp_tips.htm 
>>>
>>Joseph M. Newcomer [MVP]
>>email: newcomer@flounder.com
>>Web: http://www.flounder.com
>>MVP Tips: http://www.flounder.com/mvp_tips.htm
Joseph M. Newcomer [MVP]
email: newcomer@flounder.com
Web: http://www.flounder.com
MVP Tips: http://www.flounder.com/mvp_tips.htm
0
newcomer (15974)
7/10/2007 8:01:04 PM
I have no copy of Orcas.  

Ouch!  This is a serious issue then, and I agree: why has it not been fixed?
					joe

On Tue, 10 Jul 2007 12:24:37 -0500, "AliR \(VC++ MVP\)" <AliR@online.nospam> wrote:

>I don't have access or Orcas right now.  Joe can you test the example in 
>orcas to see if the problem is there too?
>
>AliR.
>
>"AliR (VC++ MVP)" <AliR@online.nospam> wrote in message 
>news:2bPki.4747$rL1.2877@newssvr19.news.prodigy.net...
>> Accoding to this post from 2005:
>> http://www.codeguru.com/forum/archive/index.php/t-353944.html
>>
>> Microsoft has confirmed the memory leak.
>>
>> 08-25-2005, 11:40 AM
>> I contacted Microsoft via their MSDN forums and notified them of the 
>> memory problems I am having with CAsyncSockets and CSockets. One of the 
>> Microsoft people who monitor the forums wrote that he has verified the 
>> memory allocation problems using C++ 6.0 and MFC 4.2 on a Windows XP
>> Pro system (that in addition to it occurring on XP Home Edition).
>>
>> So, the problem is now verified by Microsoft.
>>
>> It is NOT something that I was doing wrong. There actually is a memory 
>> problem with CAsyncSockets and CSockets (beyond the other problems that 
>> have been identified by others).
>>
>> AliR.
>>
>>
>> "Joseph M. Newcomer" <newcomer@flounder.com> wrote in message 
>> news:lid793dh8nso6a07fpmql60g30gp5bmb81@4ax.com...
>>>
>>> On Tue, 10 Jul 2007 10:33:26 -0500, "AliR \(VC++ MVP\)" 
>>> <AliR@online.nospam> wrote:
>>>
>>>>As far as the memory being returned, it will eventually, when the system
>>>>thinks that the program won't need it anymore.
>>> ****
>>> But that's what 'delete' is saying...
>>> ****
>>>>With the example I posted, before the loop my sample app is using 3676K 
>>>>of
>>>>memory after the loop the program is using 4024K of memory but there is 
>>>>no
>>>>memory leak.
>>> ****
>>> Based on what diagnostic tool?  Memory used as memory footprint is not 
>>> the same as memory
>>> used because it is allocated; in fact, there is no mechanism I am aware 
>>> of for the program
>>> to release memory to the operating system once it has been granted to the 
>>> heap.
>>> *****
>>>>
>>>>As far as Create not being reentrant goes, I am not really sure what 
>>>>could
>>>>be causing that.  There are some many possibilities as to why it doesn't
>>>>work the way he wants it.   Is Close being called before the next call to
>>>>Create?  Is the socket being passed from one thread to another?  Maybe 
>>>>even
>>>>a corrupted install of Visual Studio.
>>> *****
>>> Most of MFC is thread-safe insofar as its own internal data structures; 
>>> what is not
>>> thread-safe is the user-visible structures.  So if I access a CString or 
>>> CArray without
>>> synchronization, that is an error.  But if MFC uses a CArray, CMap, etc. 
>>> internally,
>>> either it is used exclusively by one thread or it is handled with 
>>> synchronization.  That's
>>> what worries me: that there is some unprotected structure inside MFC that 
>>> we should not be
>>> depending on without external synchronization.
>>> joe
>>> *****
>>>>
>>>>I think that the OP should try and recreate the problem with one socket 
>>>>in
>>>>the main thread.  And go from there.
>>>>
>>>>AliR.
>>>>
>>>>
>>>>"Joseph M. Newcomer" <newcomer@flounder.com> wrote in message
>>>>news:2c87935kti783rkaf53n5d28th16khebq9@4ax.com...
>>>>>I would have thought it *would* be returned.
>>>>>
>>>>> I'm also concerned about the concurrency problem, because I've not seen
>>>>> that particular
>>>>> problem in MFC before.  I'm wondering if there is some storage damage 
>>>>> that
>>>>> is causing both
>>>>> apparent problems.
>>>>> joe
>>>>>
>>>>> On Tue, 10 Jul 2007 09:45:46 -0500, "AliR \(VC++ MVP\)"
>>>>> <AliR@online.nospam> wrote:
>>>>>
>>>>>>I couldn't find a memory leak.  What you are most likely seeing is 
>>>>>>windows
>>>>>>memory managment doing its work.  The Create method is creating a 
>>>>>>socket
>>>>>>object, when when it's freed the memory is not given back to the system
>>>>>>right away.
>>>>>>
>>>>>>
>>>>>>(I used this method to detect a leak)
>>>>>>
>>>>>>// Declare the variables needed
>>>>>>#ifdef _DEBUG
>>>>>>   CMemoryState oldMemState, newMemState, diffMemState;
>>>>>>   oldMemState.Checkpoint();
>>>>>>#endif
>>>>>>
>>>>>>   for (int i=0; i<10; ++i)
>>>>>>   {
>>>>>>      CAsyncSocket *pAS = new CAsyncSocket;
>>>>>>      pAS->Create();
>>>>>>      pAS->Close();
>>>>>>      delete pAS;
>>>>>>   }
>>>>>>
>>>>>>#ifdef _DEBUG
>>>>>>   newMemState.Checkpoint();
>>>>>>   if( diffMemState.Difference( oldMemState, newMemState ) )
>>>>>>   {
>>>>>>      TRACE( "Memory leaked!\n" );
>>>>>>   }
>>>>>>#endif
>>>>>>
>>>>>>AliR.
>>>>>>
>>>>>>
>>>>>>"r norman" <r_s_norman@_comcast.net> wrote in message
>>>>>>news:2895939efidggi556s7fbje0euhm2jd2d0@4ax.com...
>>>>>>>I have traced a memory leak problem to CAsyncSocket::Create().  Is
>>>>>>> this a known problem?  Is there a workaround/solution/fix?  Here is
>>>>>>> sample code:
>>>>>>>
>>>>>>>  for (int i=0; i<m_nReopenCount; ++i) {
>>>>>>>    CAsyncSocket *pAS = new CAsyncSocket;
>>>>>>>    pAS->Create();
>>>>>>>    pAS->Close();
>>>>>>>    delete pAS;
>>>>>>> }
>>>>>>>
>>>>>>> Running this 1000 times uses up 1200 KBytes of memory, or just over 1
>>>>>>> KByte per call.  Commenting out the Create() leaves memory clean. 
>>>>>>> (And
>>>>>>> please don't complain about my bracketing style -- I like it.)
>>>>>>>
>>>>>>> I have Visual Studio 2005 Professional version 8.0.
>>>>>>>
>>>>>>> Incidentally, I also discovered that the call to Create() is not
>>>>>>> re-entrant.  My application involves connecting to some 10 to 20
>>>>>>> external devices and my normal code creates a CWinThread to support
>>>>>>> each socket, where the socket is created and destroyed only  within
>>>>>>> the thread routine.  Creating all the threads and starting them up
>>>>>>> simultaneously meant having multiple instances of
>>>>>>> CAsyncSocket::Create() being called at the same time, crashing my
>>>>>>> system (memory access faults).    That one I found and fixed with
>>>>>>> sentries.  Now I am left with the memory leak.
>>>>>>>
>>>>>>> The problem is that I have an rather intricate communication protocol
>>>>>>> system all self contained so that adding a new hardware device simply
>>>>>>> means creating a new instance of the whole works.  It runs fine until
>>>>>>> the external hardware goes haywire, in which case I destruct the 
>>>>>>> whole
>>>>>>> instance and start a new one which breaks and reconnects the socket
>>>>>>> with a clean start and, most of the time, results in a good
>>>>>>> connection; the external device resets itself through the disconnect.
>>>>>>> One faulty device, though, generated thousand upon thousand of
>>>>>>> disconnects over a number of days and, after a few hundred thousand 
>>>>>>> of
>>>>>>> these I my own system crashed due, I have now found out, to a lack of
>>>>>>> memory caused by this leak.
>>>>>>>
>>>>>>> My application must run essentially as an embedded system, unattended
>>>>>>> week after week, month after month so I cannot tolerate a memory 
>>>>>>> leak.
>>>>>>> Does anybody know about this?  Is there a simple clean way to force a
>>>>>>> socket disconnection on a CAsyncSocket and then reconnect?  My
>>>>>>> application is the connect() end of the socket, not the listen() end.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>> Joseph M. Newcomer [MVP]
>>>>> email: newcomer@flounder.com
>>>>> Web: http://www.flounder.com
>>>>> MVP Tips: http://www.flounder.com/mvp_tips.htm
>>>>
>>> Joseph M. Newcomer [MVP]
>>> email: newcomer@flounder.com
>>> Web: http://www.flounder.com
>>> MVP Tips: http://www.flounder.com/mvp_tips.htm
>>
>> 
>
Joseph M. Newcomer [MVP]
email: newcomer@flounder.com
Web: http://www.flounder.com
MVP Tips: http://www.flounder.com/mvp_tips.htm
0
newcomer (15974)
7/10/2007 8:02:49 PM
See below,..
On Tue, 10 Jul 2007 14:37:21 -0400, r norman <r_s_norman@_comcast.net> wrote:

>On Tue, 10 Jul 2007 13:48:46 -0400, Joseph M. Newcomer
><newcomer@flounder.com> wrote:
>
>I really do read, learn from, and appreciate your long answers on this
>forum.  Thank you very much for your continual tutelage over the
>years.
>
>I recognize that I use the wrong words to describe my problem.  As you
>have seen both in your own answer and other exchanges on this thread,
>there does seem to be a continual drain on system memory into my
>process with repeated calls to CAsyncSocket::Create even though the
>socket is properly closed and deallocated between calls.  Whether it
>results from leaks or fragmentation is irrelevant if they both cause
>my program to fail after months of operation.   Incidentally, I have
>other instances of constant allocation with new and delete, for
>example, for data passed between modules and threads in my system.  I
>have tested this with hundreds of thousands of creations and deletions
>of instances of my data management class with different size data and
>sequences of construction and destruction and none of these produce
>memory creep in my system.
*****
Memory "creep" is deadly, because this is really indicating fragmentation (I suppose you
have already seen the post that says what you are seeing is a definite bug in MFC!)   

This is where the use of local caches comes in handy.  If all your interthread messages
are the same kind of structure of the same length (or there is a reasonable maximum
length), then a cache of fixed-size blocks will eliminate the memory creep because all
allocations and deletions are working on a fixed and finite set of values.  It gets more
interesting if you have to deal with peak-usage issues, and semaphores as a means of
resource management can work here if your threads can withstand stoppages.
*****
>
>I am exploring solutions, first, to continue to use CAsyncSocket for
>the callback functions it supports (and because the code already
>exists and is tested in that form) but creating and managing low level
>sockets myself.  If that fails, I will explore dropping out of MFC for
>socket work completely although that means an awful lot of code to
>rework and retest.  I support five different network protocols
>simultaneously to five different categories of external devices, each
>with variable numbers of instances and only one specific type gives me
>problems.  Still they share the same basic set of classes that support
>generalized protocols and message formats so changing things at this
>low level will require an enormous amount of new testing.
*****
Don't fall off the deep end until you understand where the memory lossage is coming from.
If you start managing low-level sockets yourself, there is a chance that you will expend a
great deal of effort irrelevant to solving the problem.  Key here is to see WHAT object is
being lost at WHICH level before trying to implement a solution!
*****
>
>But that is the way the world works!
****
Or doesn't work :-(
****
>
>>That's a quite different problem; read my essay on how storage allocators work.
>>
>>Mem Usage is relatively useless most of the time, because it is not actually a metric of
>>how much memory is being *used*, only about how much memory is *allocated* to the process
>>(which is quite different from the amount of memory that is actually in use at any given
>>instant!).  Yes, if it monotonically increases, you have either a leak or serious
>>fragmentation issues, but that's about the only value it has: the first derivative, not
>>the actual number, is all that matters.  If you are seeing an increasing footprint without
>>actual leakage, then you have a fragmentation problem, and that is a completely different
>>problem that what you appeared to be describing.  
>>
>>See below...
>>On Tue, 10 Jul 2007 11:32:17 -0400, r norman <r_s_norman@_comcast.net> wrote:
>>
>>>On Tue, 10 Jul 2007 09:45:46 -0500, "AliR \(VC++ MVP\)"
>>><AliR@online.nospam> wrote:
>>>
>>>Thank you for you help.  Perhaps "leak" is the wrong word because my
>>>programs in debug mode never show a problem on exit.  However the
>>>Create method is consuming system memory that is not released back to
>>>the system until the program closes.
>>*****
>>Yes.  That is what is supposed to happen.  If memory is allocated to your PROCESS then it
>>is nominally allocated forever.  The REAL question is why there appears to be a need to
>>keep allocating more memory to your process when you are freeing storage up.  That is
>>actually a very important question to discover the answer to.
>>*****
>>>
>>>The memory consumption is either shown as "Mem Usage" on the Task
>>>Manager Processes page or by the following code:
>>>
>>>  PROCESS_MEMORY_COUNTERS pmc;
>>>  HANDLE hProcess = OpenProcess(PROCESS_QUERY_INFORMATION 
>>>       | PROCESS_VM_READ, FALSE, _getpid());
>>>  size_t retval = 0;
>>>  if (GetProcessMemoryInfo(hProcess, &pmc, sizeof(pmc)))
>>>      retval = pmc.WorkingSetSize;
>>>  CloseHandle(hProcess);
>>>
>>>The memory used by my process continues to grow and grow indefinitely
>>>even though I close and delete the socket.  That seems intolerable,
>>>whether or not it officially counts as a leak. 
>>>
>>>Maybe the Windows memory manager will eventually discover this and
>>>recoup the memory, but how long will it take? 
>>****
>>Forever.  There is "Windows Memory Manager" that will make such a discovery.  Not in
>>application space, and not in the kernel.  There are many, many levels of allocation going
>>on here and it is essential to distinguish them.  There is the allocation of physical
>>memory, which is handled by the only component I would refer to as the "Windows Memory
>>Manager".  Physical memory is allocated on demand and pages which have not been used in a
>>long time (that is, are not part of the working set) may be removed from physical memory
>>and left instead on the paging file or in the executable image so that other pages which
>>are more in demand may be moved in.  This deals exclusively with how the operating system
>>manages the physical memory of the computer, and has nothing to do with how applications
>>manage their own memory.
>>
>>Application memory is allocated by VirtualAlloc and freed by VirtualFree.  There is no
>>other way to get memory allocated to your application.  However, these allocations are
>>fairly coarse-grained (64K on a typical installation) and would be unsuitable for ordinary
>>allocation problems.  Therefore, the heap allocators manage this memory by allowing it to
>>be split into smaller pieces (HeapAlloc).  Various strategies such as modified-quickfit
>>and such are used to try to minimize fragmentation, but they cannot prevent fragmentation.
>>They merely reduce its effects somewhat.  But there is no mechanism for returning this
>>heap memory to the operating system in the default heap (and it is not clear what
>>HeapDestroy on custom heaps actually does, but I suspect it is the only situation in which
>>it *would* be possible to return memory to the operating system).  So what you are left
>>with is the issue of how memory is allocated internally in your app.  You can use
>>LocalAlloc/LocalFree, although those are not particularly good choices; you typically use
>>malloc/free, or in C++ these are typically wrapped in new/delete.  These functions
>>manipulate the blocks of storage allocated to your app via VirtualAlloc, breaking them
>>into small pieces and handing those pieces out and eventually returning them.  There are
>>various strategies here: quickfit, modified quickfit, first-fit, best-fit,
>>split-reduction, and combined with strategies such as agressive-coalescing and
>>lazy-coalescing, these give different performance and fragmentation profiles.  
>>
>>Generally, tuning performance for memory-intensive applications takes about as long as it
>>took to develop the application in the first place; I've literally spent months taking
>>code that came to me and making it perform well by reducing the costs of memory
>>allocation, and most commonly reducing the effects of memory fragmentation.  In some
>>cases, major rewrites were required because the performance issues had nothing to do with
>>memory footprint at all, but on working set size, and paging was killing them.   It isn't
>>a simple task; you can spend several weeks just gathering data and analyzing it.  What is
>>your percentage of fragmentation?  (Ratio of unused small blocks to total allocation).
>>What is the size distribution of small blocks?  What is the size distribution of all
>>blocks?  What is the memory span?  (This refers to how many pages you would have to touch
>>to process data in one object, and/or the number of pages you would have to touch to
>>iterate across a set of objects).  When does it make sense to preallocate buffers inline
>>(TCHAR something[FIXED_SIZE]) vs. having pointers to buffers (LPTSTR, CString,
>>std::string)?  
>>
>>_heapwalk will become overly familiar to you.  I tend to write files that can be imported
>>to Excel and used to graph what I need to look at.  Or which can be processed by some
>>scripting language into some new set of data to be shown in Excel.  You can spend a couple
>>weeks gathering data, just so you can spend a couple weeks staring at it, just so you can
>>start to form insights in what you need to *actually* gather to do the job, then gathering
>>*that* data, then staring at it, and finally seeing where the memory is going. Pre-MFC/C++
>>I actually wrote my own allocation interfaces that took __FILE__ and __LINE__ and a memory
>>tag, and tagged each block of memory (you get this free in the debug allocator now!) so I
>>knew at any given point exactly how much memory was in use, and why. I would do regular
>>snapshots and watch the balance change over the execution lifetime, and say "OK, how can I
>>change this...".  Back in the days of MS-DOS, I wrote out data which I processed with a
>>scripting language called "PostScript" and it plotted memory usage on my laser printer (I
>>have been an expert PostScript coder in the past), and the pictures were quite
>>interesting.  I am tempted at times to see if I can recover that code from old backup
>>tapes and apply it to my new color laser printer to get REALLY interesting shots, although
>>these days I could probably write the algorithm faster in MFC.  Or use Origin (a graphics
>>package) to do 3-D plots of data allocation over time (it produces VERY sexy graphs).  
>>
>>None of this is really straightforward, all of it requires a fair amount of understanding
>>of the application and its storage usage patterns, and it all requires significant effort
>>to instrument and study (I haven't looked at the performance analysis tools of VS2005 yet,
>>some of this may already be there, but gathering numbers and looking at numbers are
>>separate tasks, and deciding the best strategy to act upon the numbers is yet another
>>phase).  None of this relates to the "Windows Storage Allocator" or HeapAlloc, malloc, or
>>new directly; they may be sound, robust, thread-safe, etc., and STILL fragment you like
>>crazy if you hit them with the wrong patterns.
>>
>>This is one of the reasons I rarely worry about optimizing code these days; real
>>applications need data optimizations, and they are far harder.  
>>
>>Note that a third-party allocator is still not going to solve the basic problem, although
>>by using variants of quickfit and tuning it for either lazy or agressive coalescing may
>>give you a result more tuned to your application's needs, so I wouldn't rule it out.  But
>>it won't change the fact that in C/C++, there can be no compaction, nor can there exist
>>any mechanism to "return storage to the operating system" that could be really effective.
>>In general, any mechanism that could be invented to do this is likely to actually reduce
>>performance by a larger factor than it contributes to improving performance (since the
>>smallest unit that can be freed is 64K, it would require a contiguous 64K-aligned chunk
>>have no objects in it, a situation unlikely to arise in common practice).
>>
>>There are no easy answers.  They all take time.  If you want to do such optimizations,
>>budget yourself a couple months to do them.  Not counting the time taken to retest
>>everything after you've rewritten the algorithms to improve the performance.
>>					joe
>>*****
>>
>>> All I know is that I
>>>have a problem in the field when my program runs month after month and
>>>this is the best clue I can generate on my test bed about what might
>>>be the cause.  I now have a version running in the field that logs its
>>>memory usage once a day, but that will take days and weeks to analyze.
>>>The problem is that the CAsyncSocket delete and recreate only occurs
>>>when there are problems in the field, and they occur only
>>>sporadically.
>>>
>>>Incidentally, the memory consumption occurs both in debug and release
>>>compilations and for both Unicode and simple char string (Character
>>>Set not set) options.  I tested that because CAsyncSocket::Create uses
>>>T2A_EX within CAsyncSocket::Bind and I wonder whether that might be
>>>the problem.
>>>
>>>
>>>>I couldn't find a memory leak.  What you are most likely seeing is windows 
>>>>memory managment doing its work.  The Create method is creating a socket 
>>>>object, when when it's freed the memory is not given back to the system 
>>>>right away.
>>>>
>>>>
>>>>(I used this method to detect a leak)
>>>>
>>>>// Declare the variables needed
>>>>#ifdef _DEBUG
>>>>   CMemoryState oldMemState, newMemState, diffMemState;
>>>>   oldMemState.Checkpoint();
>>>>#endif
>>>>
>>>>   for (int i=0; i<10; ++i)
>>>>   {
>>>>      CAsyncSocket *pAS = new CAsyncSocket;
>>>>      pAS->Create();
>>>>      pAS->Close();
>>>>      delete pAS;
>>>>   }
>>>>
>>>>#ifdef _DEBUG
>>>>   newMemState.Checkpoint();
>>>>   if( diffMemState.Difference( oldMemState, newMemState ) )
>>>>   {
>>>>      TRACE( "Memory leaked!\n" );
>>>>   }
>>>>#endif
>>>>
>>>>AliR.
>>>>
>>>>
>>>>"r norman" <r_s_norman@_comcast.net> wrote in message 
>>>>news:2895939efidggi556s7fbje0euhm2jd2d0@4ax.com...
>>>>>I have traced a memory leak problem to CAsyncSocket::Create().  Is
>>>>> this a known problem?  Is there a workaround/solution/fix?  Here is
>>>>> sample code:
>>>>>
>>>>>  for (int i=0; i<m_nReopenCount; ++i) {
>>>>>    CAsyncSocket *pAS = new CAsyncSocket;
>>>>>    pAS->Create();
>>>>>    pAS->Close();
>>>>>    delete pAS;
>>>>> }
>>>>>
>>>>> Running this 1000 times uses up 1200 KBytes of memory, or just over 1
>>>>> KByte per call.  Commenting out the Create() leaves memory clean. (And
>>>>> please don't complain about my bracketing style -- I like it.)
>>>>>
>>>>> I have Visual Studio 2005 Professional version 8.0.
>>>>>
>>>>> Incidentally, I also discovered that the call to Create() is not
>>>>> re-entrant.  My application involves connecting to some 10 to 20
>>>>> external devices and my normal code creates a CWinThread to support
>>>>> each socket, where the socket is created and destroyed only  within
>>>>> the thread routine.  Creating all the threads and starting them up
>>>>> simultaneously meant having multiple instances of
>>>>> CAsyncSocket::Create() being called at the same time, crashing my
>>>>> system (memory access faults).    That one I found and fixed with
>>>>> sentries.  Now I am left with the memory leak.
>>>>>
>>>>> The problem is that I have an rather intricate communication protocol
>>>>> system all self contained so that adding a new hardware device simply
>>>>> means creating a new instance of the whole works.  It runs fine until
>>>>> the external hardware goes haywire, in which case I destruct the whole
>>>>> instance and start a new one which breaks and reconnects the socket
>>>>> with a clean start and, most of the time, results in a good
>>>>> connection; the external device resets itself through the disconnect.
>>>>> One faulty device, though, generated thousand upon thousand of
>>>>> disconnects over a number of days and, after a few hundred thousand of
>>>>> these I my own system crashed due, I have now found out, to a lack of
>>>>> memory caused by this leak.
>>>>>
>>>>> My application must run essentially as an embedded system, unattended
>>>>> week after week, month after month so I cannot tolerate a memory leak.
>>>>> Does anybody know about this?  Is there a simple clean way to force a
>>>>> socket disconnection on a CAsyncSocket and then reconnect?  My
>>>>> application is the connect() end of the socket, not the listen() end.
>>>>>
>>>>>
>>>>>
>>>>> 
>>>>
>>Joseph M. Newcomer [MVP]
>>email: newcomer@flounder.com
>>Web: http://www.flounder.com
>>MVP Tips: http://www.flounder.com/mvp_tips.htm
Joseph M. Newcomer [MVP]
email: newcomer@flounder.com
Web: http://www.flounder.com
MVP Tips: http://www.flounder.com/mvp_tips.htm
0
newcomer (15974)
7/10/2007 8:08:19 PM
See below...
On Tue, 10 Jul 2007 13:08:46 -0400, r norman <r_s_norman@_comcast.net> wrote:

>On Tue, 10 Jul 2007 11:45:00 -0500, "AliR \(VC++ MVP\)"
><AliR@online.nospam> wrote:
>
>Incidentally, the first time I create and delete CAsyncSocket, it uses
>312K of system memory.  After that it uses only 1K each time.  I am
>not concerned about one-time memory usage, only about continuous
>nibbles eating away system memory.
****
The 312K is probably loading the WInsock DLL and support and creating the initial
structures.

Note that if the leakage is in WinSock, no change you make about managing low-level
sockets (I'm reading these out-of-order) is going to fix your problem.  This is why it is
essential to understand where the lossage is happening.
****
>
>>hummm, I ran a sample that created 1000 sockets and closed them.  The memory 
>>usage went up 1 Meg, I watched it for an hour to see when the memory would 
>>be released, it didn't.
*****
I'm not surprised.  If the memory is being lost, it is being lost, and nothing is going to
magically make it be released at some indefinite future time.
*****
>>
>>I put a call to WSACleanup right after the loop and no more increase in 
>>memory.  Seems like whatever memory is allocated by the sockets gets cleaned 
>>up on after WSACleanup is called!
*****
Don't confuse the removal of the DLL and its basic data structures with the freeing of
other memory.  Again, key here is to see where the storage is being lost.  If it is being
lost in the WinSock library because you are using asynchronous sockets, you are screwed.
If it is being lost in MFC, there is a slight chance you might be able to work around it.
				joe
*****
>>
>>Ok I'm baffled.
>>
>>AliR.
>>
>>"r norman" <r_s_norman@_comcast.net> wrote in message 
>>news:cpb793pdjqar537ms5nctjavvbqmumgo76@4ax.com...
>>> On Tue, 10 Jul 2007 10:33:26 -0500, "AliR \(VC++ MVP\)"
>>> <AliR@online.nospam> wrote:
>>>
>>>>As far as the memory being returned, it will eventually, when the system
>>>>thinks that the program won't need it anymore.
>>>>With the example I posted, before the loop my sample app is using 3676K of
>>>>memory after the loop the program is using 4024K of memory but there is no
>>>>memory leak.
>>>>
>>>>As far as Create not being reentrant goes, I am not really sure what could
>>>>be causing that.  There are some many possibilities as to why it doesn't
>>>>work the way he wants it.   Is Close being called before the next call to
>>>>Create?  Is the socket being passed from one thread to another?  Maybe 
>>>>even
>>>>a corrupted install of Visual Studio.
>>>>
>>>>I think that the OP should try and recreate the problem with one socket in
>>>>the main thread.  And go from there.
>>>
>>> I responded to Joe's post about the concurrency problem and explained
>>> what I was doing there.  It was multiple instances of CAsyncSocket
>>> that was the problem, not successive Creates on the same instance or
>>> sockets shared between threads.  And I found a workaround by
>>> serializing the calls to Create so I left it at that.
>>>
>>> I still worry about Windows memory management and whether it is solid
>>> enough to support embedded programs that run continuously for month
>>> after month.  I know for a fact that older versions of Visual C
>>> programs (version 3.0 or 4.0 if my memory serves me well) running on
>>> 16 bit MS-DOS  failed miserably -- memory became so fragmented that it
>>> was unusable even if the total available byte count was maintained. At
>>> that time, I had to go to a commercial third party memory manager. I
>>> only reluctantly migrated from C to C++ because of my fears of  the
>>> object oriented code's proclivity to create and delete objects at
>>> will, thereby stressing the memory manager to its limits.  My roots
>>> are in 8 bit embedded systems with really severe memory constraints
>>> and I still have a probably unfounded fear of that problem.
>>>
>>>
>>>
>>>
>>>
>>>>AliR.
>>>>
>>>>
>>>>"Joseph M. Newcomer" <newcomer@flounder.com> wrote in message
>>>>news:2c87935kti783rkaf53n5d28th16khebq9@4ax.com...
>>>>>I would have thought it *would* be returned.
>>>>>
>>>>> I'm also concerned about the concurrency problem, because I've not seen
>>>>> that particular
>>>>> problem in MFC before.  I'm wondering if there is some storage damage 
>>>>> that
>>>>> is causing both
>>>>> apparent problems.
>>>>> joe
>>>>>
>>>>> On Tue, 10 Jul 2007 09:45:46 -0500, "AliR \(VC++ MVP\)"
>>>>> <AliR@online.nospam> wrote:
>>>>>
>>>>>>I couldn't find a memory leak.  What you are most likely seeing is 
>>>>>>windows
>>>>>>memory managment doing its work.  The Create method is creating a socket
>>>>>>object, when when it's freed the memory is not given back to the system
>>>>>>right away.
>>>>>>
>>>>>>
>>>>>>(I used this method to detect a leak)
>>>>>>
>>>>>>// Declare the variables needed
>>>>>>#ifdef _DEBUG
>>>>>>   CMemoryState oldMemState, newMemState, diffMemState;
>>>>>>   oldMemState.Checkpoint();
>>>>>>#endif
>>>>>>
>>>>>>   for (int i=0; i<10; ++i)
>>>>>>   {
>>>>>>      CAsyncSocket *pAS = new CAsyncSocket;
>>>>>>      pAS->Create();
>>>>>>      pAS->Close();
>>>>>>      delete pAS;
>>>>>>   }
>>>>>>
>>>>>>#ifdef _DEBUG
>>>>>>   newMemState.Checkpoint();
>>>>>>   if( diffMemState.Difference( oldMemState, newMemState ) )
>>>>>>   {
>>>>>>      TRACE( "Memory leaked!\n" );
>>>>>>   }
>>>>>>#endif
>>>>>>
>>>>>>AliR.
>>>>>>
>>>>>>
>>>>>>"r norman" <r_s_norman@_comcast.net> wrote in message
>>>>>>news:2895939efidggi556s7fbje0euhm2jd2d0@4ax.com...
>>>>>>>I have traced a memory leak problem to CAsyncSocket::Create().  Is
>>>>>>> this a known problem?  Is there a workaround/solution/fix?  Here is
>>>>>>> sample code:
>>>>>>>
>>>>>>>  for (int i=0; i<m_nReopenCount; ++i) {
>>>>>>>    CAsyncSocket *pAS = new CAsyncSocket;
>>>>>>>    pAS->Create();
>>>>>>>    pAS->Close();
>>>>>>>    delete pAS;
>>>>>>> }
>>>>>>>
>>>>>>> Running this 1000 times uses up 1200 KBytes of memory, or just over 1
>>>>>>> KByte per call.  Commenting out the Create() leaves memory clean. (And
>>>>>>> please don't complain about my bracketing style -- I like it.)
>>>>>>>
>>>>>>> I have Visual Studio 2005 Professional version 8.0.
>>>>>>>
>>>>>>> Incidentally, I also discovered that the call to Create() is not
>>>>>>> re-entrant.  My application involves connecting to some 10 to 20
>>>>>>> external devices and my normal code creates a CWinThread to support
>>>>>>> each socket, where the socket is created and destroyed only  within
>>>>>>> the thread routine.  Creating all the threads and starting them up
>>>>>>> simultaneously meant having multiple instances of
>>>>>>> CAsyncSocket::Create() being called at the same time, crashing my
>>>>>>> system (memory access faults).    That one I found and fixed with
>>>>>>> sentries.  Now I am left with the memory leak.
>>>>>>>
>>>>>>> The problem is that I have an rather intricate communication protocol
>>>>>>> system all self contained so that adding a new hardware device simply
>>>>>>> means creating a new instance of the whole works.  It runs fine until
>>>>>>> the external hardware goes haywire, in which case I destruct the whole
>>>>>>> instance and start a new one which breaks and reconnects the socket
>>>>>>> with a clean start and, most of the time, results in a good
>>>>>>> connection; the external device resets itself through the disconnect.
>>>>>>> One faulty device, though, generated thousand upon thousand of
>>>>>>> disconnects over a number of days and, after a few hundred thousand of
>>>>>>> these I my own system crashed due, I have now found out, to a lack of
>>>>>>> memory caused by this leak.
>>>>>>>
>>>>>>> My application must run essentially as an embedded system, unattended
>>>>>>> week after week, month after month so I cannot tolerate a memory leak.
>>>>>>> Does anybody know about this?  Is there a simple clean way to force a
>>>>>>> socket disconnection on a CAsyncSocket and then reconnect?  My
>>>>>>> application is the connect() end of the socket, not the listen() end.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>> Joseph M. Newcomer [MVP]
>>>>> email: newcomer@flounder.com
>>>>> Web: http://www.flounder.com
>>>>> MVP Tips: http://www.flounder.com/mvp_tips.htm
>>>> 
>>
Joseph M. Newcomer [MVP]
email: newcomer@flounder.com
Web: http://www.flounder.com
MVP Tips: http://www.flounder.com/mvp_tips.htm
0
newcomer (15974)
7/10/2007 8:12:09 PM
"r norman" <r_s_norman@_comcast.net> wrote in message
news:2895939efidggi556s7fbje0euhm2jd2d0@4ax.com...
> I have traced a memory leak problem to CAsyncSocket::Create().  Is
> this a known problem?  Is there a workaround/solution/fix?  Here is
> sample code:
>
>   for (int i=0; i<m_nReopenCount; ++i) {
>     CAsyncSocket *pAS = new CAsyncSocket;
>     pAS->Create();
>     pAS->Close();
>     delete pAS;
> }
>
> Running this 1000 times uses up 1200 KBytes of memory, or just over 1
> KByte per call.  Commenting out the Create() leaves memory clean. (And
> please don't complain about my bracketing style -- I like it.)
>


This is a known problem, acknowledged by Microsoft.

Apparently, when using CAsyncSocket in a GUI app that also has XP visual
styles (themes), there is a memory leak.  The problem seems to be
OS-dependent, and it might exist only under XP (running themes). It did not
manifest itself under Win 2000, and it might be gone now from Vista.

For one report, see "CSocket Consuming Memory Uncontrollably" at
http://www.codeguru.com/forum/showthread.php?t=353944 .  I am fairly certain
that this is the same problem that you are seeing, since this report also
includes the exact same number for the amount of leaked memory (i.e., 1200
bytes per call to Create()).

There is a parallel posting in the newsgroups at "HELP!! CSocket and
CAsyncSocket Consumes Memory Uncontrollably" in the
microsoft.public.win32.programmer.networks newsgroup, at
http://groups.google.com/group/microsoft.public.win32.programmer.networks/browse_frm/thread/8490cdb5c4f18c76/b87332dbee0cb54c?tvc=1

The "solution", if you can call it that, is to disable themes.  See
"CAsynCSocket Memory Leak Fix" at
http://www.codeguru.com/forum/showthread.php?t=370761

Mike


0
7/10/2007 8:45:46 PM
On Tue, 10 Jul 2007 13:45:46 -0700, "Michael K. O'Neill"
<MikeAThon2000@nospam.hotmail.com> wrote:

My sincere thanks to your knowledge of this problem.  I haven't
enabled themes (to my knowledge) on my system, but this may do it.  If
it does, it saves me a lot of work.  If it doesn't, at least I know
that the problem isn't me doing stupid things.  Or at least, part of
the problem isn't;  no doubt there are more stupid things that are
really my fault still waiting to bite me.

>
>"r norman" <r_s_norman@_comcast.net> wrote in message
>news:2895939efidggi556s7fbje0euhm2jd2d0@4ax.com...
>> I have traced a memory leak problem to CAsyncSocket::Create().  Is
>> this a known problem?  Is there a workaround/solution/fix?  Here is
>> sample code:
>>
>>   for (int i=0; i<m_nReopenCount; ++i) {
>>     CAsyncSocket *pAS = new CAsyncSocket;
>>     pAS->Create();
>>     pAS->Close();
>>     delete pAS;
>> }
>>
>> Running this 1000 times uses up 1200 KBytes of memory, or just over 1
>> KByte per call.  Commenting out the Create() leaves memory clean. (And
>> please don't complain about my bracketing style -- I like it.)
>>
>
>
>This is a known problem, acknowledged by Microsoft.
>
>Apparently, when using CAsyncSocket in a GUI app that also has XP visual
>styles (themes), there is a memory leak.  The problem seems to be
>OS-dependent, and it might exist only under XP (running themes). It did not
>manifest itself under Win 2000, and it might be gone now from Vista.
>
>For one report, see "CSocket Consuming Memory Uncontrollably" at
>http://www.codeguru.com/forum/showthread.php?t=353944 .  I am fairly certain
>that this is the same problem that you are seeing, since this report also
>includes the exact same number for the amount of leaked memory (i.e., 1200
>bytes per call to Create()).
>
>There is a parallel posting in the newsgroups at "HELP!! CSocket and
>CAsyncSocket Consumes Memory Uncontrollably" in the
>microsoft.public.win32.programmer.networks newsgroup, at
>http://groups.google.com/group/microsoft.public.win32.programmer.networks/browse_frm/thread/8490cdb5c4f18c76/b87332dbee0cb54c?tvc=1
>
>The "solution", if you can call it that, is to disable themes.  See
>"CAsynCSocket Memory Leak Fix" at
>http://www.codeguru.com/forum/showthread.php?t=370761
>
>Mike
>
0
r
7/10/2007 10:26:06 PM
On Tue, 10 Jul 2007 18:26:06 -0400, r norman <r_s_norman@_comcast.net>
wrote:

That did the trick! Many, many thanks, Michael.  I don't understand
it, but then if it works, I don't care.  The first CAsyncSocket uses a
bunch of memory no doubt because that is when all the socket code gets
loaded, as Joe pointed out.  From then on it is all 0 bytes per
instance!

And many thanks again both to you Joe and you AliR for your major
assistance in getting to the bottom of this. 

The problem, as Joe points out, is how  such a serious problem can go
unfixed and unremarked.  I won't  comment  on the corporation that is
responsible because I use and depend on so much of their products and
have done so since CP/M MASM and LINK.  But the many  people who blow
a gasket at the mere hint of that corporate name do have at least some
justification.

>On Tue, 10 Jul 2007 13:45:46 -0700, "Michael K. O'Neill"
><MikeAThon2000@nospam.hotmail.com> wrote:
>
>My sincere thanks to your knowledge of this problem.  I haven't
>enabled themes (to my knowledge) on my system, but this may do it.  If
>it does, it saves me a lot of work.  If it doesn't, at least I know
>that the problem isn't me doing stupid things.  Or at least, part of
>the problem isn't;  no doubt there are more stupid things that are
>really my fault still waiting to bite me.
>
>>
>>"r norman" <r_s_norman@_comcast.net> wrote in message
>>news:2895939efidggi556s7fbje0euhm2jd2d0@4ax.com...
>>> I have traced a memory leak problem to CAsyncSocket::Create().  Is
>>> this a known problem?  Is there a workaround/solution/fix?  Here is
>>> sample code:
>>>
>>>   for (int i=0; i<m_nReopenCount; ++i) {
>>>     CAsyncSocket *pAS = new CAsyncSocket;
>>>     pAS->Create();
>>>     pAS->Close();
>>>     delete pAS;
>>> }
>>>
>>> Running this 1000 times uses up 1200 KBytes of memory, or just over 1
>>> KByte per call.  Commenting out the Create() leaves memory clean. (And
>>> please don't complain about my bracketing style -- I like it.)
>>>
>>
>>
>>This is a known problem, acknowledged by Microsoft.
>>
>>Apparently, when using CAsyncSocket in a GUI app that also has XP visual
>>styles (themes), there is a memory leak.  The problem seems to be
>>OS-dependent, and it might exist only under XP (running themes). It did not
>>manifest itself under Win 2000, and it might be gone now from Vista.
>>
>>For one report, see "CSocket Consuming Memory Uncontrollably" at
>>http://www.codeguru.com/forum/showthread.php?t=353944 .  I am fairly certain
>>that this is the same problem that you are seeing, since this report also
>>includes the exact same number for the amount of leaked memory (i.e., 1200
>>bytes per call to Create()).
>>
>>There is a parallel posting in the newsgroups at "HELP!! CSocket and
>>CAsyncSocket Consumes Memory Uncontrollably" in the
>>microsoft.public.win32.programmer.networks newsgroup, at
>>http://groups.google.com/group/microsoft.public.win32.programmer.networks/browse_frm/thread/8490cdb5c4f18c76/b87332dbee0cb54c?tvc=1
>>
>>The "solution", if you can call it that, is to disable themes.  See
>>"CAsynCSocket Memory Leak Fix" at
>>http://www.codeguru.com/forum/showthread.php?t=370761
>>
>>Mike
>>
0
r
7/10/2007 11:36:29 PM
On Tue, 10 Jul 2007 13:45:46 -0700, "Michael K. O'Neill"
<MikeAThon2000@nospam.hotmail.com> wrote:

>"r norman" <r_s_norman@_comcast.net> wrote in message
>news:2895939efidggi556s7fbje0euhm2jd2d0@4ax.com...
>> I have traced a memory leak problem to CAsyncSocket::Create().  Is
>> this a known problem?  Is there a workaround/solution/fix?  Here is
>> sample code:
>>
>>   for (int i=0; i<m_nReopenCount; ++i) {
>>     CAsyncSocket *pAS = new CAsyncSocket;
>>     pAS->Create();
>>     pAS->Close();
>>     delete pAS;
>> }
>>
>> Running this 1000 times uses up 1200 KBytes of memory, or just over 1
>> KByte per call.  Commenting out the Create() leaves memory clean. (And
>> please don't complain about my bracketing style -- I like it.)
>>
>
>This is a known problem, acknowledged by Microsoft.
>
>Apparently, when using CAsyncSocket in a GUI app that also has XP visual
>styles (themes), there is a memory leak.  The problem seems to be
>OS-dependent, and it might exist only under XP (running themes). It did not
>manifest itself under Win 2000, and it might be gone now from Vista.
>
>For one report, see "CSocket Consuming Memory Uncontrollably" at
>http://www.codeguru.com/forum/showthread.php?t=353944 .  I am fairly certain
>that this is the same problem that you are seeing, since this report also
>includes the exact same number for the amount of leaked memory (i.e., 1200
>bytes per call to Create()).
>
>There is a parallel posting in the newsgroups at "HELP!! CSocket and
>CAsyncSocket Consumes Memory Uncontrollably" in the
>microsoft.public.win32.programmer.networks newsgroup, at
>http://groups.google.com/group/microsoft.public.win32.programmer.networks/browse_frm/thread/8490cdb5c4f18c76/b87332dbee0cb54c?tvc=1
>
>The "solution", if you can call it that, is to disable themes.  See
>"CAsynCSocket Memory Leak Fix" at
>http://www.codeguru.com/forum/showthread.php?t=370761

Interesting! The problem appears to be the creation of a top-level,
non-child window by CAsyncSocket::AttachHandle. For example, if I replace
norman's loop body with the following, I observe the same "leak" in Process
Explorer:

      CWnd* p = new CSocketWnd;
#if 1
      p->CreateEx(0, AfxRegisterWndClass(0), 0,
            WS_OVERLAPPED, CRect(), 0, 0);
#else
      p->Create(AfxRegisterWndClass(0), 0,
            WS_CHILD, CRect(), AfxGetMainWnd(), 0);
#endif
      p->DestroyWindow();
      delete p;

If I use #if 0, a child window is created instead, and I don't observe the
leak.

-- 
Doug Harrison
Visual C++ MVP
0
dsh (2498)
7/11/2007 1:50:33 AM
"Doug Harrison [MVP]" <dsh@mvps.org> wrote in message
news:kad893pun39rv6hlti7dt50kgu2j43kue2@4ax.com...
> On Tue, 10 Jul 2007 13:45:46 -0700, "Michael K. O'Neill"
> <MikeAThon2000@nospam.hotmail.com> wrote:
>
> >"r norman" <r_s_norman@_comcast.net> wrote in message
> >news:2895939efidggi556s7fbje0euhm2jd2d0@4ax.com...
> >> I have traced a memory leak problem to CAsyncSocket::Create().  Is
> >> this a known problem?  Is there a workaround/solution/fix?  Here is
> >> sample code:
> >>
> >>   for (int i=0; i<m_nReopenCount; ++i) {
> >>     CAsyncSocket *pAS = new CAsyncSocket;
> >>     pAS->Create();
> >>     pAS->Close();
> >>     delete pAS;
> >> }
> >>
> >> Running this 1000 times uses up 1200 KBytes of memory, or just over 1
> >> KByte per call.  Commenting out the Create() leaves memory clean. (And
> >> please don't complain about my bracketing style -- I like it.)
> >>
> >
> >This is a known problem, acknowledged by Microsoft.
> >
> >Apparently, when using CAsyncSocket in a GUI app that also has XP visual
> >styles (themes), there is a memory leak.  The problem seems to be
> >OS-dependent, and it might exist only under XP (running themes). It did
not
> >manifest itself under Win 2000, and it might be gone now from Vista.
> >
> >For one report, see "CSocket Consuming Memory Uncontrollably" at
> >http://www.codeguru.com/forum/showthread.php?t=353944 .  I am fairly
certain
> >that this is the same problem that you are seeing, since this report also
> >includes the exact same number for the amount of leaked memory (i.e.,
1200
> >bytes per call to Create()).
> >
> >There is a parallel posting in the newsgroups at "HELP!! CSocket and
> >CAsyncSocket Consumes Memory Uncontrollably" in the
> >microsoft.public.win32.programmer.networks newsgroup, at
>
>http://groups.google.com/group/microsoft.public.win32.programmer.networks/b
rowse_frm/thread/8490cdb5c4f18c76/b87332dbee0cb54c?tvc=1
> >
> >The "solution", if you can call it that, is to disable themes.  See
> >"CAsynCSocket Memory Leak Fix" at
> >http://www.codeguru.com/forum/showthread.php?t=370761
>
> Interesting! The problem appears to be the creation of a top-level,
> non-child window by CAsyncSocket::AttachHandle. For example, if I replace
> norman's loop body with the following, I observe the same "leak" in
Process
> Explorer:
>
>       CWnd* p = new CSocketWnd;
> #if 1
>       p->CreateEx(0, AfxRegisterWndClass(0), 0,
>             WS_OVERLAPPED, CRect(), 0, 0);
> #else
>       p->Create(AfxRegisterWndClass(0), 0,
>             WS_CHILD, CRect(), AfxGetMainWnd(), 0);
> #endif
>       p->DestroyWindow();
>       delete p;
>
> If I use #if 0, a child window is created instead, and I don't observe the
> leak.
>
> -- 
> Doug Harrison
> Visual C++ MVP

That *is* very interesting.  When we looked at this two years ago, we
noticed that the leak disappeared when creating a child window, but we
didn't look any furhter, and we didn't think to try a top-level window.  We
eventually concluded that the problem was related to the creation of the
socket's hidden window, but that it didn't show itself unless XP visual
styles (themes) was enabled, i.e., inclusion of a manifest with the program.

With a top-level window, does it make a difference as to whether themes is
or is not enabled?  Was our answer back then completely off-base?

Mike


0
7/11/2007 4:19:23 AM
On Tue, 10 Jul 2007 21:19:23 -0700, "Michael K. O'Neill"
<mikeathon2000@nospam.hotmail.com> wrote:

>That *is* very interesting.  When we looked at this two years ago, we
>noticed that the leak disappeared when creating a child window, but we
>didn't look any furhter, and we didn't think to try a top-level window.  We
>eventually concluded that the problem was related to the creation of the
>socket's hidden window, but that it didn't show itself unless XP visual
>styles (themes) was enabled, i.e., inclusion of a manifest with the program.
>
>With a top-level window, does it make a difference as to whether themes is
>or is not enabled?  Was our answer back then completely off-base?

I haven't tested the theme aspect, but assuming it's correct, there's
clearly a bad interaction between top-level windows and themes that has
nothing to do with sockets. The CreateEx call is a simplified version of
what CAsyncSocket::AttachHandle does. You can easily test this. Just create
a default MFC SDI app and put the loop in OnAppAbout; at least, that's my
favorite way to test stuff like this. So I had:

void Cdef_sockApp::OnAppAbout()
{
   for (int i=0; i<1000; ++i)
   {
#if 0
      CAsyncSocket *pAS = new CAsyncSocket;
      pAS->Create();
      pAS->Close();
      delete pAS;
#else
      CWnd* p = new CWnd;
#if 1
      p->CreateEx(0, AfxRegisterWndClass(0), 0,
            WS_OVERLAPPED, CRect(), 0, 0);
#else
      p->Create(AfxRegisterWndClass(0), 0,
            WS_CHILD, CRect(), AfxGetMainWnd(), 0);
#endif
      p->DestroyWindow();
      delete p;
#endif
   }

   CAboutDlg aboutDlg;
   aboutDlg.DoModal();
}

The next step would be to reduce this to the Windows API level and do it
all in terms of raw HWNDs. The window and window class styles might also be
a factor.

-- 
Doug Harrison
Visual C++ MVP
0
dsh (2498)
7/11/2007 5:41:54 AM
Reply:

Similar Artilces:

Delivery Failure after creating calendar event
> This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. --B_3205343077_45724 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Folks, Every time I Create a calendar event, I receive the following email: This is an automatically generated Delivery Status Notification. Delivery to the following recipients failed. /O=First Organization/ou=First Administrative Group/cn=Configuration/cn=Servers/cn=MKPSERVER/cn=Microsoft System Attendant Reporting-MTA: dns;mar...

The virtual memory necessary to run your Exchange server is fragmented
I get the following error on a daily basis... The virtual memory necessary to run your Exchange server is fragmented in such a way that normal operation may begin to fail. It is highly recommended that you restart all Exchange services to correct this issue. I have exchange 2000, service pack 3. I have failed to get any workable answers. Any ideas? http://support.microsoft.com/default.aspx?scid=kb;en-us;325044 -- Bharat Suneja MCSE, MCT -------------------------------- "asdfg" <asda@asda.com> wrote in message news:eXfxsVfvFHA.2504@TK2MSFTNGP15.phx.gbl... >I get t...

How do I create an area and line combination chart?
I want tocreate a combination chart, using the chart types area and line, it is not listed as a custom type in Excel - is it possible? Hi, Ignore the built-in combinations and create your own. Start with creating a Line chart with your data. Then select the line series that you want as an area chart. Right click and pick Chart Type to select the required Area chart type. Cheers Andy Nuala wrote: > I want tocreate a combination chart, using the chart types area and line, it > is not listed as a custom type in Excel - is it possible? -- Andy Pope, Microsoft MVP - Excel http://ww...

Need to create a printed order from a row of data
Maybe that's not the best way to do it but this is the goal: We need to enter a row of data. Order number, color, size, thickness, etc for an ongoing file for recording this information. CURRENTLY, what we now have to do is, after we enter the row, we have to get a pre-printed form, fill out the SAME information by hand, for faxing to various places. Is there any way to incorporate both operations simply by entering the data in the row, then having a nicely formatted form print out with all the information filled out? I know that this probably requires a way too detailed response...

Missing Measures in .cub file created in Excel
After using Excel's Offline OLAP features which create a local cube file (.cub), many of the desired measures which I included in the wizard setup are not displayed. I'm looking for a reason for this. The data populating this cube file is from a virtual cube, so background calculations take place. I considered that those measures requiring calculation would not be included once seperated from the original virtual cube... but many calculated measures WERE included so I don't think that's the complete explination. I've varied the measures included in the .cub file...

Creating a rule for email by date in OWA?
Is there a way to create a rule using the OWA client to move all email older than 14 days to another folder and out of the inbox folder? thanks. Don't think so. OWA does not give you the same rich Rule functionality provided by Outlook client, though whatever's available is better than no Rule functionality at all. -- Bharat Suneja MCSE, MCT www.zenprise.com blog: www.suneja.com/blog ----------------------------------------- "Jason" <none@none.invalid> wrote in message news:eBOx8QlFGHA.2912@tk2msftngp13.phx.gbl... > Is there a way to create a rule using th...

Memory leakage!!
Please look at the following URL: http://forums.devbox.net/showthread.php?p=3082#post3082 Can anyone suggest me the solution to this? Thanks a lot!! Do you still have a leak if you call Clear on that variant? Andy Mortimer [MS] Please do not send email directly to this alias. This alias is for newsgroup purposes only This posting is provided "AS IS" with no warranties, and confers no rights. OR if you wish to include a script sample in your post please add "Use of included script samples are subject to the terms specified at http://www.microsoft.com/info/cpyright.htm...

Invalid object name when I try to create a scaler udf in a stored procedure.
I am trying to create a report that shows a company name, all the advisors in that company, then the company's stats (there should be one row for each company). Each of these come from a different table. The advisors are currently being listed as one per line, which means the company name and the stats are all being repeated unneccesarily. I want to concatinate the advisors, so that all the ones that belong to the same company show in the same cell. I wrote this UDF which is located right before my SP: CREATE FUNCTION [dbo].[ConcatAdvisorNames](@Company as varchar(20)) RETURNS VA...

Error 8000ffff when creating 2-way ADC
Lab set up with copies of our Exchange 5.5 and Windows 2000 AD directories, attempting to test upgrade from 5.5 to 2003. Error message shows up on screen whenever a two-way Recipient CA of any kind is created: "An unexpected error occurred. ID no. 8000ffff Microsoft Active Directory Connector Management". This error prevents the CA from being actually created. You can vary the Containers, the object classes involved (e.g., Groups only, Contacts only, Users only), account permissions, anything. As long as it's two-way Recipient, it fails. This does not happen with a Public Fold...

Creating thousands of hyperlinks in Excel
I'm creating a huge directory and I'd like to include internal hyperlinks. Is there a FAST way to do this? i.e. I need to create 2000 hyperlinks, leading from one cell to a cell on another worksheet. Tips? Thanks! I'd use the =hyperlink() worksheet function. David McRitchie posted this and it might help you: =HYPERLINK("#"&CELL("address",C5),C5) =HYPERLINK("#"&CELL("address",sheetone!C5),sheetone!C5) =HYPERLINK("#"&CELL("address",'sheet two'!C5),'sheet two'!C5) Megan wrote: > >...

How to create a report based on multiple queries
Hi all, I am trying to create a report based on 30 queries . These 30 queries are based on one table. When I use the Report wizard and select more than one query I get the message " You have chosen fields from record source which the wizard can't connect. You may have chosen fields from a table and from a query based on that table. If so try choosing fields from only tha table or only the query". I do not know what to do.? Any guidance will help thanks, Dave On Wed, 29 Aug 2007 08:21:02 -0700, davidstevans@gmail.com wrote: >Hi all, > >I am trying to create a rep...

How to create a View using Jet SQL in Access?
Can any body tell me how to create a View in Access using Jet SQL? And where to get the detailed specific reference for Jet SQL? The Access/Jet equivalent to a view is a Select query. Jet SQL is documented in Microsoft Access help. In recent English-language versions, display the Help window and look on the Contents tab for "Microsoft Jet SQL reference". On Sun, 7 Sep 2003 00:16:39 +0800, "lu_yanfeng" <lu_yanfeng@163.com> wrote: >Can any body tell me how to create a View in Access using Jet SQL? And where >to get the detailed specific reference for Jet SQ...

GP 10
We are on GP10 and getting the following error... "a save operation on table SY_Period_HDR (45)" when we try to create a new fiscal period. We ran the database maintenance tool thinking that might resolve a stored proc issue but that didnt help. Has anyone ran into this? ...

How do I create Folders/Sub folders from User Form control button
I have a User Form that I would like to take one of the spare control buttons and have it set up my job folder on the hard drive. I am not sure how to code it to achive the following results: These are my text box ID's Main Folder --------------------------------------------------------------------- User Selects drive: \ CES_No_1 \ CLLI_Code_1\ TEO_No_1 Sub Folders inside Main Folder ---------------------------------------------------------------------- Completed Drawings Elec Job Folder Misc Job Documents Site Pictures If you can dig up a copy of the VBS doc...

Unable to click on charts & unable to create any new charts Excel
Hi Just started using Microsoft Excel 2007 and I seem to have done something that means the chart option is unavailable. The chart section is now grey and all charts I have created are 'locked' I cannot click on them anywhere. I'm working on a project and need it finished by Monday. Do you mean that if you click on a chart it does not get a border to show it is activated and there are no range-finder borders around the data that makes the chart? This is a bit odd. Have you tried a new workbook to see if the problem is with just this one? My advice with all odd things is...

dynamically creating a list of controls
The user needs to be able to create a single object composed of other various elements. To specify a single element, the user will have to fill out a series of controls, which looks like this: Cedit |combobox| cedit ||| cedit How do you dynamically create a list of these controls? I was thinking I could create a borderless dialog template that contains the series of controls, then create a dialog class(or window class?) that allows me to externally manipulate the control's values. Whenever an user wants to create a new element I simply create a new modeless dialog(or window clas...

Insert Contact Name and E-mail address into Subject When Creating A Phone Call
Hi guys, I am looking to insert the name of the sender\contact and e-mail address into the phone call subject line. I have figured out how to get the name of the sender inserted into the subject line but cannot figure out how to get the e-mail address in there. The code I used is below. Can anyone outline what code I need to use to get at the e-mail address, maybe it's not actually possible? Cheers, Mark var lookupItem = new Array(); //Get the lookup for the from attribute on the phone call form, this is the person calling lookupItem = crmForm.all.from.DataValue; //Set the current sub...

Create Order functionality
Can someone point me to some references and/or guides on what CRM's "Create Order" button offers in terms of functionality. What can you do when you close a Quote and then "Create an Order" from it? Any help would be greatly appreciated. ...

How do I create an introduction for my website using Publisher 20.
I have my website posted, but I want to create a short introduction video, possibly from Powerpoint, just showing my website name and with maybe some audio merging into the first page of the website. Also, is there a way in publisher to keep music going the whole time during all pages in the website? Check out http://www.atxbass.com to see what I'm working with Don't know how, but I know when I visit a site with music that cannot be turned off, my stay on the site is brief and a one time only. Might think about making your pages more narrow... -- Mary Sauer MS MVP http://off...

Creating an offline OLAP cube
HELP! So I try to create an offline OLAP cube using the New Database Query dialog with its options. When I go to select a data source, it only gives me the option of ODBC cataloged databases (I am trying to get to DB2). Interestingly enough, when I just use Import Data it gives the the OLE DB route. Anyway. So I get the data and I can see it, but when I select the option to build an OLAP cube it crashes and gives me "Format of the initialization string does not conform to the OLE DB specification" and crashes the OLAP building part (well errors) but will return the data. Now, ...

Invoice created from Excel ocr Access and sent via Outlook
Hi, I am all new to this and do not know any programming. I have an excel sheet with Name, Address, e-mail address, amount to pay and an ID number. Is it possible to automate this and send an email with invoice to each and everyone of them? I would like for Outlook to atleast to put the different information in the body of the message to make an invoice. If I have to make another solution with the distribution addresses that is ok with me. I just dont want to copy and paste all the hondreds of posts into e-mails, because something will definatly go wrong, and it is very ...

Creating Visual Excel Cell Alerts
Hi there guys, I'm relatively new to this but would appreciate your help. Basically what I am trying to do in excell to create visual alerts in the same cell that I am inserting values. For example in the cell I am entering data (for example a weight in kg) for all entered data in the range of 0 - 200kg I would like the background in the same cell to show as green (highlighting within weight limits). For all data entered between 200kg - 250kg I would like the background to show yellow (diplaying slightly overweight) and for entered data between 250kg - 300kg I would like ...

"create mscrm.sql" error on 3rd (re-)install
I have installed CRM 1.2 on SBS2K3 twice as "advetureworks cycle" and "msdn subscriber" with only 90% success because of conflicts on the web site. I have removed the conflicts but now CRM installer errors trying to create the new database as (default) local service. My login has SQLServer db_creator permissions and I can create a DB, so I'm totally confused. Can I workaround this somehow ? What login needs to be given db_creator rights so Install can run te SQL scripts ? Hi Marcus, have you deleate the complete the old date from AWC? Have you delete the ...

Memory leak #9
Hello- I have an unmanaged C++ application that's always run great when compiled with Visual Studio 2003. However, when I compile and run the same code under Visual Studio 2005, it leaks about 100k per second! When I close down the application, no MFC memory leaks are reported in the debugger console. The app makes pretty extensive use of STL. Does anyone know what changes between the compilers could cause this? Thanks. <rtberman@gmail.com> wrote in message news:1154228452.253201.189740@i3g2000cwc.googlegroups.com... >... > Does anyone know what changes between the comp...

VB.Net program to create XML file
I need to create an xml file from a VB program using fields returned from an sql query. I have a template that was provided for me and need to use verbatim. Since this is just a text file, I gather that I can just write out the file with all of the tags, etc., and name it .xml. Assuming that is correct what I might want to do is create a template so that if this file changes the names of the elements, all I have to do is change the template and if more elements get added I can then add them. Is there a tool that can do this for me? If I have to create one, any suggestions as to what I ...