WriteFile and ERROR_INSUFFICIENT_BUFFER

I'm using the WriteFileEx function to write a file asynchronously. 
However I sometimes get the error code ERROR_INSUFFICIENT_BUFFER (with 
GetLastError), although the function still returns a success and the 
data gets written ok. However I don't think it is actually doing an 
asynchronous write.

I open the file with a call to CreateFile, with the flags: 
FILE_ATTRIBUTE_NORMAL | FILE_FLAG_OVERLAPPED | FILE_FLAG_NO_BUFFERING

To satisfy the No buffering, I use VirtualAlloc and make sure 
source/dest and size values are all aligned to sector sizes. That bit 
seems fine.

Also, if I try to write out a block bigger than around 60Mb in one go, I 
get an error ERROR_NO_SYSTEM_RESOURCES and the whole thing fails. Any 
ideas? Although I have managed to work around this by writing the file 
out in chunks. Have tried it first by setting all the chunks off at the 
same time and also by setting the next chunk off in the callback 
completion function. Both methods have no effect on the 
ERROR_INSUFFICIENT_BUFFER error or on which chunk triggers this error.

Any suggestions?

Arthur.
0
arthur4871 (13)
8/23/2005 11:00:50 AM
vc.mfc 33608 articles. 0 followers. Follow

3 Replies
1606 Views

Similar Articles

[PageSpeed] 28

If the function returns a success code, then the meaning of GetLastError is irrelevant.
GetLastError is meaningful ONLY if you get an error return.  Otherwise, it is a random
number.

for example, this code is erroneous:

BOOL ok = WriteFile(...);
if(GetLastError() != ERROR_SUCCESS)
    ... deal with error here

and is approximately as meaningful as

BOOL ok = WriteFile(...);
if(rand() != ERROR_SUCCESS)
   ...deal with error here

this code is correct:

BOOL ok = WriteFile(...);
if(!ok)
    { /* failed */
      DWORD err = ::GetLastError();
     ...
   } /* failed */
else
  { /* success */
  ...
  } /* success */

ERROR_NO_SYSTEM_RESOURCES means that you can't do that.  60MB requires that the system
lock down 60MB of pages while the I/O is in progress.  If you can't allocate 60MB of
contiguous kernel space (to hold an alias map) and/or cannot find 60MB of physical pages
to lock down, it can't be done.  So it means you can't write 60MB, and the workaround is
to break your transfer into smaller pieces.

It is not clear why VirtualAlloc is required to deal with the no buffering flag; new
BYTE[n] will work just as well; just make sure you allocate enough space and then start at
the next sector-sized boundary within the allocate space.

int nsectors = ... number of sectors

LPBYTE b = new BYTE[(nsectors + 1) * SECTOR_SIZE];

LPBYTE buffer = (b + SECTOR_SIZE) & ~(SECTOR_SIZE - 1);

				joe

On Tue, 23 Aug 2005 11:00:50 GMT, Arthur <arthur@fubaby.com> wrote:

>I'm using the WriteFileEx function to write a file asynchronously. 
>However I sometimes get the error code ERROR_INSUFFICIENT_BUFFER (with 
>GetLastError), although the function still returns a success and the 
>data gets written ok. However I don't think it is actually doing an 
>asynchronous write.
>
>I open the file with a call to CreateFile, with the flags: 
>FILE_ATTRIBUTE_NORMAL | FILE_FLAG_OVERLAPPED | FILE_FLAG_NO_BUFFERING
>
>To satisfy the No buffering, I use VirtualAlloc and make sure 
>source/dest and size values are all aligned to sector sizes. That bit 
>seems fine.
>
>Also, if I try to write out a block bigger than around 60Mb in one go, I 
>get an error ERROR_NO_SYSTEM_RESOURCES and the whole thing fails. Any 
>ideas? Although I have managed to work around this by writing the file 
>out in chunks. Have tried it first by setting all the chunks off at the 
>same time and also by setting the next chunk off in the callback 
>completion function. Both methods have no effect on the 
>ERROR_INSUFFICIENT_BUFFER error or on which chunk triggers this error.
>
>Any suggestions?
>
>Arthur.
Joseph M. Newcomer [MVP]
email: newcomer@flounder.com
Web: http://www.flounder.com
MVP Tips: http://www.flounder.com/mvp_tips.htm
0
newcomer (15974)
8/23/2005 11:36:47 PM
FILE_FLAG_NO_BUFFERING has specific alignment requirements, which may or may 
not be actually required for different disk controllers. In any case, 
they're no more strict than a page boundary. To be safe, I'd follow them. 
operator new doesn't give you such alignment.

"Joseph M. Newcomer" <newcomer@flounder.com> wrote in message 
news:b8cng1555kfan8oo2gtb6q52h8763agbf7@4ax.com...
>
> It is not clear why VirtualAlloc is required to deal with the no buffering 
> flag; new
> BYTE[n] will work just as well; just make sure you allocate enough space 
> and then start at
> the next sector-sized boundary within the allocate space.
>
> int nsectors = ... number of sectors
>
> LPBYTE b = new BYTE[(nsectors + 1) * SECTOR_SIZE];
>
> LPBYTE buffer = (b + SECTOR_SIZE) & ~(SECTOR_SIZE - 1);
>
> joe


0
alegr (1130)
8/24/2005 5:53:30 AM
Joseph M. Newcomer wrote:
 > If the function returns a success code, then the meaning of 
GetLastError is irrelevant.
 > GetLastError is meaningful ONLY if you get an error return. 
Otherwise, it is a random
 > number.
 >

That was my initially thought, however on reading the docs for 
WriteFileEx, it says the following:-

Remarks
When using WriteFileEx you should check GetLastError even when the 
function returns "success" to check for conditions that are "successes" 
but have some outcome you might want to know about. For example, a 
buffer overflow when calling WriteFileEx will return TRUE, but 
GetLastError will report the overflow with ERROR_MORE_DATA. If the 
function call is successful and there are no warning conditions, 
GetLastError will return ERROR_SUCCESS.

And so I was unsure if the ERROR_INSUFFICIENT_BUFFER, was one of these 
outcomes that you should be aware of.

 > ERROR_NO_SYSTEM_RESOURCES means that you can't do that.  60MB 
requires that the system
 > lock down 60MB of pages while the I/O is in progress.  If you can't 
allocate 60MB of
 > contiguous kernel space (to hold an alias map) and/or cannot find 
60MB of physical pages
 > to lock down, it can't be done.  So it means you can't write 60MB, 
and the workaround is
 > to break your transfer into smaller pieces.
 >

So I can then assume that this 'limit' won't be constant and will vary 
from machine to machine and should be handled accordingly in code. Or is 
there a safe amount I could use to split the file into, which I can 
guarantee will work on all/most machines?

 > It is not clear why VirtualAlloc is required to deal with the no 
buffering flag; new
 > BYTE[n] will work just as well; just make sure you allocate enough 
space and then start at
 > the next sector-sized boundary within the allocate space.
 >
 > int nsectors = ... number of sectors
 >
 > LPBYTE b = new BYTE[(nsectors + 1) * SECTOR_SIZE];
 >
 > LPBYTE buffer = (b + SECTOR_SIZE) & ~(SECTOR_SIZE - 1);
 >

I'll give this a go. As VirtualAlloc zero's everything you allocate, 
with seemingly no way to prevent it. Zero'ing huge chunks of memory 
surely can't be very quick.

Thanks,

Arthur.

 > 				joe
 >
 > On Tue, 23 Aug 2005 11:00:50 GMT, Arthur <arthur@fubaby.com> wrote:
 >
 >
 >>I'm using the WriteFileEx function to write a file asynchronously.
 >>However I sometimes get the error code ERROR_INSUFFICIENT_BUFFER (with
 >>GetLastError), although the function still returns a success and the
 >>data gets written ok. However I don't think it is actually doing an
 >>asynchronous write.
 >>
 >>I open the file with a call to CreateFile, with the flags:
 >>FILE_ATTRIBUTE_NORMAL | FILE_FLAG_OVERLAPPED | FILE_FLAG_NO_BUFFERING
 >>
 >>To satisfy the No buffering, I use VirtualAlloc and make sure
 >>source/dest and size values are all aligned to sector sizes. That bit
 >>seems fine.
 >>
 >>Also, if I try to write out a block bigger than around 60Mb in one go, I
 >>get an error ERROR_NO_SYSTEM_RESOURCES and the whole thing fails. Any
 >>ideas? Although I have managed to work around this by writing the file
 >>out in chunks. Have tried it first by setting all the chunks off at the
 >>same time and also by setting the next chunk off in the callback
 >>completion function. Both methods have no effect on the
 >>ERROR_INSUFFICIENT_BUFFER error or on which chunk triggers this error.
 >>
 >>Any suggestions?
 >>
 >>Arthur.
 >
 > Joseph M. Newcomer [MVP]
 > email: newcomer@flounder.com
 > Web: http://www.flounder.com
 > MVP Tips: http://www.flounder.com/mvp_tips.htm
0
arthur4871 (13)
8/24/2005 9:11:45 AM
Reply:

Similar Artilces:

WriteFile
Hi , I am using WriteFile function for writting some data to harddisk. Is there any limit in size of data it can write. Basically this function is writting imcomplete data. I checked the contents of file. ( Writs encoded data to file then decode the data using Winzip ( MIME data ) no of Bytes are at boundries like : fff, 9fff, ffff ... Any idea why this is hapening ? I'm not sure if there is a limit, or how to find out. Are you checking the parameter that tells the number of bytes written to see if it equals the number of bytes to write? Perhaps you can just keep calling WriteFile...

WriteFile and ERROR_INSUFFICIENT_BUFFER
I'm using the WriteFileEx function to write a file asynchronously. However I sometimes get the error code ERROR_INSUFFICIENT_BUFFER (with GetLastError), although the function still returns a success and the data gets written ok. However I don't think it is actually doing an asynchronous write. I open the file with a call to CreateFile, with the flags: FILE_ATTRIBUTE_NORMAL | FILE_FLAG_OVERLAPPED | FILE_FLAG_NO_BUFFERING To satisfy the No buffering, I use VirtualAlloc and make sure source/dest and size values are all aligned to sector sizes. That bit seems fine. Also, if I try...

WriteFile need help,
Can some one explaint to me why is my output file print out garbage string, event I print BOM at the begining of the file? Did I miss something? output.txt ⁥Tue Feb 07 15:22:02 2006⁢ Testing log file 1. ///////////MY CODE//////////////////// WORD BOM=0xfeff; DWORD cBytes; hFile = CreateFile(szDirectory, GENERIC_WRITE, FILE_SHARE_WRITE, NULL, CREATE_NEW, FILE_ATTRIBUTE_NORMAL, NULL); if (NULL != hFile && INVALID_HANDLE_VALUE != hFile) { SetFilePointer(hFile, 0, NULL, FILE_BEGIN); WriteFile(hFile, &BOM, sizeof(BOM), &cBytes, NULL); } void WriteToFile(wchar_t* p_...

Is there a solution for writefile fails in vista
I was trying to write from starting sector of the harddisk through writefile function, In vista WriteFile() function fails(Access denied) due to security methods in vista and server 2008. Is there a solution for this ??? There are better Vista programmers here than me, but I think you either need to: Elevate your program to a status that allows this function Have your user run as an administrator account to do this kind of advanced functionality Disable UAC Some things to try anyway. Tom "StudioX" <StudioX@discussions.microsoft.com> wrote in message news:3837A5AB-2FF...

ReadFile & WriteFile Problem?
I need to read/write on raw disk drive and it works. After many trials, I found a little bit strange thing. The API ReadFile and WriteFlle read/write datas only from sector size boundaries. I don't know why it should do like that. Following is my code. void readBytes ( void *readBuf, long bytesToMove, long bytesToRead) { int ret = SetFilePointer (diskHandle, bytesToMove, NULL, FILE_BEGIN); unsigned long bytesRead; BOOL result = ReadFile ( diskHandle, readBuf, bytesToRead, &bytesRead , NULL); printf("read %d bytes!!\n", bytesRead); } When char readBuf[512]; long by...

WriteFile(), LPT, and spooler
I can open an LPT port with CreateFile() and then write data to that port with WriteFile(). However, one of my test ports has a spooler attached to it so sometimes when I write data to it the spooler holds the data and doesn't squirt the data out of the port. So this brings up 2 questions: 1) When I open a parallel port with CreateFile() is there a reliable way to determine if a spooler is attached to the LPT? BTW, I tried using "\\\\.\\NONSPOOLED_LPT1" as the port name and GetLastError() gave me an error code 2 ("The system cannot find the file specified."). 2) ...

serial WriteFile hangs
I need to get an old Win98 program running on WinXP. The program works on Win98, but hangs in WriteFile to COM port on Windows-XP. The Visual-C++ program remote controls an embedded board through serial port. Any ideas? _Doug wrote: > I need to get an old Win98 program running on WinXP. > > The program works on Win98, but hangs in WriteFile to COM port on Windows-XP. > > The Visual-C++ program remote controls an embedded board through serial port. > > Any ideas? There is something wrong with the program. Done properly, the same code should work on Win98 and XP. ...

WriteFile function hangs!
Hi, I am printing something through the LPT using the API WriteFile. This is working fine when the printer is connected to the system.But if the printer is not connected then it hangs!! When debugged it found that the program hangs in WriteFile function. Its not coming out from that function!! How to solve it? Pls help... rgds, RENAK. It may just be trying to write all of the characters and waiting for each to time out since the printer is not there. You may want to check for access to the printer perhaps by trying to write a single innocuous control character to the printer first (ba...

printing to parallel port with writefile
I was wondering if someone might be able to tell me how to use writefile to send a string of text to a parallel printer. The text is special IPL printer codes for a barcode printer so it has to be sent directly to a printer. I have used the system command() though with mixed results in Visual C++ and windows. I am used to programming in DOS where all I had to do was send a file to a printer using the copy command like: "copy 227-029.fmt lpt1" or "copy 227-029.fmt com1" (after setting up the com port with another system command. Even so I would like to know how to...

CString UNICODE WriteFile problem...
Hi all, I am trying to send command to a manchine via serial port. I have created a serial port class called as CSerPort,and it has a function SendString() as below. My problem is here that when i try to send a string ,such as "AT+X", via this function,the machine does not respond. If i write the command directly to the WriteFile function as a parameter; .... .. if( (WriteFile(hCOMx,"AT+X",sCommand.GetLength(),&wWrittenBytes, NULL)) == NULL) ... ... then it responds properly. I have an ASCII version of this class and it works very well. I tried to assign the sComma...

WriteFile, insert data in the middle of the file.
Hello, What I want to do is to insert data (not overwrite existing data) in the middle of a big file. Does WriteFile( ) have this option? If there's no way to insert data in the middle, I guess I have to read out all the data from the insertion point, and append my new data, and write them back to the file? Isn't this too inefficient? Anybody knows how OS handles this kind of "insert in the middle" scenario? Thank you, I appreciate your opinion. wk Well you answered your own question. There is no way to write to the middle of the file without overwriting what's ...

Question on how to monitor spool status with WriteFile
I'm sending a file to the parallel port using CreateFile() and WriteFile(). The whole file is immediately being sent to the spooler however it takes some time for the data to be completely sent to the printer. Is there a way to monitor the status on this spool job from my program so that I can take some action when the job is complete? Thanks Take a look at EnumJobs and GetJob function. Using these two functions you can get the status of the print jobs for a particular printer. AliR. "mao" <NoSpam> wrote in message news:%23s8miKqxEHA.4040@TK2MSFTNGP11.phx.gbl... > ...

how to work writefile function with Microsoft Word
Hi, If we write any DOC file using MS Word i think we need ReadFile CreateFile WriteFile FlushFileBuffers CloseFile above functions.... Can anyone tell me what happned between WriteFile & closefile functions. if i hook WriteFile function & im using it like:::: BOOL WINAPI _WriteFile( HANDLE hFile, // handle to file to write to LPCVOID lpBuffer, // pointer to data to write to file DWORD nNumberOfBytesToWrite, // number of bytes to write LPDWORD lpNumberOfBytesWritten, // pointer to number of bytes...

"ERROR_INVALID_PARAMETER": ReadFile() and WriteFile()
I am writing a program that connects to a serial port and writes data. I follow the guides from MSDN and I use ReadFile() and WriteFile() to do data I/O. The port is opened in non-overlapped I/O mode. But whenever I call ReadFile() or WriteFile(), it returns error number 87, and when I look that up, it is the ERROR_INVALID_PARAMETER. So I made sure that everything was okay. The buffer I pass to those functions is actually allocated and okay, and the variables for how many bytes to read/write, and the lpdword for how bytes were actually read/written are okay as well. And the over...

ReadFile then WriteFile to Console without the line breaks,...
Hi, well, this sound a little strange to ask such a "simple" question, but i would like tho hear someone with a good recommendation on this. I integrated a little "dum file content as hex" into my console application but whever the console output finds a line break it does what a line break is meant to do: goes to teh next line. But how do i i have two columns(10:10): One for the Hex Data and the other for the plain file content. You can imagine what happens, when the console reaches a line break inside a file a prints it out to the std output, which the breaks the line t...

WriteFile access error on UMDF USB driver
I am creating a driver based on the UMDF FX2_driver sample. My device differs in that it has one input and one output interrupt endpoint. Also am providing a symbolic name. I have a test application I would like to use with this driver/device that uses WriteFile to send data to the driver. I am able to verify thru tracing and a line trace that the data from the WriteFile is being received by the driver and passed on to the usb device. However, the WriteFile fails with a GetLastError() of 1117 (ERROR_IO_DEVICE). Need to determine why I am getting this error. ricobandito &l...

Calling Response to WriteFile and also update page problem
Hello, I have a VB.net app the sends an Excel file to the browser and it also need to hide some fields on the form at the same time. But the form is not hiding the controlls. The "Open/Save/Cancel" diaolog opens on the browser but the page stays the same, the controls are not hidden. Below is a sample of the code I'm using to send the file to the browser and hide the controls. What am I doing wrong? Sub SendReport(ByRef SaveFile As String) tblMessage.Visible = False lblMessage.Text = "" 'Response.Flush() 'Response.Clos...

WriteFile doesn't send data to LPT1 until CloseHandle is called
Greetings groups. Please, forgive me for posting twice, didn't really know where this question should really go to, nor where could I get an answer. My program tries to send data directly to the parallel port to a printer, code lies at the end. In short it does this: First opens with CreateFile the device "\\.\\LPT1" as read/write in a non-overlapped mode, next sends a string to the printer on the other side, finally waits for the user to enter any string to close handle and quit. I've noted with the aid of sniffers (http://www.sysinternals.com) that WriteFile doesn't ...