Puzzled by HPPT post Request Headers not arriving

Problem with HTTP Request Headers

I have an app that uses an HTTP Post to send some stuff to a server and get 
some data back.  It works OK but I would like to save bandwidth by 
compressing the return data.  My problem is that the header that I want to 
send, to allow compression on the server, is not being received by the 
server.   I have patched the serverscript to  reflect back the headers and 
neither of the header strings seems to have got there.  The post string 
itself gets there, but the 'Accept-Encoding' header does not.  Please, what 
am I doing wrong?  The code that tries to do what I want follows the 
signature.

TIA
Bev

CString rqheaders = "Content-Type: 
application/x-www-form-urlencoded\r\nAccept-Encoding: gzip, deflate\r\n";
CInternetSession 
Ins(NULL,1,INTERNET_OPEN_TYPE_PRECONFIG,NULL,NULL,INTERNET_FLAG_DONT_CACHE);
CHttpConnection *Htc = 
Ins.GetHttpConnection(myserver,(INTERNET_PORT)80,NULL,NULL);
CHttpFile *Htf = 
Htc->OpenRequest("POST",myscript,NULL,1,NULL,NULL,INTERNET_FLAG_EXISTING_CONNECT);
Htf->AddRequestHeaders(rqheaders,HTTP_ADDREQ_FLAG_ADD);
Htf->SendRequest(NULL,0,poststring,strlen(poststring));
CString aContType;
Htf->QueryInfo(HTTP_QUERY_CONTENT_TYPE,aContType); // always reports text
CString aContEncode;
Htf->QueryInfo(HTTP_QUERY_CONTENT_TRANSFER_ENCODING,aContEncode); // always 
returns an empty string
// here we should read a whole buffer and then parse it
char buf[200000];
buf[0] = 0;
char bbb[2048];
int k = 0;
while ((k = Htf->Read(bbb,2047)) > 0)
{
	bbb[k] = 0;
	if ((strlen(buf)+strlen(bbb)) < 199000)
		strcat(buf,bbb);
}

0
9/5/2005 10:50:02 PM
vc.mfc 33608 articles. 0 followers. Follow

9 Replies
1557 Views

Similar Articles

[PageSpeed] 17

higher things aside:

do you need a double / in your Content-Type string?, ie. "blah blah //x-www
blah blah."




"Bev Ewen-Smith" <BevEwenSmith@discussions.microsoft.com> wrote in message
news:78AE657E-FFB1-491B-BC4E-B2B17F1DCAA5@microsoft.com...
> Problem with HTTP Request Headers
>
> I have an app that uses an HTTP Post to send some stuff to a server and
get
> some data back.  It works OK but I would like to save bandwidth by
> compressing the return data.  My problem is that the header that I want to
> send, to allow compression on the server, is not being received by the
> server.   I have patched the serverscript to  reflect back the headers and
> neither of the header strings seems to have got there.  The post string
> itself gets there, but the 'Accept-Encoding' header does not.  Please,
what
> am I doing wrong?  The code that tries to do what I want follows the
> signature.
>
> TIA
> Bev
>
> CString rqheaders = "Content-Type:
> application/x-www-form-urlencoded\r\nAccept-Encoding: gzip, deflate\r\n";
> CInternetSession
>
Ins(NULL,1,INTERNET_OPEN_TYPE_PRECONFIG,NULL,NULL,INTERNET_FLAG_DONT_CACHE);
> CHttpConnection *Htc =
> Ins.GetHttpConnection(myserver,(INTERNET_PORT)80,NULL,NULL);
> CHttpFile *Htf =
>
Htc->OpenRequest("POST",myscript,NULL,1,NULL,NULL,INTERNET_FLAG_EXISTING_CON
NECT);
> Htf->AddRequestHeaders(rqheaders,HTTP_ADDREQ_FLAG_ADD);
> Htf->SendRequest(NULL,0,poststring,strlen(poststring));
> CString aContType;
> Htf->QueryInfo(HTTP_QUERY_CONTENT_TYPE,aContType); // always reports text
> CString aContEncode;
> Htf->QueryInfo(HTTP_QUERY_CONTENT_TRANSFER_ENCODING,aContEncode); //
always
> returns an empty string
> // here we should read a whole buffer and then parse it
> char buf[200000];
> buf[0] = 0;
> char bbb[2048];
> int k = 0;
> while ((k = Htf->Read(bbb,2047)) > 0)
> {
> bbb[k] = 0;
> if ((strlen(buf)+strlen(bbb)) < 199000)
> strcat(buf,bbb);
> }
>


0
9/5/2005 11:11:41 PM
"Bev Ewen-Smith" <BevEwenSmith@discussions.microsoft.com> wrote in message
news:78AE657E-FFB1-491B-BC4E-B2B17F1DCAA5@microsoft.com...
> Problem with HTTP Request Headers
>
> I have an app that uses an HTTP Post to send some stuff to a server and
get
> some data back.  It works OK but I would like to save bandwidth by
> compressing the return data.  My problem is that the header that I want to
> send, to allow compression on the server, is not being received by the
> server.   I have patched the serverscript to  reflect back the headers and
> neither of the header strings seems to have got there.  The post string
> itself gets there, but the 'Accept-Encoding' header does not.  Please,
what
> am I doing wrong?  The code that tries to do what I want follows the
> signature.
>
> TIA
> Bev
>
> CString rqheaders = "Content-Type:
> application/x-www-form-urlencoded\r\nAccept-Encoding: gzip, deflate\r\n";
> CInternetSession
>
Ins(NULL,1,INTERNET_OPEN_TYPE_PRECONFIG,NULL,NULL,INTERNET_FLAG_DONT_CACHE);
> CHttpConnection *Htc =
> Ins.GetHttpConnection(myserver,(INTERNET_PORT)80,NULL,NULL);
> CHttpFile *Htf =
>
Htc->OpenRequest("POST",myscript,NULL,1,NULL,NULL,INTERNET_FLAG_EXISTING_CON
NECT);
> Htf->AddRequestHeaders(rqheaders,HTTP_ADDREQ_FLAG_ADD);

What is the return value of AddRequestHeaders?  It should be nonzero if
successful; otherwise 0.

Note that the "Accept-Encoding" header is only valid in HTTP/1.1, whereas
your OpenRequest call specified HTTP/1.0.  Maybe that's part of your
problem.  Try calling OpenRequest with "HTTP/1.1" as the value for the
pstrVersion parameter.



> Htf->SendRequest(NULL,0,poststring,strlen(poststring));
> CString aContType;
> Htf->QueryInfo(HTTP_QUERY_CONTENT_TYPE,aContType); // always reports text
> CString aContEncode;
> Htf->QueryInfo(HTTP_QUERY_CONTENT_TRANSFER_ENCODING,aContEncode); //
always
> returns an empty string
> // here we should read a whole buffer and then parse it
> char buf[200000];
> buf[0] = 0;
> char bbb[2048];
> int k = 0;
> while ((k = Htf->Read(bbb,2047)) > 0)
> {
> bbb[k] = 0;
> if ((strlen(buf)+strlen(bbb)) < 199000)

Since you are trying to get compressed data, it's probably wrong to test for
strlen(buf) or strlen(bbb).  Compressed data very likely includes NULL bytes
as valid data, and all bytes after the first NULL byte will be lost if you
use strlen.  Don't use any of the strxxx functions (such as the strcat
function in the next line).  Use raw buffer manipulation functions like
memcpy, for example:

int cbRecd = 0;
int cbTotal = 0;
while ((cbRecd = Htf->Read(bbb,2047)) > 0)
{
  if ( cbRecd+cbTotal < 199000)
  {
    memcpy( buf+cbTotal, bbb, cbRecd );
    cbTotal += cdRecd;
  }
}



> strcat(buf,bbb);
> }
>


0
9/5/2005 11:44:37 PM
Hi Michael

The return values are both 1 (TRUE) and putting "HTTP/1.1" does not change 
the outcome.

I tried putting likely Mime strings in the pstrAcceptTypes field of 
OpenRequest but the server responded "Bad Request" adding "Request Header 
field is missing colon separator" which is not the case in the strings I 
tried.

Can it really be this elusive?  Somebody must have done this successfully!

Bev


"Michael K. O'Neill" wrote:

> "Bev Ewen-Smith" <BevEwenSmith@discussions.microsoft.com> wrote in message
> news:78AE657E-FFB1-491B-BC4E-B2B17F1DCAA5@microsoft.com...
> > Problem with HTTP Request Headers
> >
> > I have an app that uses an HTTP Post to send some stuff to a server and
> get
> > some data back.  It works OK but I would like to save bandwidth by
> > compressing the return data.  My problem is that the header that I want to
> > send, to allow compression on the server, is not being received by the
> > server.   I have patched the serverscript to  reflect back the headers and
> > neither of the header strings seems to have got there.  The post string
> > itself gets there, but the 'Accept-Encoding' header does not.  Please,
> what
> > am I doing wrong?  The code that tries to do what I want follows the
> > signature.
> >
> > TIA
> > Bev
> >
> > CString rqheaders = "Content-Type:
> > application/x-www-form-urlencoded\r\nAccept-Encoding: gzip, deflate\r\n";
> > CInternetSession
> >
> Ins(NULL,1,INTERNET_OPEN_TYPE_PRECONFIG,NULL,NULL,INTERNET_FLAG_DONT_CACHE);
> > CHttpConnection *Htc =
> > Ins.GetHttpConnection(myserver,(INTERNET_PORT)80,NULL,NULL);
> > CHttpFile *Htf =
> >
> Htc->OpenRequest("POST",myscript,NULL,1,NULL,NULL,INTERNET_FLAG_EXISTING_CON
> NECT);
> > Htf->AddRequestHeaders(rqheaders,HTTP_ADDREQ_FLAG_ADD);
> 
> What is the return value of AddRequestHeaders?  It should be nonzero if
> successful; otherwise 0.
> 
> Note that the "Accept-Encoding" header is only valid in HTTP/1.1, whereas
> your OpenRequest call specified HTTP/1.0.  Maybe that's part of your
> problem.  Try calling OpenRequest with "HTTP/1.1" as the value for the
> pstrVersion parameter.
> 
> 
> 
> > Htf->SendRequest(NULL,0,poststring,strlen(poststring));
> > CString aContType;
> > Htf->QueryInfo(HTTP_QUERY_CONTENT_TYPE,aContType); // always reports text
> > CString aContEncode;
> > Htf->QueryInfo(HTTP_QUERY_CONTENT_TRANSFER_ENCODING,aContEncode); //
> always
> > returns an empty string
> > // here we should read a whole buffer and then parse it
> > char buf[200000];
> > buf[0] = 0;
> > char bbb[2048];
> > int k = 0;
> > while ((k = Htf->Read(bbb,2047)) > 0)
> > {
> > bbb[k] = 0;
> > if ((strlen(buf)+strlen(bbb)) < 199000)
> 
> Since you are trying to get compressed data, it's probably wrong to test for
> strlen(buf) or strlen(bbb).  Compressed data very likely includes NULL bytes
> as valid data, and all bytes after the first NULL byte will be lost if you
> use strlen.  Don't use any of the strxxx functions (such as the strcat
> function in the next line).  Use raw buffer manipulation functions like
> memcpy, for example:
> 
> int cbRecd = 0;
> int cbTotal = 0;
> while ((cbRecd = Htf->Read(bbb,2047)) > 0)
> {
>   if ( cbRecd+cbTotal < 199000)
>   {
>     memcpy( buf+cbTotal, bbb, cbRecd );
>     cbTotal += cdRecd;
>   }
> }
> 
> 
> 
> > strcat(buf,bbb);
> > }
> >
> 
> 
> 
0
9/6/2005 2:21:03 PM
Hi Joe,

Thanks for the suggestion.  A single forward slash appears correct in the 
CString.  With a backslash it would certainly require two.

I tried but it made no difference.

Regards
Bev
COAA

"Joe Butler" wrote:

> higher things aside:
> 
> do you need a double / in your Content-Type string?, ie. "blah blah //x-www
> blah blah."
> 
> 
> 
> 
> "Bev Ewen-Smith" <BevEwenSmith@discussions.microsoft.com> wrote in message
> news:78AE657E-FFB1-491B-BC4E-B2B17F1DCAA5@microsoft.com...
> > Problem with HTTP Request Headers
> >
> > I have an app that uses an HTTP Post to send some stuff to a server and
> get
> > some data back.  It works OK but I would like to save bandwidth by
> > compressing the return data.  My problem is that the header that I want to
> > send, to allow compression on the server, is not being received by the
> > server.   I have patched the serverscript to  reflect back the headers and
> > neither of the header strings seems to have got there.  The post string
> > itself gets there, but the 'Accept-Encoding' header does not.  Please,
> what
> > am I doing wrong?  The code that tries to do what I want follows the
> > signature.
> >
> > TIA
> > Bev
> >
> > CString rqheaders = "Content-Type:
> > application/x-www-form-urlencoded\r\nAccept-Encoding: gzip, deflate\r\n";
> > CInternetSession
> >
> Ins(NULL,1,INTERNET_OPEN_TYPE_PRECONFIG,NULL,NULL,INTERNET_FLAG_DONT_CACHE);
> > CHttpConnection *Htc =
> > Ins.GetHttpConnection(myserver,(INTERNET_PORT)80,NULL,NULL);
> > CHttpFile *Htf =
> >
> Htc->OpenRequest("POST",myscript,NULL,1,NULL,NULL,INTERNET_FLAG_EXISTING_CON
> NECT);
> > Htf->AddRequestHeaders(rqheaders,HTTP_ADDREQ_FLAG_ADD);
> > Htf->SendRequest(NULL,0,poststring,strlen(poststring));
> > CString aContType;
> > Htf->QueryInfo(HTTP_QUERY_CONTENT_TYPE,aContType); // always reports text
> > CString aContEncode;
> > Htf->QueryInfo(HTTP_QUERY_CONTENT_TRANSFER_ENCODING,aContEncode); //
> always
> > returns an empty string
> > // here we should read a whole buffer and then parse it
> > char buf[200000];
> > buf[0] = 0;
> > char bbb[2048];
> > int k = 0;
> > while ((k = Htf->Read(bbb,2047)) > 0)
> > {
> > bbb[k] = 0;
> > if ((strlen(buf)+strlen(bbb)) < 199000)
> > strcat(buf,bbb);
> > }
> >
> 
> 
> 
0
9/6/2005 4:25:02 PM
I think the difference is in the URL designation.  In that case you always 
use double forward slash like http://www...  you use double backslash in 
CString because the compiler will consider the backslash an escape character 
and \\ is the same as one \ ...

Tom

"Bev Ewen-Smith" <BevEwenSmith@discussions.microsoft.com> wrote in message 
news:ED0B80D3-64B3-4E59-A2B9-777A7367198C@microsoft.com...
> Hi Joe,
>
> Thanks for the suggestion.  A single forward slash appears correct in the
> CString.  With a backslash it would certainly require two.
>
> I tried but it made no difference.


0
tserface (3861)
9/6/2005 7:53:58 PM
"Bev Ewen-Smith" <BevEwenSmith@discussions.microsoft.com> wrote in message
news:628F3E8A-B7D0-410A-A29A-99396E72FCEC@microsoft.com...
> Hi Michael
>
> The return values are both 1 (TRUE) and putting "HTTP/1.1" does not change
> the outcome.
>
> I tried putting likely Mime strings in the pstrAcceptTypes field of
> OpenRequest but the server responded "Bad Request" adding "Request Header
> field is missing colon separator" which is not the case in the strings I
> tried.
>
> Can it really be this elusive?  Somebody must have done this successfully!
>
> Bev
>
>

Please show us the exact headings that the server is getting.

Mike


0
9/6/2005 9:57:13 PM
It was my mistake - I got my slash directions mixed up.


"Bev Ewen-Smith" <BevEwenSmith@discussions.microsoft.com> wrote in message
news:ED0B80D3-64B3-4E59-A2B9-777A7367198C@microsoft.com...
> Hi Joe,
>
> Thanks for the suggestion.  A single forward slash appears correct in the
> CString.  With a backslash it would certainly require two.
>
> I tried but it made no difference.
>
> Regards
> Bev
> COAA
>
> "Joe Butler" wrote:
>
> > higher things aside:
> >
> > do you need a double / in your Content-Type string?, ie. "blah blah
//x-www
> > blah blah."
> >
> >
> >
> >
> > "Bev Ewen-Smith" <BevEwenSmith@discussions.microsoft.com> wrote in
message
> > news:78AE657E-FFB1-491B-BC4E-B2B17F1DCAA5@microsoft.com...
> > > Problem with HTTP Request Headers
> > >
> > > I have an app that uses an HTTP Post to send some stuff to a server
and
> > get
> > > some data back.  It works OK but I would like to save bandwidth by
> > > compressing the return data.  My problem is that the header that I
want to
> > > send, to allow compression on the server, is not being received by the
> > > server.   I have patched the serverscript to  reflect back the headers
and
> > > neither of the header strings seems to have got there.  The post
string
> > > itself gets there, but the 'Accept-Encoding' header does not.  Please,
> > what
> > > am I doing wrong?  The code that tries to do what I want follows the
> > > signature.
> > >
> > > TIA
> > > Bev
> > >
> > > CString rqheaders = "Content-Type:
> > > application/x-www-form-urlencoded\r\nAccept-Encoding: gzip,
deflate\r\n";
> > > CInternetSession
> > >
> >
Ins(NULL,1,INTERNET_OPEN_TYPE_PRECONFIG,NULL,NULL,INTERNET_FLAG_DONT_CACHE);
> > > CHttpConnection *Htc =
> > > Ins.GetHttpConnection(myserver,(INTERNET_PORT)80,NULL,NULL);
> > > CHttpFile *Htf =
> > >
> >
Htc->OpenRequest("POST",myscript,NULL,1,NULL,NULL,INTERNET_FLAG_EXISTING_CON
> > NECT);
> > > Htf->AddRequestHeaders(rqheaders,HTTP_ADDREQ_FLAG_ADD);
> > > Htf->SendRequest(NULL,0,poststring,strlen(poststring));
> > > CString aContType;
> > > Htf->QueryInfo(HTTP_QUERY_CONTENT_TYPE,aContType); // always reports
text
> > > CString aContEncode;
> > > Htf->QueryInfo(HTTP_QUERY_CONTENT_TRANSFER_ENCODING,aContEncode); //
> > always
> > > returns an empty string
> > > // here we should read a whole buffer and then parse it
> > > char buf[200000];
> > > buf[0] = 0;
> > > char bbb[2048];
> > > int k = 0;
> > > while ((k = Htf->Read(bbb,2047)) > 0)
> > > {
> > > bbb[k] = 0;
> > > if ((strlen(buf)+strlen(bbb)) < 199000)
> > > strcat(buf,bbb);
> > > }
> > >
> >
> >
> >


0
9/7/2005 1:01:57 AM
Thanks Michael,

I found that I could change any request header with the intuitive syntax but 
not the Accept-Encoding.  On a hunch I disabled the firewall and then the 
Accept-Encoding header reached the server as intended.  It was a firewall 
problem!

Now I have to find out how to stop Nortons Personal Firewall from blocking 
just that request header.

Regards and thanks
Bev
COAA

"Michael K. O'Neill" wrote:

> 
> "Bev Ewen-Smith" <BevEwenSmith@discussions.microsoft.com> wrote in message
> news:628F3E8A-B7D0-410A-A29A-99396E72FCEC@microsoft.com...
> > Hi Michael
> >
> > The return values are both 1 (TRUE) and putting "HTTP/1.1" does not change
> > the outcome.
> >
> > I tried putting likely Mime strings in the pstrAcceptTypes field of
> > OpenRequest but the server responded "Bad Request" adding "Request Header
> > field is missing colon separator" which is not the case in the strings I
> > tried.
> >
> > Can it really be this elusive?  Somebody must have done this successfully!
> >
> > Bev
> >
> >
> 
> Please show us the exact headings that the server is getting.
> 
> Mike
> 
> 
> 
0
9/8/2005 4:35:04 PM
"Bev Ewen-Smith" <BevEwenSmith@discussions.microsoft.com> wrote in message
news:E0ECA6DC-50ED-47B4-AF84-66DD15A4E7C1@microsoft.com...
> Thanks Michael,
>
> I found that I could change any request header with the intuitive syntax
but
> not the Accept-Encoding.  On a hunch I disabled the firewall and then the
> Accept-Encoding header reached the server as intended.  It was a firewall
> problem!
>
> Now I have to find out how to stop Nortons Personal Firewall from blocking
> just that request header.
>
> Regards and thanks
> Bev
> COAA
>

Extraordinary!!  See this link (and the link inside it which purportedly
contains an e-mail from Symantec), which confirms the behavior you're seeing
on Norton Personal Firewall:
http://www.newestindustry.org/index.php/2005/03/10/symantec_firewall_problems_with_accept_e

Mike


0
9/8/2005 11:28:10 PM
Reply:

Similar Artilces: