New authentication code added, particularly noticable when doing POST or PUT

                                  Changelog

Daniel (6 April 2004)

- Gisle Vanem's fixed bug #927979 reported by Nathan O'Sullivan. The problem

  made libcurl on Windows leak a small amount of memory in each name resolve

  when not used as a DLL.

- New authentication code added, particularly noticable when doing POST or PUT

  with Digest or NTLM. libcurl will now use HEAD to negotiate the

  authentication and when done perform the requested POST. Previously libcurl

  sent POST immediately and expected the server to reply a final status code

  with an error and then libcurl would not send the request-body but instead

  send then next request in the sequence.

  The reason for this change is due to IIS6 barfing on libcurl when we attempt

  to POST with NTLM authentication. The reason for the problems is found in

  RFC2616 section 8.2.3 regarding how servers should deal with the 100

  continue request-header:

        If it responds with a final status code, it MAY close the transport

        connection or it MAY continue to read and discard the rest of the

        request.

  Previous versions of IIS clearly did close the connection in this case,

  while this newer version decided it should "read and discard". That would've

  forced us to send the whole POST (or PUT) data only to have it discarded and

  then be forced to send it again. To avoid that huge penality, we switch to

  using HEAD until we are authenticated and then send the POST.

  The only actual drawback I can think of (except for the odd sites that might

  treat HEAD differently than they would treat POST/PUT when given the same

  URL) is that if you do POST with CURLAUTH_ANY set and the site requires NO

  authentication, libcurl will still use a HEAD in a first round and then do a

  POST.

  If you do a HEAD or a GET on a site using CURLAUTH_ANY, libcurl will send

  an un-authenticated request at once, which then is the only request if the

  site requires no auth.

  Alan Pinstein helped me work out the protocol details by figuring out why

  libcurl failed and what IIS6 expects.

- The --limit-rate logic was corrected and now it works a lot better for

  higher speeds, such as '10m' or similar. Reported in bug report #930249.

This release includes the following bugfixes:

 o fixed minor memory leak in libcurl for Windows when staticly linked

 o POST/PUT using Digest/NTLM/Negotiate (including anyauth) now work better

 o --limit-rate with high speed rates is a lot more accurate now

 o curl_strnqual.3 "refer-to" man page fix

 o fixed a minor very old progress meter final update bug

 o added checks for a working NI_WITHSCOPEID before that is used

This release would not have looked like this without help, code, reports and

advice from friends like these:

 Thomas Schwinge, Marty Kuhrt, Günter Knauf, Kevin Roth, Glen Nakamura,

 Gisle Vanem, Greg Hewgill, Joe Halpin, Tor Arntsen, Dirk Manske, Roy Shan,

 Mitz Wark, Andrés García, Robin Kay

 Thomas Schwinge, Marty Kuhrt, Günter Knauf, Kevin Roth, Glen Nakamura, Gisle

 Vanem, Greg Hewgill, Joe Halpin, Tor Arntsen, Dirk Manske, Roy Shan, Mitz

 Wark, Andrés García, Robin Kay, Alan Pinstein, David Byron, Nathan O'Sullivan

        Thanks! (and sorry if I forgot to mention someone)

  /* If we have selected NOBODY and HEADER, it means that we only want file

     information. Which for FILE can't be much more than the file size and

     date. */

  if(data->set.no_body && data->set.include_header && fstated) {

  if(conn->bits.no_body && data->set.include_header && fstated) {

    CURLcode result;

    sprintf(buf, "Content-Length: %" FORMAT_OFF_T "\r\n", expected_size);

    result = Curl_client_write(data, CLIENTWRITE_BOTH, buf, 0);

      return result;

  }

  else if(!data->set.no_body) {

  else if(!conn->bits.no_body) {

    /* Retrieve file or directory */

    bool dirlist=FALSE;

    curl_off_t downloadsize=-1;

  /* If we have selected NOBODY and HEADER, it means that we only want file

     information. Which in FTP can't be much more than the file size and

     date. */

  if(data->set.no_body && data->set.include_header && ftp->file) {

  if(conn->bits.no_body && data->set.include_header && ftp->file) {

    /* The SIZE command is _not_ RFC 959 specified, and therefor many servers

       may not support it! It is however the only way we have to get a file's

       size! */

    return CURLE_OK;

  }

  if(data->set.no_body)

  if(conn->bits.no_body)

    /* doesn't really transfer any data */

    ftp->no_transfer = TRUE;

  /* Get us a second connection up and connected */

      conn->newurl = strdup(data->change.url); /* clone URL */

    data->state.authavail = CURLAUTH_NONE; /* clear it here */

  }

  else if(!data->state.authdone && (data->info.httpcode < 400)) {

    /* no (known) authentication available,

       authentication is not "done" yet and

       no authentication seems to be required and

       we didn't try HEAD or GET */

    if((data->set.httpreq != HTTPREQ_GET) &&

       (data->set.httpreq != HTTPREQ_HEAD)) {

      conn->newurl = strdup(data->change.url); /* clone URL */

      data->state.authdone = TRUE;

    }

  }

}

/**

  char *auth=NULL;

  curlassert(data);

  data->state.authdone = FALSE; /* default is no */

  if(!data->state.authstage) {

    if(conn->bits.httpproxy && conn->bits.proxy_user_passwd)

    if(conn->bits.httpproxy && conn->bits.proxy_user_passwd) {

      data->state.authdone = FALSE;

      Curl_http_auth_stage(data, 407);

    else if(conn->bits.user_passwd)

    }

    else if(conn->bits.user_passwd) {

      data->state.authdone = FALSE;

      Curl_http_auth_stage(data, 401);

    }

    else {

      data->state.authdone = TRUE;

      return CURLE_OK; /* no authentication with no user or password */

  const char *te = ""; /* tranfer-encoding */

  char *ptr;

  char *request;

  Curl_HttpReq httpreq = data->set.httpreq;

  if(!conn->proto.http) {

    /* Only allocate this struct if we don't already have it! */

  if ( (conn->protocol&(PROT_HTTP|PROT_FTP)) &&

       data->set.upload) {

    data->set.httpreq = HTTPREQ_PUT;

    httpreq = HTTPREQ_PUT;

  }

  request = data->set.customrequest?

    data->set.customrequest:

    (data->set.no_body?(char *)"HEAD":

     ((HTTPREQ_POST == data->set.httpreq) ||

      (HTTPREQ_POST_FORM == data->set.httpreq))?(char *)"POST":

     (HTTPREQ_PUT == data->set.httpreq)?(char *)"PUT":(char *)"GET");

  /* Now set the 'request' pointer to the proper request string */

  if(data->set.customrequest)

    request = data->set.customrequest;

  else {

    if(conn->bits.no_body)

      request = (char *)"HEAD";

    else {

      curlassert((httpreq > HTTPREQ_NONE) && (httpreq < HTTPREQ_LAST));

      switch(httpreq) {

      case HTTPREQ_POST:

      case HTTPREQ_POST_FORM:

        request = (char *)"POST";

        break;

      case HTTPREQ_PUT:

        request = (char *)"PUT";

        break;

      case HTTPREQ_GET:

        request = (char *)"GET";

        break;

      case HTTPREQ_HEAD:

        request = (char *)"HEAD";

        break;

      default: /* this should never happen */

        break;

      }

    }

  }

  /* The User-Agent string has been built in url.c already, because it might

     have been used in the proxy connect, but if we have got a header with

     the user-agent string specified, we erase the previously made string

  /* The User-Agent string might have been allocated in url.c already, because

     it might have been used in the proxy connect, but if we have got a header

     with the user-agent string specified, we erase the previously made string

     here. */

  if(checkheaders(data, "User-Agent:") && conn->allocptr.uagent) {

    free(conn->allocptr.uagent);

  if(result)

    return result;

  if(!data->state.authdone && (httpreq != HTTPREQ_GET)) {

    /* Until we are authenticated, we switch over to HEAD. Unless its a GET

       we want to do. The explanation for this is rather long and boring, but

       the point is that it can't be done otherwise without risking having to

       send the POST or PUT data multiple times. */

    httpreq = HTTPREQ_HEAD;

    request = (char *)"HEAD";

    conn->bits.no_body = TRUE;

  }

  Curl_safefree(conn->allocptr.ref);

  if(data->change.referer && !checkheaders(data, "Referer:"))

    conn->allocptr.ref = aprintf("Referer: %s\015\012", data->change.referer);

  else

    conn->allocptr.cookie = NULL;

  if(!conn->bits.upload_chunky && (data->set.httpreq != HTTPREQ_GET)) {

  if(!conn->bits.upload_chunky && (httpreq != HTTPREQ_GET)) {

    /* not a chunky transfer yet, but data is to be sent */

    ptr = checkheaders(data, "Transfer-Encoding:");

    if(ptr) {

    /* The path sent to the proxy is in fact the entire URL */

    ppath = data->change.url;

  }

  if(HTTPREQ_POST_FORM == data->set.httpreq) {

  if(HTTPREQ_POST_FORM == httpreq) {

    /* we must build the whole darned post sequence first, so that we have

       a size of the whole shebang before we start to send it */

     result = Curl_getFormData(&http->sendit, data->set.httppost,

  if(!checkheaders(data, "Accept:"))

    http->p_accept = "Accept: */*\r\n";

  if(( (HTTPREQ_POST == data->set.httpreq) ||

       (HTTPREQ_POST_FORM == data->set.httpreq) ||

       (HTTPREQ_PUT == data->set.httpreq) ) &&

  if(( (HTTPREQ_POST == httpreq) ||

       (HTTPREQ_POST_FORM == httpreq) ||

       (HTTPREQ_PUT == httpreq) ) &&

     conn->resume_from) {

    /**********************************************************************

     * Resuming upload in HTTP means that we PUT or POST and that we have

     * or uploading and we always let customized headers override our internal

     * ones if any such are specified.

     */

    if((data->set.httpreq == HTTPREQ_GET) &&

    if((httpreq == HTTPREQ_GET) &&

       !checkheaders(data, "Range:")) {

      /* if a line like this was already allocated, free the previous one */

      if(conn->allocptr.rangeline)

        free(conn->allocptr.rangeline);

      conn->allocptr.rangeline = aprintf("Range: bytes=%s\r\n", conn->range);

    }

    else if((data->set.httpreq != HTTPREQ_GET) &&

    else if((httpreq != HTTPREQ_GET) &&

            !checkheaders(data, "Content-Range:")) {

      if(conn->resume_from) {

    http->postdata = NULL;  /* nothing to post at this point */

    Curl_pgrsSetUploadSize(data, 0); /* upload size is 0 atm */

    /* If 'authdone' is still FALSE, we must not set the write socket index to

       the Curl_transfer() call below, as we're not ready to actually upload

       any data yet. */

    /* If 'authdone' is FALSE, we must not set the write socket index to the

       Curl_transfer() call below, as we're not ready to actually upload any

       data yet. */

    switch(data->set.httpreq) {

    switch(httpreq) {

    case HTTPREQ_POST_FORM:

      if(Curl_FormInit(&http->form, http->sendit)) {