Commit 6b761669 authored by Richy Kim's avatar Richy Kim Committed by Daniel Stenberg
Browse files

CURLOPT_BUFFERSIZE: support enlarging receive buffer 

Replace use of fixed macro BUFSIZE to define the size of the receive
buffer.  Reappropriate CURLOPT_BUFFERSIZE to include enlarging receive
buffer size.  Upon setting, resize buffer if larger than the current
default size up to a MAX_BUFSIZE (512KB). This can benefit protocols
like SFTP.

Closes #1222
parent 81cb255c

docs/libcurl/curl_easy_setopt.3

+1 −1
Original line number Diff line number Diff line
@@ -190,7 +190,7 @@ Timeout for DNS cache. See \fICURLOPT_DNS_CACHE_TIMEOUT(3)\fP 
.IP CURLOPT_DNS_USE_GLOBAL_CACHE

OBSOLETE Enable global DNS cache. See \fICURLOPT_DNS_USE_GLOBAL_CACHE(3)\fP

.IP CURLOPT_BUFFERSIZE

Ask for smaller buffer size. See \fICURLOPT_BUFFERSIZE(3)\fP

Ask for alternate buffer size. See \fICURLOPT_BUFFERSIZE(3)\fP

.IP CURLOPT_PORT

Port number to connect to. See \fICURLOPT_PORT(3)\fP

.IP CURLOPT_TCP_FASTOPEN

docs/libcurl/opts/CURLOPT_BUFFERSIZE.3

+10 −8
Original line number Diff line number Diff line
@@ -5,7 +5,7 @@ 
.\" *                            | (__| |_| |  _ <| |___

.\" *                             \___|\___/|_| \_\_____|

.\" *

.\" * Copyright (C) 1998 - 2016, Daniel Stenberg, <daniel@haxx.se>, et al.

.\" * Copyright (C) 1998 - 2017, Daniel Stenberg, <daniel@haxx.se>, et al.

.\" *

.\" * This software is licensed as described in the file COPYING, which

.\" * you should have received as part of this distribution. The terms
@@ -30,20 +30,22 @@ CURLcode curl_easy_setopt(CURL *handle, CURLOPT_BUFFERSIZE, long size); 
.SH DESCRIPTION

Pass a long specifying your preferred \fIsize\fP (in bytes) for the receive

buffer in libcurl.  The main point of this would be that the write callback

gets called more often and with smaller chunks. This is just treated as a

request, not an order. You cannot be guaranteed to actually get the given

size.

gets called more often and with smaller chunks. Secondly, for some protocols,

there's a benefit of having a larger buffer for performance.



This size is by default set as big as possible (\fICURL_MAX_WRITE_SIZE\fP), so

it only makes sense to use this option if you want it smaller.

This is just treated as a request, not an order. You cannot be guaranteed to

actually get the given size.



This buffer size is by default \fICURL_MAX_WRITE_SIZE\fP (16kB). The maximum

buffer size allowed to set is \fICURL_MAX_READ_SIZE\fP (512kB).

.SH DEFAULT

CURL_MAX_WRITE_SIZE

CURL_MAX_WRITE_SIZE (16kB)

.SH PROTOCOLS

All

.SH EXAMPLE

TODO

.SH AVAILABILITY

Added in 7.10

Added in 7.10.  Growing the buffer was added in 7.53.0.

.SH RETURN VALUE

Returns CURLE_OK if the option is supported, and CURLE_UNKNOWN_OPTION if not.

.SH "SEE ALSO"

docs/libcurl/symbols-in-versions

+1 −0
Original line number Diff line number Diff line
@@ -748,6 +748,7 @@ CURL_LOCK_TYPE_DNS 7.10 - 7.10.2 
CURL_LOCK_TYPE_NONE             7.10          -           7.10.2

CURL_LOCK_TYPE_SSL_SESSION      7.10          -           7.10.2

CURL_MAX_HTTP_HEADER            7.19.7

CURL_MAX_READ_SIZE              7.53.0

CURL_MAX_WRITE_SIZE             7.9.7

CURL_NETRC_IGNORED              7.9.8

CURL_NETRC_OPTIONAL             7.9.8

include/curl/curl.h

+5 −0
Original line number Diff line number Diff line
@@ -193,6 +193,11 @@ typedef int (*curl_xferinfo_callback)(void *clientp, 
                                      curl_off_t ultotal,

                                      curl_off_t ulnow);



#ifndef CURL_MAX_READ_SIZE

  /* The maximum receive buffer size configurable via CURLOPT_BUFFERSIZE. */

#define CURL_MAX_READ_SIZE 524288

#endif



#ifndef CURL_MAX_WRITE_SIZE

  /* Tests have proven that 20K is a very bad buffer size for uploads on

     Windows, while 16K for some odd reason performed a lot better.

lib/easy.c

+6 −0
Original line number Diff line number Diff line
@@ -870,6 +870,11 @@ struct Curl_easy *curl_easy_duphandle(struct Curl_easy *data) 
   * get setup on-demand in the code, as that would probably decrease

   * the likeliness of us forgetting to init a buffer here in the future.

   */

  outcurl->set.buffer_size = data->set.buffer_size;

  outcurl->state.buffer = malloc(CURL_BUFSIZE(outcurl->set.buffer_size) + 1);

  if(!outcurl->state.buffer)

    goto fail;



  outcurl->state.headerbuff = malloc(HEADERSIZE);

  if(!outcurl->state.headerbuff)

    goto fail;
@@ -940,6 +945,7 @@ struct Curl_easy *curl_easy_duphandle(struct Curl_easy *data) 
  if(outcurl) {

    curl_slist_free_all(outcurl->change.cookielist);

    outcurl->change.cookielist = NULL;

    Curl_safefree(outcurl->state.buffer);

    Curl_safefree(outcurl->state.headerbuff);

    Curl_safefree(outcurl->change.url);

    Curl_safefree(outcurl->change.referer);