Skip to content
Snippets Groups Projects
Commit a737864a authored by Daniel Stenberg's avatar Daniel Stenberg
Browse files

ispell-buffer

parent c68a6805
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
Showing
with 76 additions and 80 deletions
docs/libcurl-the-guide
+ 76
− 80
View file @ a737864a
......@@ -16,7 +16,7 @@ About this Document
This document will refer to 'the user' as the person writing the source code
that uses libcurl. That would probably be you or someone in your position.
What will be generally refered to as 'the program' will be the collected
What will be generally referred to as 'the program' will be the collected
source code that you write that is using libcurl for transfers. The program
is outside libcurl and libcurl is outside of the program.
......@@ -44,7 +44,7 @@ Building
When having compiled the program, you need to link your object files to
create a single executable. For that to succeed, you need to link with
libcurl and possibly also with other libraries that libcurl itself depends
on. Like OpenSSL librararies, but even some standard OS libraries may be
on. Like OpenSSL libraries, but even some standard OS libraries may be
needed on the command line. To figure out which flags to use, once again
the 'curl-config' tool comes to the rescue:
......@@ -88,22 +88,22 @@ Global Preparation
curl_global_init()
and it takes one parameter which is a bit pattern that tells libcurl what to
intialize. Using CURL_GLOBAL_ALL will make it initialize all known internal
initialize. Using CURL_GLOBAL_ALL will make it initialize all known internal
sub modules, and might be a good default option. The current two bits that
are specified are:
CURL_GLOBAL_WIN32 which only does anything on Windows machines. When used on
a Windows machine, it'll make libcurl intialize the win32 socket
a Windows machine, it'll make libcurl initialize the win32 socket
stuff. Without having that initialized properly, your program cannot use
sockets properly. You should only do this once for each application, so if
your program already does this or of another library in use does it, you
should not tell libcurl to do this as well.
CURL_GLOBAL_SSL which only does anything on libcurls compiled and built
SSL-enabled. On these systems, this will make libcurl init OpenSSL properly
for this application. This is only needed to do once for each application so
if your program or another library already does this, this bit should not be
needed.
SSL-enabled. On these systems, this will make libcurl initialize OpenSSL
properly for this application. This is only needed to do once for each
application so if your program or another library already does this, this
bit should not be needed.
libcurl has a default protection mechanism that detects if curl_global_init()
hasn't been called by the time curl_easy_perform() is called and if that is
......@@ -121,7 +121,7 @@ Global Preparation
Features libcurl Provides
It is considered best-practise to determine libcurl features run-time rather
It is considered best-practice to determine libcurl features run-time rather
than build-time (if possible of course). By calling curl_version_info() and
checking tout he details of the returned struct, your program can figure out
exactly what the currently running libcurl supports.
......@@ -138,7 +138,7 @@ Handle the Easy libcurl
interface first, so please continue reading for better understanding.
To use the easy interface, you must first create yourself an easy handle. You
need one handle for each easy session you want to perform. Basicly, you
need one handle for each easy session you want to perform. Basically, you
should use one handle for every thread you plan to use for transferring. You
must never share the same handle in multiple threads.
......@@ -155,7 +155,7 @@ Handle the Easy libcurl
set in the handle until set again to something different. Alas, multiple
requests using the same handle will use the same options.
Many of the informationals you set in libcurl are "strings", pointers to data
Many of the options you set in libcurl are "strings", pointers to data
terminated with a zero byte. Keep in mind that when you set strings with
curl_easy_setopt(), libcurl will not copy the data. It will merely point to
the data. You MUST make sure that the data remains available for libcurl to
......@@ -167,7 +167,7 @@ Handle the Easy libcurl
curl_easy_setopt(easyhandle, CURLOPT_URL, "http://curl.haxx.se/");
Let's assume for a while that you want to receive data as the URL indentifies
Let's assume for a while that you want to receive data as the URL identifies
a remote resource you want to get here. Since you write a sort of application
that needs this transfer, I assume that you would like to get the data passed
to you directly instead of simply getting it passed to stdout. So, you write
......@@ -229,26 +229,21 @@ Handle the Easy libcurl
Multi-threading issues
libcurl is completely thread safe, except for two issues: signals and alarm
handlers. Signals are needed for a SIGPIPE handler, and the alarm() syscall
is used to catch timeouts (mostly during DNS lookup).
handlers. Signals are needed for a SIGPIPE handler, and the alarm() Bacall
is used to catch timeouts (mostly during ENS lookup).
If you are accessing HTTPS or FTPS URLs in a multi-threaded manner, you are
then of course using OpenSSL multi-threaded and it has itself a few
requirements on this. Basicly, you need to provide one or two functions to
requirements on this. Basilio, you need to provide one or two functions to
allow it to function properly. For all details, see this:
http://www.openssl.org/docs/crypto/threads.html#DESCRIPTION
When using multiple threads you should first ignore SIGPIPE in your main
thread and set the CURLOPT_NOSIGNAL option to TRUE for all handles.
Everything will work fine except that timeouts are not honored during the DNS
lookup - which you can work around by building libcurl with ares-support.
Ares is a library that provides asynchronous name resolves. Unfortunately,
ares does not yet support IPv6.
For SIGPIPE info see the UNIX Socket FAQ at
http://www.unixguide.net/network/socketfaq/2.22.shtml
When using multiple threads you should set the CURLOPT_NOSIGNAL option to
TRUE for all handles. Everything will work fine except that timeouts are not
honored during the DNS lookup - which you can work around by building libcurl
with c-ares support. c-ares is a library that provides asynchronous name
resolves. Unfortunately, c-ares does not yet support IPv6.
Also, note that CURLOPT_DNS_USE_GLOBAL_CACHE is not thread-safe.
......@@ -261,10 +256,10 @@ When It Doesn't Work
There's one golden rule when these things occur: set the CURLOPT_VERBOSE
option to TRUE. It'll cause the library to spew out the entire protocol
details it sends, some internal info and some received protcol data as well
details it sends, some internal info and some received protocol data as well
(especially when using FTP). If you're using HTTP, adding the headers in the
received output to study is also a clever way to get a better understanding
wht the server behaves the way it does. Include headers in the normal body
why the server behaves the way it does. Include headers in the normal body
output with CURLOPT_HEADER set TRUE.
Of course there are bugs left. We need to get to know about them to be able
......@@ -352,7 +347,7 @@ Passwords
curl_easy_setopt(easyhandle, CURLOPT_USERPWD, "myname:thesecret");
Another case where name and password might be needed at times, is for those
users who need to athenticate themselves to a proxy they use. libcurl offers
users who need to authenticate themselves to a proxy they use. libcurl offers
another option for this, the CURLOPT_PROXYUSERPWD. It is used quite similar
to the CURLOPT_USERPWD option like this:
......@@ -392,7 +387,7 @@ HTTP Authentication
many different ways a client can provide those credentials to the server and
you can control what way libcurl will (attempt to) use. The default HTTP
authentication method is called 'Basic', which is sending the name and
password in clear-text in the HTTP request, base64-encoded. This is unsecure.
password in clear-text in the HTTP request, base64-encoded. This is insecure.
At the time of this writing libcurl can be built to use: Basic, Digest, NTLM,
Negotiate, GSS-Negotiate and SPNEGO. You can tell libcurl which one to use
......@@ -437,7 +432,7 @@ HTTP POSTing
curl_easy_perform(easyhandle); /* post away! */
Simple enough, huh? Since you set the POST options with the
CURLOPT_POSTFIELDS, this automaticly switches the handle to use POST in the
CURLOPT_POSTFIELDS, this automatically switches the handle to use POST in the
upcoming request.
Ok, so what if you want to post binary data that also requires you to set the
......@@ -464,12 +459,12 @@ HTTP POSTing
curl_slist_free_all(headers); /* free the header list */
While the simple examples above cover the majority of all cases where HTTP
POST operations are required, they don't do multipart formposts. Multipart
POST operations are required, they don't do multi-part formposts. Multi-part
formposts were introduced as a better way to post (possibly large) binary
data and was first documented in the RFC1867. They're called multipart
data and was first documented in the RFC1867. They're called multi-part
because they're built by a chain of parts, each being a single unit. Each
part has its own name and contents. You can in fact create and post a
multipart formpost with the regular libcurl POST support described above, but
multi-part formpost with the regular libcurl POST support described above, but
that would require that you build a formpost yourself and provide to
libcurl. To make that easier, libcurl provides curl_formadd(). Using this
function, you add parts to the form. When you're done adding parts, you post
......@@ -563,7 +558,7 @@ Showing Progress
libcurl with C++
There's basicly only one thing to keep in mind when using C++ instead of C
There's basically only one thing to keep in mind when using C++ instead of C
when interfacing libcurl:
"The Callbacks Must Be Plain C"
......@@ -590,8 +585,8 @@ Proxies
as a substitute for another".
Proxies are exceedingly common these days. Companies often only offer
internet access to employees through their HTTP proxies. Network clients or
user-agents ask the proxy for docuements, the proxy does the actual request
Internet access to employees through their HTTP proxies. Network clients or
user-agents ask the proxy for documents, the proxy does the actual request
and then it returns them.
libcurl has full support for HTTP proxies, so when a given URL is wanted,
......@@ -601,7 +596,7 @@ Proxies
The fact that the proxy is a HTTP proxy puts certain restrictions on what can
actually happen. A requested URL that might not be a HTTP URL will be still
be passed to the HTTP proxy to deliver back to libcurl. This happens
transparantly, and an application may not need to know. I say "may", because
transparently, and an application may not need to know. I say "may", because
at times it is very important to understand that all operations over a HTTP
proxy is using the HTTP protocol. For example, you can't invoke your own
custom FTP commands or even proper FTP directory listings.
......@@ -622,9 +617,9 @@ Proxies
Environment Variables
libcurl automaticly checks and uses a set of environment variables to know
what proxies to use for certain protocols. The names of the variables are
following an ancient de facto standard and are built up as
libcurl automatically checks and uses a set of environment variables to
know what proxies to use for certain protocols. The names of the variables
are following an ancient de facto standard and are built up as
"[protocol]_proxy" (note the lower casing). Which makes the variable
'http_proxy' checked for a name of a proxy to use when the input URL is
HTTP. Following the same rule, the variable named 'ftp_proxy' is checked
......@@ -632,11 +627,12 @@ Proxies
names of the variables simply allows different HTTP proxies to be used.
The proxy environment variable contents should be in the format
"[protocol://]machine[:port]". Where the protocol:// part is simply
ignored if present (so http://proxy and bluerk://proxy will do the same)
and the optional port number specifies on which port the proxy operates on
the host. If not specified, the internal default port number will be used
and that is most likely *not* the one you would like it to be.
"[protocol://][user:password]machine[:port]". Where the protocol:// part
is simply ignored if present (so http://proxy and bluerk://proxy will do
the same) and the optional port number specifies on which port the proxy
operates on the host. If not specified, the internal default port number
will be used and that is most likely *not* the one you would like it to
be.
There are two special environment variables. 'all_proxy' is what sets
proxy for any URL in case the protocol specific variable wasn't set, and
......@@ -647,7 +643,7 @@ Proxies
SSL and Proxies
SSL is for secure point-to-point connections. This involves strong
encryption and similar things, which effectivly makes it impossible for a
encryption and similar things, which effectively makes it impossible for a
proxy to operate as a "man in between" which the proxy's task is, as
previously discussed. Instead, the only way to have SSL work over a HTTP
proxy is to ask the proxy to tunnel trough everything without being able
......@@ -678,7 +674,7 @@ Proxies
operations over a HTTP proxy. You can in fact use things such as FTP
upload or FTP custom commands this way.
Again, this is often prevented by the adminstrators of proxies and is
Again, this is often prevented by the administrators of proxies and is
rarely allowed.
Tell libcurl to use proxy tunneling like this:
......@@ -692,13 +688,13 @@ Proxies
Proxy Auto-Config
Netscape first came up with this. It is basicly a web page (usually using
a .pac extension) with a javascript that when executed by the browser with
the requested URL as input, returns information to the browser on how to
connect to the URL. The returned information might be "DIRECT" (which
means no proxy should be used), "PROXY host:port" (to tell the browser
where the proxy for this particular URL is) or "SOCKS host:port" (to
direct the brower to a SOCKS proxy).
Netscape first came up with this. It is basically a web page (usually
using a .pac extension) with a javascript that when executed by the
browser with the requested URL as input, returns information to the
browser on how to connect to the URL. The returned information might be
"DIRECT" (which means no proxy should be used), "PROXY host:port" (to tell
the browser where the proxy for this particular URL is) or "SOCKS
host:port" (to direct the browser to a SOCKS proxy).
libcurl has no means to interpret or evaluate javascript and thus it
doesn't support this. If you get yourself in a position where you face
......@@ -716,7 +712,7 @@ Proxies
- Ask your admins to stop this, for a static proxy setup or similar.
Persistancy Is The Way to Happiness
Persistence Is The Way to Happiness
Re-cycling the same easy handle several times when doing multiple requests is
the way to go.
......@@ -727,11 +723,11 @@ Persistancy Is The Way to Happiness
reduces network impact a lot.
Even if the connection is dropped, all connections involving SSL to the same
host again, will benefit from libcurl's session ID cache that drasticly
host again, will benefit from libcurl's session ID cache that drastically
reduces re-connection time.
FTP connections that are kept alive saves a lot of time, as the command-
response roundtrips are skipped, and also you don't risk getting blocked
response round-trips are skipped, and also you don't risk getting blocked
without permission to login again like on many FTP servers only allowing N
persons to be logged in at the same time.
......@@ -757,13 +753,13 @@ Persistancy Is The Way to Happiness
used for the longest time. This is the default behavior.
CURLCLOSEPOLICY_OLDEST closes the oldest connection, the one that was
createst the longest time ago.
created the longest time ago.
There are, or at least were, plans to support a close policy that would call
a user-specified callback to let the user be able to decide which connection
to dump when this is necessary and therefor is the CURLOPT_CLOSEFUNCTION an
existing option still today. Nothing ever uses this though and this will not
be used within the forseeable future either.
be used within the foreseeable future either.
To force your upcoming request to not use an already existing connection (it
will even close one first if there happens to be one alive to the same host
......@@ -775,8 +771,8 @@ Persistancy Is The Way to Happiness
HTTP Headers Used by libcurl
When you use libcurl to do HTTP requeests, it'll pass along a series of
headers automaticly. It might be good for you to know and understand these
When you use libcurl to do HTTP requests, it'll pass along a series of
headers automatically. It might be good for you to know and understand these
ones.
Host
......@@ -787,7 +783,7 @@ HTTP Headers Used by libcurl
Pragma
"no-cache". Tells a possible proxy to not grap a copy from the cache but
"no-cache". Tells a possible proxy to not grab a copy from the cache but
to fetch a fresh one.
Accept:
......@@ -861,7 +857,7 @@ Customizing Operations
headers = curl_slist_append(headers, "Accept:");
Both replacing and cancelling internal headers should be done with careful
Both replacing and canceling internal headers should be done with careful
consideration and you should be aware that you may violate the HTTP
protocol when doing so.
......@@ -871,7 +867,7 @@ Customizing Operations
chunked" when doing a non-GET HTTP operation, libcurl will switch over to
"chunked" upload, even though the size of the data to upload might be
known. By default, libcurl usually switches over to chunked upload
automaticly if the upload data size is unknown.
automatically if the upload data size is unknown.
HTTP Version
......@@ -891,10 +887,10 @@ Customizing Operations
you want to make for example your FTP transfers to behave differently.
Sending custom commands to a FTP server means that you need to send the
comands exactly as the FTP server expects them (RFC959 is a good guide
commands exactly as the FTP server expects them (RFC959 is a good guide
here), and you can only use commands that work on the control-connection
alone. All kinds of commands that requires data interchange and thus needs
a data-connection must be left to libcurl's own judgement. Also be aware
a data-connection must be left to libcurl's own judgment. Also be aware
that libcurl will do its very best to change directory to the target
directory before doing any transfer, so if you change directory (with CWD
or similar) you might confuse libcurl and then it might not attempt to
......@@ -958,16 +954,16 @@ Cookies Without Chocolate Chips
curl_easy_setopt(easyhandle, CURLOPT_COOKIE, "name1=var1; name2=var2;");
In many cases, that is not enough. You might want to dynamicly save whatever
cookies the remote server passes to you, and make sure those cookies are then
use accordingly on later requests.
In many cases, that is not enough. You might want to dynamically save
whatever cookies the remote server passes to you, and make sure those cookies
are then use accordingly on later requests.
One way to do this, is to save all headers you receive in a plain file and
when you make a request, you tell libcurl to read the previous headers to
figure out which cookies to use. Set header file to read cookies from with
CURLOPT_COOKIEFILE.
The CURLOPT_COOKIEFILE option also automaticly enables the cookie parser in
The CURLOPT_COOKIEFILE option also automatically enables the cookie parser in
libcurl. Until the cookie parser is enabled, libcurl will not parse or
understand incoming cookies and they will just be ignored. However, when the
parser is enabled the cookies will be understood and the cookies will be kept
......@@ -980,7 +976,7 @@ Cookies Without Chocolate Chips
If you rather use existing cookies that you've previously received with your
Netscape or Mozilla browsers, you can make libcurl use that cookie file as
input. The CURLOPT_COOKIEFILE is used for that too, as libcurl will
automaticly find out what kind of file it is and act accordingly.
automatically find out what kind of file it is and act accordingly.
The perhaps most advanced cookie operation libcurl offers, is saving the
entire internal cookie state back into a Netscape/Mozilla formatted cookie
......@@ -1000,7 +996,7 @@ FTP Peculiarities We Need
libcurl can either connect to the server a second time or tell the server to
connect back to it. The first option is the default and it is also what works
best for all the people behind firewalls, NATs or IP-masquarading setups.
best for all the people behind firewalls, NATs or IP-masquerading setups.
libcurl then tells the server to open up a new port and wait for a second
connection. This is by default attempted with EPSV first, and if that doesn't
work it tries PASV instead. (EPSV is an extension to the original FTP spec
......@@ -1073,7 +1069,7 @@ Security Considerations
.netrc
.netrc is a pretty handy file/feature that allows you to login quickly and
automaticly to frequently visited sites. The file contains passwords in
automatically to frequently visited sites. The file contains passwords in
clear text and is a real security risk. In some cases, your .netrc is also
stored in a home directory that is NFS mounted or used on another network
based file system, so the clear text password will fly through your
......@@ -1087,8 +1083,8 @@ Security Considerations
Many of the protocols libcurl supports send name and password unencrypted
as clear text (HTTP Basic authentication, FTP, TELNET etc). It is very
easy for anyone on your network or a network nearby yours, to just fire up
a network analyzer tool and evesdrop on your passwords. Don't let the fact
that HTTP uses base64 encoded passwords fool you. They may not look
a network analyzer tool and eavesdrop on your passwords. Don't let the
fact that HTTP uses base64 encoded passwords fool you. They may not look
readable at a first glance, but they very easily "deciphered" by anyone
within seconds.
......@@ -1100,14 +1096,14 @@ Security Considerations
Showing What You Do
On a related issue, be aware that even in situations like when you have
problems with libcurl and ask somone for help, everything you reveal in
problems with libcurl and ask someone for help, everything you reveal in
order to get best possible help might also impose certain security related
risks. Host names, user names, paths, operating system specifics etc (not
to mention passwords of course) may in fact be used by intruders to gain
additional information of a potential target.
To avoid this problem, you must of course use your common sense. Often,
you can just edit out the senstive data or just rearch/replace your true
you can just edit out the sensitive data or just search/replace your true
information with faked data.
......@@ -1152,8 +1148,8 @@ Multiple Transfers Using the multi Interface
wants to do. Take note that libcurl does also feature some time-out code so
we advice you to never use very long timeouts on select() before you call
curl_multi_perform(), which thus should be called unconditionally every now
and then even if none of its file descriptors have signalled ready. Another
precation you should use: always call curl_multi_fdset() immediately before
and then even if none of its file descriptors have signaled ready. Another
precaution you should use: always call curl_multi_fdset() immediately before
the select() call since the current set of file descriptors may change when
calling a curl function.
......@@ -1183,7 +1179,7 @@ Sharing Data Between Easy Handles
Footnotes:
[1] = libcurl 7.10.3 and later have the ability to switch over to chunked
Tranfer-Encoding in cases were HTTP uploads are done with data of an
Transfer-Encoding in cases were HTTP uploads are done with data of an
unknown size.
[2] = This happens on Windows machines when libcurl is built and used as a
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment