From c1312cab1fe017e2e09d60a5adda2d6126baeb39 Mon Sep 17 00:00:00 2001
From: Daniel Stenberg <daniel@haxx.se>
Date: Fri, 10 Dec 2004 09:45:55 +0000
Subject: [PATCH] Added two chapters: Custom Request Elements and Debug.
---
docs/TheArtOfHttpScripting | 78 +++++++++++++++++++++++++++++++-------
1 file changed, 64 insertions(+), 14 deletions(-)
diff --git a/docs/TheArtOfHttpScripting b/docs/TheArtOfHttpScripting
index 3368a8d771..cac8cd70d9 100644
--- a/docs/TheArtOfHttpScripting
+++ b/docs/TheArtOfHttpScripting
@@ -1,5 +1,5 @@
Online: http://curl.haxx.se/docs/httpscripting.shtml
-Date: December 7, 2004
+Date: December 9, 2004
The Art Of Scripting HTTP Requests Using Curl
=============================================
@@ -137,11 +137,11 @@ Date: December 7, 2004
you need to replace that space with %20 etc. Failing to comply with this
will most likely cause your data to be received wrongly and messed up.
- 4.3 FILE UPLOAD POST
+ 4.3 File Upload POST
- Back in late 1995 they defined a new way to post data over HTTP. It was
- documented in the RFC 1867, why this method sometimes is referred to as
- a RFC1867-posting.
+ Back in late 1995 they defined an additional way to post data over HTTP. It
+ is documented in the RFC 1867, why this method sometimes is referred to as
+ RFC1867-posting.
This method is mainly designed to better support file uploads. A form that
allows a user to upload a file could be written like this in HTML:
@@ -158,7 +158,7 @@ Date: December 7, 2004
curl -F upload=@localfilename -F press=OK [URL]
- 4.4 HIDDEN FIELDS
+ 4.4 Hidden Fields
A very common way for HTML based application to pass state information
between pages is to add hidden fields to the forms. Hidden fields are
@@ -179,7 +179,7 @@ Date: December 7, 2004
curl -d "birthyear=1905&press=OK&person=daniel" [URL]
- 4.5 FIGURE OUT WHAT A POST LOOKS LIKE
+ 4.5 Figure Out What A POST Looks Like
When you're about fill in a form and send to a server by using curl instead
of a browser, you're of course very interested in sending a POST exactly the
@@ -202,7 +202,7 @@ Date: December 7, 2004
curl -T uploadfile www.uploadhttp.com/receive.cgi
-6. AUTHENTICATION
+6. Authentication
Authentication is the ability to tell the server your username and password
so that it can verify that you're allowed to do the request you're doing. The
@@ -237,7 +237,7 @@ Date: December 7, 2004
able to watch your passwords if you pass them as plain command line
options. There are ways to circumvent this.
-7. REFERER
+7. Referer
A HTTP request may include a 'referer' field (yes it is misspelled), which
can be used to tell from which URL the client got to this particular
@@ -251,7 +251,7 @@ Date: December 7, 2004
curl -e http://curl.haxx.se daniel.haxx.se
-8. USER AGENT
+8. User Agent
Very similar to the referer field, all HTTP requests may set the User-Agent
field. It names what user agent (client) that is being used. Many
@@ -273,7 +273,7 @@ Date: December 7, 2004
curl -A "Mozilla/4.73 [en] (X11; U; Linux 2.2.15 i686)" [URL]
-9. REDIRECTS
+9. Redirects
When a resource is requested from a server, the reply from the server may
include a hint about where the browser should go next to find this page, or a
@@ -292,7 +292,7 @@ Date: December 7, 2004
page, you can safely use -L and -d/-F together. Curl will only use POST in
the first request, and then revert to GET in the following operations.
-10. COOKIES
+10. Cookies
The way the web browsers do "client side state control" is by using
cookies. Cookies are just names with associated contents. The cookies are
@@ -364,7 +364,7 @@ Date: December 7, 2004
curl https://that.secure.server.com
- 11.1 CERTIFICATES
+ 11.1 Certificates
In the HTTPS world, you use certificates to validate that you are the one
you you claim to be, as an addition to normal passwords. Curl supports
@@ -387,7 +387,57 @@ Date: December 7, 2004
http://curl.haxx.se/docs/sslcerts.html
-12. REFERENCES
+12. Custom Request Elements
+
+ Doing fancy stuff, you may need to add or change elements of a single curl
+ request.
+
+ For example, you can change the POST request to a PROPFIND and send the data
+ as "Content-Type: text/xml" (instead of the default Content-Type) like this:
+
+ curl -d "<xml>" -H "Content-Type: text/xml" -X PROPFIND url.com
+
+ You can delete a default header by providing one without content. Like you
+ can ruin the request by chopping off the Host: header:
+
+ curl -H "Host:" http://mysite.com
+
+ You can add headers the same way. Your server may want a "Destination:"
+ header, and you can add it:
+
+ curl -H "Destination: http://moo.com/nowhere" http://url.com
+
+13. Debug
+
+ Many times when you run curl on a site, you'll notice that the site doesn't
+ seem to respond the same way to your curl requests as it does to your
+ browser's.
+
+ Then you need to start making your curl requests more similar to your
+ browser's requests:
+
+ * Use the --trace-ascii option to store fully detailed logs of the requests
+ for easier analyzing and better understanding
+
+ * Make sure you check for and use cookies when needed (both reading with -b
+ and writing with -c)
+
+ * Set user-agent to one like a recent popular browser does
+
+ * Set referer like it is set by the browser
+
+ * If you use POST, make sure you send all the fields and in the same order as
+ the browser does it. (See chapter 4.5 above)
+
+ A very good helper to make sure you do this right, is the LiveHTTPHeader tool
+ that lets you view all headers you send and receive with Mozilla/Firefox
+ (even when using HTTPS).
+
+ A more raw approach is to capture the HTTP traffic on the network with tools
+ such as ethereal or tcpdump and check what headers that were sent and
+ received by the browser. (HTTPS makes this technique inefficient.)
+
+14. References
RFC 2616 is a must to read if you want in-depth understanding of the HTTP
protocol.
--
GitLab