Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
T
TLMSP curl
Manage
Activity
Members
Labels
Plan
Issues
0
Issue boards
Milestones
Wiki
Code
Merge requests
0
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Package Registry
Container Registry
Operate
Environments
Terraform modules
Monitor
Incidents
Service Desk
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
CYBER - Cyber Security
TS 103 523 MSP
TLMSP
TLMSP curl
Commits
47925f3d
Commit
47925f3d
authored
16 years ago
by
Daniel Stenberg
Browse files
Options
Downloads
Patches
Plain Diff
Added a new "13. Web Login" chapter
parent
82c5950c
Branches
Branches containing commit
Tags
Tags containing commit
No related merge requests found
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
docs/TheArtOfHttpScripting
+48
-10
48 additions, 10 deletions
docs/TheArtOfHttpScripting
with
48 additions
and
10 deletions
docs/TheArtOfHttpScripting
+
48
−
10
View file @
47925f3d
Online: http://curl.haxx.se/docs/httpscripting.html
Date:
December 9
, 200
4
Date:
May 28
, 200
8
The Art Of Scripting HTTP Requests Using Curl
=============================================
...
...
@@ -137,6 +137,10 @@ Date: December 9, 2004
you need to replace that space with %20 etc. Failing to comply with this
will most likely cause your data to be received wrongly and messed up.
Recent curl versions can in fact url-encode POST data for you, like this:
curl --data-urlencode "name=I am Daniel" www.example.com
4.3 File Upload POST
Back in late 1995 they defined an additional way to post data over HTTP. It
...
...
@@ -202,14 +206,14 @@ Date: December 9, 2004
curl -T uploadfile www.uploadhttp.com/receive.cgi
6. Authentication
6.
HTTP
Authentication
Authentication is the ability to tell the server your username and
password
so that it can verify that you're allowed to do the request you're
doing. The
Basic authentication used in HTTP (which is the type curl uses by
default) is
*plain* *text* based, which means it sends username and password
only
slightly obfuscated, but still fully readable by anyone that sniffs on
the
network between you and the remote server.
HTTP
Authentication is the ability to tell the server your username and
password
so that it can verify that you're allowed to do the request you're
doing. The
Basic authentication used in HTTP (which is the type curl uses by
default) is
*plain* *text* based, which means it sends username and password
only
slightly obfuscated, but still fully readable by anyone that sniffs on
the
network between you and the remote server.
To tell curl to use a user and password for authentication:
...
...
@@ -237,6 +241,10 @@ Date: December 9, 2004
able to watch your passwords if you pass them as plain command line
options. There are ways to circumvent this.
It is worth noting that while this is how HTTP Authentication works, very
many web sites will not use this concept when they provide logins etc. See
the Web Login chapter further below for more details on that.
7. Referer
A HTTP request may include a 'referer' field (yes it is misspelled), which
...
...
@@ -407,7 +415,37 @@ Date: December 9, 2004
curl -H "Destination: http://moo.com/nowhere" http://url.com
13. Debug
13. Web Login
While not strictly just HTTP related, it still cause a lot of people problems
so here's the executive run-down of how the vast majority of all login forms
work and how to login to them using curl.
It can also be noted that to do this properly in an automated fashion, you
will most certainly need to script things and do multiple curl invokes etc.
First, servers mostly use cookies to track the logged-in status of the
client, so you will need to capture the cookies you receive in the
responses. Then, many sites also set a special cookie on the login page (to
make sure you got there through their login page) so you should make a habit
of first getting the login-form page to capture the cookies set there.
Some web-based login systems features various amounts of javascript, and
sometimes they use such code to set or modify cookie contents. Possibly they
do that to prevent programmed logins, like this manual describes how to...
Anyway, if reading the code isn't enough to let you repeat the behavior
manually, capturing the HTTP requests done by your browers and analyzing the
sent cookies is usually a working method to work out how to shortcut the
javascript need.
In the actual <form> tag for the login, lots of sites fill-in random/session
or otherwise secretly generated hidden tags and you may need to first capture
the HTML code for the login form and extract all the hidden fields to be able
to do a proper login POST. Remember that the contents need to be URL encoded
when sent in a normal POST.
14. Debug
Many times when you run curl on a site, you'll notice that the site doesn't
seem to respond the same way to your curl requests as it does to your
...
...
@@ -437,7 +475,7 @@ Date: December 9, 2004
such as ethereal or tcpdump and check what headers that were sent and
received by the browser. (HTTPS makes this technique inefficient.)
1
4
. References
1
5
. References
RFC 2616 is a must to read if you want in-depth understanding of the HTTP
protocol.
...
...
This diff is collapsed.
Click to expand it.
Preview
0%
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment