curl: my own cheatsheet

curl is a project providing a library and command-line tool for transferring data using various protocols.
The command line tool can be used inside shell scripts with ease and it has around 120 command line options for various tasks.

Is a really useful tool, in both application debug and web penetration testing activities.

Today i’d like to share my own cheatsheet, built during day by day activities.


GET Requests

These curl recipes show you how to send GET requests with curl. The GET method is the default method for making requests in curl so you don’t have to specify any arguments.

Send a GET Request and Print the Response to Screen

$ curl https://www.andreafortuna.org

You don’t need to specify any arguments to make a GET request: the command makes a GET request to https://www.andreafortuna.org and prints the returned content to screen.

Send a GET Request and Save the Response to a File

$ curl -o response.txt https://www.andreafortuna.org

With this command, curl makes a GET request to https://www.andreafortuna.org and saves the body of the HTTP response to response.txt file.


POST requests

By default, curl sends GET requests: to make it send POST requests, use the -X POST command line argument and use the -d argument to add POST data to the request.
To change the Content-Type header that tells the web server what type of data you’re sending, use the -H argument.

Send an Empty POST Request

curl -X POST https://www.andreafortuna.org

This recipe uses the -X POST option that makes curl send a POST request to https://www.andreafortuna.org. Once it receives a response from the server, it prints the response body to the screen. It doesn’t send any POST data.

Send a POST request with form data

curl -d 'login=emma&password=123' -X POST https://site.com/login

This recipe sends a POST request to https://site.com/login with login=emma&password=123 data in request’s body. When using the -d argument, curl also sets the Content-Type header to application/x-www-form-urlencoded. Additionally, when the -d option is set, the -X POST argument can be skipped as curl will automatically set the request’s type to POST.

Send a POST request and follow a redirect

curl -L -d 'tweet=hi' https://api.twitter.com/tweet

By default, curl doesn’t follow the redirects .
The -L command line option tells curl to follow any possible redirects that it may encounter in the way. (the-d argument forces curl to make a POST request)

Send a POST request with JSON data

curl -d '{"login": "andrea", "password": "verystrongpassword"}' -H 'Content-Type: application/json' https://site.com/login

This request can be accomplished passing JSON to the -d option and also using the -H option that sets the Content-Type header to application/json (necessary because otherwise, the web server won’t know what type of data this is).

Send a POST Request with XML Data

curl -d '<user><login>andrea</login><password>verystrongpassword</password></user>' -H 'Content-Type: text/xml' https://site.com/login

Just like the form data and JSON data, XML data is specified in the -d argument, and the Content-Type header is changed to text/xml via the -H argument.

Send a POST request with plain text data

curl -d 'hello, world!' -H 'Content-Type: text/plain' https://site.com/login

Just sends a POST request with a plain text string hello, world! in request’s body. The Content-Type header is filled with text/plain to tell web server that it’s just plain text coming in.

Send a POST Request with Data from a File

curl -d '@data.txt' https://site.com/login

The POST data are loaded from a file called data.txt.
The extra @ symbol before the filename tells curl that data.txt is a file and not just a string.

URL-encode POST Data

curl --data-urlencode 'comment=hello, world!' https://site.com/login

The -d argument assumes that data are already URL-encoded.
If it is not, just replace -d with --data-urlencode: it works exactly the same way as -d, except the data gets URL-encoded by curl before the request.

POST a binary file

curl -F '[email protected]' https://site.com/upload

The -F argument forces curl to make a multipart form data POST request and also sets the Content-Type header to multipart/form-data.

POST a binary file and set its MIME type

curl -F '[email protected];type=image/png' https://site.com/upload

Similar to the previous example, the -F argument is used to upload a binary file via a multipart POST request, but now is also specified the MIME type of this file as image/png.
If no type is specified, then curl sets it to application/octet-stream.

POST a binary file and change its filename

curl -F '[email protected];filename=selfie.png' https://site.com/upload

This example uses the -F argument to upload a photo.png via a POST request and set the filename sent to the web server to selfie.png.


HTTP Headers manipulation

Add a single header

curl -H 'Accept-Language: en-US' https://www.andreafortuna.org

The -H argument is used to add Accept-Language: en-US header to a GET request to https://www.andreafortuna.org: this header tells site to serve the English version of the page.

Add two or more headers

curl -H 'Accept-Language: en-US' -H 'Other header: xyz' -H 'Final header: zzz' https://www.andreafortuna.org

Using multiple -H argoments is possible to send multiple headers to the request.

Add an empty header

curl -H 'Nothing;' https://google.com

If you pass Nothing; (or other text, with a semicolon at the end) as an argument to the -H option, curl converts it to an empty header Nothing: with no value.


User Agent manipulation

Change the User Agent to Firefox

curl -A 'Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:60.0) Gecko/20100101 Firefox/60.0' https://www.andreafortuna.org

The -A argument is used sets the user agent to Firefox 60 (Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:60.0) Gecko/20100101 Firefox/60.0).

Change the User Agent to Chrome

curl -A 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36' https://google.com

Similar to the previous example.

Change the User Agent to Google Bot

curl -A 'Googlebot/2.1 (+http://www.google.com/bot.html)' https://www.andreafortuna.org

Really useful to skip paywalls: often, websites will serve the content to a Google bot (so that they are indexed) but when a real browser requests them, they’ll show a paywall.

Remove the User Agent

curl -A '' https://www.andreafortuna.org

The empty string passed to the -A command line option tells curl to remove the user agent header and not to send User-Agent HTTP header.


Working with cookies

curl -b 'session=xyz' https://www.andreafortuna.org

The -b name=value argument sets a cookie with the name session to the value xyz in a GET request to https://www.andreafortuna.org.

Add two or more cookies

curl -b 'session=xyz' -b 'logged=1' https://www.andreafortuna.org

The same syntax of previous example can be used to set two o more cookies.

curl -b 'session=' https://www.andreafortuna.org

Using only the first half of -b name=value option, curl can send an empty cookie that in the HTTP header looks like this Cookie: session=.

Save cookies to a file

curl -c cookies.txt https://www.andreafortuna.org

The -c cookies.txt option saves the response cookies to cookies.txt file. When curl makes a GET request to https://www.andreafortuna.org, the web server responds with one or more Set-Cookie: name=value header values.
Curl takes them and saves them to a file.

Load cookies from a file

curl -b cookies.txt https://www.andreafortuna.org

The cookies saved in the previous example can be loaded with the -b cookies.txt option.


Referrer manipulation

Add a referrer

curl -e 'https://google.com?q=devops' https://www.andreafortuna.org

This -e argument is used to set the HTTP referrer to https://google.com?q=devops and makes a GET request to https://www.andreafortuna.org. In the access.log file of andreafortuna.org’s webserver, it will appear that someone came from Google while searching for devops.

Send an empty referrer

curl -e '' https://www.andreafortuna.org

Using the -e option leaving the argument empty '' clears the value part of Referer: HTTP header and sends an empty HTTP referer.


Use the Basic HTTP authentication

Set the username and password via the -u argument

curl -u 'andrea:strongpassword' https://greatapp.andreafortuna.org/login

This example uses the -u 'andrea:strongpassword' argument to set the username to andreaand password to strongpassword.
The username and password is then base64-encoded by curl and sent to the server at https://greatapp.andreafortuna.org/login in the Authorization HTTP header.

Set the username and password in the URL

curl https://andrea:[email protected]/login

If you put the username and password in the URL , curl is smart enough to figure it out and it does exactly the same as in the previous example, performing a basic HTTP authentication.

Make curl prompt for the password

curl -u 'andrea' https://greatapp.andreafortuna.org/login

If you skip the password part, curl will asks for the password on the command line.
This is useful if you don’t want the password to be saved in the shell history.


Working with proxies

Use socks proxies

We can use the -x argument to set the proxy protocol, for examplesocks5:

curl -x socks5://andrea:[email protected]:8080 https://www.andreafortuna.org

In this example, the proxy username is set to andrea, proxy password to strongpassword, proxy hostname to proxy.com, and proxy port to 8080. After connecting to the proxy, curl then makes a GET request to https://www.andreafortuna.org.
The same syntax can be used for socks4 proxies:

curl -x socks4://andrea:[email protected]:8080 https://www.andreafortuna.org

Use an HTTP proxy

curl -x andrea:[email protected]:8080 https://www.andreafortuna.org

If you avoid to specify the protocol:// part of the proxy address, curl will assumes that an HTTP proxy is used.


Debugging requests

Set verbosity

curl -v https://www.andreafortuna.org

The -v argument asks curl to print detailed information about the request and the response.
Lines prefixed by > is the data sent to the server, lines prefixed by < is the data received from the server, and lines starting with * is misc information, such as connection information, SSL handshake information, and protocol information.

Detailed trace

curl --trace - https://www.andreafortuna.org

The --trace - argument enables a full trace hexdump of all incoming and outgoing data.
Adding --trace-time argument to the previous example makes curl add timestamps to full trace output

curl --trace - --trace-time https://www.andreafortuna.org

Print only specific debug messages

curl -i https://www.andreafortuna.org

By default, curl prints the response body to the screen: adding the -i argument makes it also print response headers.
If you need, you can also print only the response headers:

curl -s -o /dev/null -D - https://www.andreafortuna.org

The -s argument makes curl silent and hides errors and progress bar, then -o /dev/null makes curl ignore the response body, and -D - prints response headers to stdout.
Further, you can also print only the request headers:

curl -v -s -o /dev/null --stderr - https://www.andreafortuna.org | grep '^>'

A little more tricky: the -v argument enables the verbose output, the -s argument makes curl silent, then the -o /dev/null argument makes curl ignore the output from the server , the --stderr - argument redirects stderr to stdout , and finally print all lines that begin with > (that contain request headers) using grep.
And finally, you can print only the response code:

curl -w '%{response_code}' -s -o /dev/null https://www.andreafortuna.org

The -w argument makes curl print the extra information %{response_code} ( the response code of the request ) after the request has completed.
To make curl only print the code and not the content or other information, we also use -s to silence curl and -o /dev/null that ignores the response output.


References

Related posts

  1. Red Commander: open source Red Team C2 Infrastructure
  2. Backdoorplz, a privilege escalation tool for Windows
  3. Pen Test Partners: Boeing 747 walk through, from a hacker’s perspective
  4. Windows Service Accounts enumeration using Powershell
  5. TLDR #2: Cross-Site Request Forgery