Curl is a command-line tool used to transfer data to and from servers. It supports various protocols including HTTP, HTTPS, FTP, SFTP, and many more. The tool can be used to download files, upload files, make requests to APIs, and more. In this article, we will explore the different options available in Curl and understand how to use them with examples.
How to Install Curl in Linux
Curl is pre-installed on most Linux distributions, but if it is not installed on your system, it can easily be installed using the package manager.
Install Curl on Ubuntu and Debian:
- To install curl on Ubuntu and Debian, simply run the following command in your terminal:
sudo apt update
sudo apt install curlThis will download and install the latest version of curl from the official Ubuntu/Debian repository. Once installed, you can verify the installation by running the following command:
curl --versionThis should display the version of curl that you have installed, along with information about the features that are enabled.
Install Curl on CentOS and Fedora:
To install curl on CentOS and Fedora, simply run the following command in your terminal:
sudo yum install curlThis will download and install the latest version of curl from the official CentOS/Fedora repository. Once installed, you can verify the installation by running the following command:
curl --versionThis should display the version of curl that you have installed, along with information about the features that are enabled.
Getting started with Curl Command
Before we start with the different options, let’s see how to use Curl to download a file from the internet. The basic syntax of the command is as follows:
curl [options] [URL]Where URL is the address of the resource you want to transfer and options are optional parameters that control how the transfer is performed.
For example, to download the index.html file from the example.com website, the command would be:
curl https://example.com/index.htmlThe above command will download the file and print its contents to the terminal. If you want to save the file to your local machine, use the -o option followed by the filename:
curl -o index.html https://example.com/index.htmlHow to Save the Output to a File With Curl Command
The -o option allows you to save the output of the curl command to a file. This option can be used to download a file from a server and save it to your local machine. The syntax for this option is as follows:
curl -o <file_name> <URL>For example, to download a file named ‘index.html’ from the ‘example.com’ website, the command would be:
curl -o index.html https://example.com/index.htmlHow to Follow Redirects
By default, curl will follow redirects if a server returns a 3xx status code. However, if the server returns a 4xx or 5xx status code, curl will not follow the redirect. The -L option can be used to make curl follow redirects even if the server returns a 4xx or 5xx status code. The syntax for this option is as follows:
curl -L <URL>For example, to follow a redirect from the ‘example.com’ website, the command would be:
curl -L https://example.com/redirectHow to Get Response Headers Only
The -I option can be used to get only the headers of the response from a server. This option is useful if you only need to retrieve information about the response, such as the content type, without downloading the entire content. The syntax for this option is as follows:
curl -I <URL>For example, to get the headers of the ‘index.html’ file from the ‘example.com’ website, the command would be:
curl -I https://example.com/index.htmlHow to Specify the Request Method
The -X option allows you to specify the request method to use when making a request to a server. The default request method is GET, but other methods such as POST, PUT, and DELETE can also be used. The syntax for this option is as follows:
curl -X <request_method> <URL>For example, to make a POST request to the ‘example.com’ API, the command would be:
curl -X POST https://example.com/api/create-userHow to Send Request Data
The -d option allows you to send data in the request body. This option can be used to send data to an API, for example. The data can be sent as plain text or as a JSON object. The syntax for this option is as follows:
curl -X <request_method> -d <data> <URL>For example, to send a JSON object as data in a POST request to the ‘example.com’ API, the command would be:
curl -X POST -d '{"username": "user", "password": "pass"}' https://example.com/api/create-userHow to Set Request Headers
The -H option allows you to set request headers for the curl request. This option can be used to specify the content type, for example. The syntax for this option is as follows:
curl -X <request_method> -H <header> <URL>For example, to set the content type to ‘application/json’ in a POST request to the ‘example.com’ API, the command would be:
curl -X POST -H 'Content-Type: application/json' -d '{"username": "user", "password": "pass"}' https://example.com/api/create-userHow to Ignore SSL Certificates
The -k option allows you to ignore SSL certificates and make a request to a server even if the SSL certificate is not valid. This option should be used with caution, as it can leave you vulnerable to man-in-the-middle attacks. The syntax for this option is as follows:
curl -k <URL>For example, to ignore the SSL certificate and make a request to the ‘example.com’ website, the command would be:
curl -k https://example.comHow to Display Progress Information
The -# option can be used to display progress information for the curl request. This option shows a progress bar with information about the download speed, time remaining, and the percentage of the download that has completed. The syntax for this option is as follows:
curl -# <URL>For example, to display progress information while downloading the ‘index.html’ file from the ‘example.com’ website, the command would be:
curl -# https://example.com/index.htmlHow to Change the User-Agent
To change the User-Agent string that curl sends in its request, use the -A or –user-agent option, followed by the desired User-Agent string. The syntax for this option is as follows:
curl -A <user_agent_string> <URL>For example, to send a request with the User-Agent string “MyCurlClient”, the command would be:
curl -A "MyCurlClient" https://example.comHow to Download Multiple Files
To download multiple files using curl, simply specify multiple URLs in the same command. The syntax for this is as follows:
curl <URL1> <URL2> <URL3> ...For example, to download the files ‘file1.txt’, ‘file2.txt’, and ‘file3.txt’ from the same server, the command would be:
curl http://example.com/file1.txt http://example.com/file2.txt http://example.com/file3.txtHow to Resume a Download
To resume a download that was previously interrupted, use the -C or –continue-at option, followed by the byte offset where the download should resume. The syntax for this option is as follows:
curl -C <byte_offset> <URL>For example, to resume a download that was previously interrupted at byte 5000, the command would be:
curl -C 5000 http://example.com/large_file.zipHow to Test if a Website Supports HTTP/2
To test if a website supports HTTP/2, use the –http2 option. This will force curl to use HTTP/2 when making a request to the specified URL. The syntax for this option is as follows:
curl --http2 <URL>For example, to test if ‘https://example.com‘ supports HTTP/2, the command would be:
curl --http2 https://example.comHow to Specify a Maximum Transfer Rate
To specify a maximum transfer rate for curl, use the –limit-rate option, followed by the maximum rate in bytes per second. The syntax for this option is as follows:
curl --limit-rate <rate> <URL>For example, to limit the transfer rate to 10 KB/s, the command would be:
curl --limit-rate 10240 https://example.com/large_file.zipHow to Transfer Files via FTP
To transfer a file via FTP, simply use the URL of the file in the FTP server. The syntax for this is as follows:
curl ftp://<ftp_server>/<file_path>For example, to download the file ‘file.txt’ from the FTP server ‘ftp.example.com’, the command would be:
curl ftp://ftp.example.com/file.txtHow to Send Cookies
To send cookies with a curl request, use the -b or –cookie option, followed by the cookies in the format ‘key=value’. The syntax for this option is as follows:
curl -b "key1=value1; key2=value2; ... " <URL>
For example, to send the cookies ‘session_id=abcdefg’ and ‘user_id=123456’, the command would be:
curl -b "session_id=abcdefg; user_id=123456" https://example.com
How to Use Proxies
To use a proxy with curl, use the -x or –proxy option, followed by the URL of the proxy server. The syntax for this option is as follows:
curl -x <proxy_server_url> <URL>
For example, to use a proxy located at ‘proxy.example.com:8080’, the command would be:
curl -x http://proxy.example.com:8080 https://example.comAdditionally, you can specify the type of proxy being used with the –proxy-type option. For example, to use a SOCKS5 proxy, the command would be:
curl --proxy-type socks5 -x socks5://proxy.example.com:1080 https://example.comNote that some proxy servers may require authentication. To use authentication with a proxy, use the -U or –proxy-user option, followed by the username and password in the format ‘username:password’. The syntax for this option is as follows:
curl -x <proxy_server_url> -U <username:password> <URL>
For example, to use the proxy server ‘http://proxy.example.com:8080‘ with the username ‘user’ and password ‘pass’, the command would be:
curl -x http://proxy.example.com:8080 -U user:pass https://example.com
Conclusion
Curl is a powerful command-line tool that can be used to make HTTP requests and retrieve information from web servers. The options discussed in this article cover the most common use cases, but there are many other options available for more advanced use cases.
Got questions or suggestions on the article? Don’t hesitate to leave a comment below. Your feedback is valuable to us and helps improve the content quality. And don’t forget to share this article with others who might find it useful. Looking forward to hearing from you!
If our tutorials helped you, please consider buying us a coffee. We appreciate your support!
Thank you for your support.
