• Get application security done the right way! Detect, Protect, Monitor, Accelerate, and more…
  • cURL is a command-line tool to get or send data using URL syntax.

    If you are working as a developer or in the support function, you must be aware of cURL command usage to troubleshoot web applications. cURL is cross-platform utility means you can use on Windows, MAC, and UNIX.

    The following are some of the most used syntaxes with an example to help you.

    Verify if you can connect to URL

    If you are working on a UNIX system and trying to connect the external URL, the first thing you wish to do is to check if you can access URL through curl.

    curl yoururl.com

    It will not throw any output. However, if the server can’t connect, then you will get error such as could not resolve host.

    [[email protected] tmp]# curl helloitdoesntexist.com
    curl: (6) Could not resolve host: helloitdoesntexist.com; Unknown error
    [[email protected] tmp]#

    Save URL/URI output to file

    If you have to save the URL or URI contents to a specific file, you can use the following syntax

    curl https://yoururl.com > yoururl.html

    ex:

    [[email protected] tmp]# curl https://gf.dev > /tmp/gfhtml
      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                     Dload  Upload   Total   Spent    Left  Speed
    100 18557    0 18557    0     0  72565      0 --:--:-- --:--:-- --:--:-- 72772
    [[email protected] tmp]#

    Above example will save all the contents from  gf.dev to /tmp/gf.html

    Show request and response header

    If you are having issues and would like to validate, you are getting the expected request and response header.

    curl -v yoururl.com

    Ex:

    [[email protected] tmp]# curl -v https://geekflare.com
    * About to connect() to geekflare.com port 443 (#0)
    *   Trying 104.25.134.107...
    * Connected to geekflare.com (104.25.134.107) port 443 (#0)
    * Initializing NSS with certpath: sql:/etc/pki/nssdb
    *   CAfile: /etc/pki/tls/certs/ca-bundle.crt
      CApath: none
    * SSL connection using TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
    * Server certificate:
    * 	subject: CN=ssl371609.cloudflaressl.com,OU=PositiveSSL Multi-Domain,OU=Domain Control Validated
    * 	start date: Nov 07 00:00:00 2019 GMT
    * 	expire date: May 15 23:59:59 2020 GMT
    * 	common name: ssl371609.cloudflaressl.com
    * 	issuer: CN=COMODO ECC Domain Validation Secure Server CA 2,O=COMODO CA Limited,L=Salford,ST=Greater Manchester,C=GB
    > GET / HTTP/1.1
    > User-Agent: curl/7.29.0
    > Host: geekflare.com
    > Accept: */*
    > 
    < HTTP/1.1 200 OK
    < Date: Sat, 09 Nov 2019 19:41:37 GMT
    < Content-Type: text/html; charset=UTF-8
    < Transfer-Encoding: chunked
    < Connection: keep-alive
    < Set-Cookie: __cfduid=d2ce6cd359ebc0b6eb5ff3a454ed042021573328497; expires=Sun, 08-Nov-20 19:41:37 GMT; path=/; domain=.geekflare.com; HttpOnly; Secure
    < Vary: Accept-Encoding
    < Link: <https://geekflare.com/wp-json/>; rel="https://api.w.org/"
    < Link: <https://geekflare.com/>; rel=shortlink
    < X-SRCache-Fetch-Status: HIT
    < X-SRCache-Store-Status: BYPASS
    < X-Frame-Options: SAMEORIGIN
    < X-Powered-By: EasyEngine v4.0.12
    < Via: 1.1 google
    < CF-Cache-Status: DYNAMIC
    < Strict-Transport-Security: max-age=15552000; preload
    < X-Content-Type-Options: nosniff
    < Alt-Svc: h3-23=":443"; ma=86400
    < Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
    < Server: cloudflare
    < CF-RAY: 533243e4bcd4bbf4-LHR
    <

    Download at a limit rate

    If you are working on optimization and would like to see how much time does it take to download at a particular speed, you can:-

    curl –-limit-rate 2000B

    Ex:

    curl –-limit-rate 2000B https://gf.dev

    Using a proxy to connect

    Very handy if you are working on the DMZ server where you need to connect to the external world using a proxy.

    curl --proxy yourproxy:port https://yoururl.com

    Test URL with injecting header

    You can use curl by inserting a header with your data to test or troubleshoot the particular issue. Let’s see the following example to request with Content-Type.

    curl --header 'Content-Type: application/json' http://yoururl.com

    By doing above, you are asking curl to pass Content-Type as application/json in the request header.

    Display only response header

    If you are doing some troubleshooting and quickly want to check the response header, you can use the following syntax.

    curl --head http://yoururl.com

    Ex:

    [[email protected] tmp]# curl --head https://chandan.io
    HTTP/1.1 200 OK
    Date: Sat, 09 Nov 2019 19:51:23 GMT
    Content-Type: text/html
    Connection: keep-alive
    Set-Cookie: __cfduid=d3cb2c7b8e566ad99c870b0af12b0f1eb1573329083; expires=Sun, 08-Nov-20 19:51:23 GMT; path=/; domain=.chandan.io; HttpOnly
    X-GUploader-UploadID: AEnB2Uo96JhvJmR2zYUL-Ndh2ta3UD_ykQAB5C7O8cjZQhCf-GxHQ0MsodSzRnl3guSN3ywAYNjtWcPXfwDXjLg3bQ-P5vQMOA
    Expires: Sat, 09 Nov 2019 20:51:23 GMT
    Cache-Control: public, max-age=3600
    Last-Modified: Mon, 06 Aug 2018 10:45:47 GMT
    x-goog-generation: 1533552347482034
    x-goog-metageneration: 1
    x-goog-stored-content-encoding: identity
    x-goog-stored-content-length: 24620
    x-goog-hash: crc32c=DpDPAQ==
    x-goog-hash: md5=cIP/3rusdUx12Zla1kf1yA==
    x-goog-storage-class: MULTI_REGIONAL
    Accept-Ranges: bytes
    CF-Cache-Status: DYNAMIC
    Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
    Server: cloudflare
    CF-RAY: 53325234dc2fbb9a-LHR
    
    [[email protected] tmp]#

    Connect HTTPS/SSL URL and ignore any SSL certificate error

    When you try to access SSL/TLS cert secured URL and if that is having the wrong cert or CN doesn’t match, then you will get the following error.

    curl: (51) Unable to communicate securely with peer: requested domain name does not match the server's certificate.

    Good news, you can instruct cURL to ignore the cert error with --insecure flag.

    curl --insecure https://yoururl.com

    Connect using a specific protocol (SSL/TLS)

    Very handy to test if a particular URL can handshake over specific SSL/TLS protocol.

    
    

    To connect using SSL v3

    curl --sslv3 https://yoururl.com

    and for different TLS versions

    curl --tlsv1 https://example.com
    curl --tlsv1.0 https://example.com
    curl --tlsv1.1 https://example.com
    curl --tlsv1.2 https://example.com
    curl --tlsv1.3 https://example.com

    Download file from FTP Server

    You can use curl to download the file as well by specifying username and password.

    curl -u user:password -O ftp://ftpurl/style.css

    You can always use “-v” with any syntax to print in verbose mode.

    Using Host Header

    The host header is useful to test the target URL over IP when the requested content is only available when host header matches. Or, if you want to test application using load balancer IP/URL.

    curl --header 'Host: targetapplication.com' https://192.0.0.1:8080/

    How about using cURL online?

    Yes, that’s possible with the following tools. You can execute cURL remotely.

    Online CURL – a lightweight tool to fetch the URL online and the possibility to add the following options.

    --connect-timeout
    --cookie
    --data
    --header
    --head
    --location
    --max-time
    --proxy
    --request
    --user
    --url
    --user-agent
    
    

    cURL command line builder – this one is different. It helps you to build the curl command where you can enter information in nice UI, and at the bottom, you will get the cURL command.

    curl-builder

    cURL is a useful utility to troubleshoot a real-time connectivity issue, and I hope the above helps you. If you are looking to learn more, then I would recommend Linux Command Line Basics online course.