httpx is a fast and multi-purpose HTTP toolkit allow to run multiple probers using retryablehttp library, it is designed to maintain the result reliability with increased threads.

Resources

Features

• Simple and modular code base making it easy to contribute.
• Fast And fully configurable flags to probe mutiple elements.
• Supports multiple HTTP based probings.
• Smart auto fallback from https to http as default.
• Supports hosts, URLs and CIDR as input.
• Handles edge cases doing retries, backoffs etc for handling WAFs.

Supported probes:-

Probes Default check Probes Default check URL true IP true Title true CNAME true Status Code true Raw HTTP false Content Length true HTTP2 false TLS Certificate true HTTP 1.1 Pipeline false CSP Header true Virtual host false Location Header true CDN false Web Server true Path false Web Socket true Ports false Response Time true Request method false

Installation Instructions

From Binary

The installation is easy. You can download the pre-built binaries for your platform from the Releases page. Extract them using tar, move it to your $PATHand you're ready to go.

From Source

httpx requires go1.14+ to install successfully. Run the following command to get the repo -

From Github

Usage

This will display help for the tool. Here are all the switches it supports.

Flag Description Example H Custom Header input httpx -H 'x-bug-bounty: hacker' follow-redirects Follow URL redirects (default false) httpx -follow-redirects follow-host-redirects Follow URL redirects only on same host(default false) httpx -follow-host-redirects http-proxy URL of the proxy server httpx -http-proxy hxxp://proxy-host:80 l File containing HOST/URLs/CIDR to process httpx -l hosts.txt no-color Disable colors in the output. httpx -no-color o File to save output result (optional) httpx -o output.txt json Prints all the probes in JSON format (default false) httpx -json vhost Probes to detect vhost from list of subdomains httpx -vhost threads Number of threads (default 50) httpx -threads 100 http2 HTTP2 probing httpx -http2 pipeline HTTP1.1 Pipeline probing httpx -pipeline ports Ports ranges to probe (nmap syntax: eg 1,2-10,11) httpx -ports 80,443,100-200 title Prints title of page if available httpx -title path Request path/file httpx -path /api paths Request list of paths from file httpx -paths paths.txt content-length Prints content length in the output httpx -content-length ml Match content length in the output httpx -content-length -ml 125 fl Filter content length in the output httpx -content-length -fl 0,43 status-code Prints status code in the output httpx -status-code mc Match status code in the output httpx -status-code -mc 200,302 fc Filter status code in the output httpx -status-code -fc 404,500 tech-detect Perform wappalyzer based technology detection httpx -tech-detect tls-probe Send HTTP probes on the extracted TLS domains httpx -tls-probe tls-grab Perform TLS data grabbing httpx -tls-grab content-type Prints content-type httpx -content-type location Prints location header httpx -location csp-probe Send HTTP probes on the extracted CSP domains httpx -csp-probe web-server Prints running web sever if available httpx -web-server sr Store responses to file (default false) httpx -sr srd Directory to store response (optional) httpx -srd httpx-output unsafe Send raw requests skipping golang normalization httpx -unsafe request File containing raw request to process httpx -request retries Number of retries httpx -retries random-agent Use randomly selected HTTP User-Agent header value httpx -random-agent silent Prints only results in the output httpx -silent stats Prints statistic every 5 seconds httpx -stats timeout Timeout in seconds (default 5) httpx -timeout 10 verbose Verbose Mode httpx -verbose version Prints current version of the httpx httpx -version x Request Method (default 'GET') httpx -x HEAD method Output requested method httpx -method response-time Output the response time httpx -response-time response-in-json Include response in stdout (only works with -json) httpx -response-in-json websocket Prints if a websocket is exposed httpx -websocket ip Prints the host IP httpx -ip cname Prints the cname record if available httpx -cname cdn Check if domain's ip belongs to known CDN httpx -cdn filter-string Filter results based on filtered string httpx -filter-string XXX match-string Filter results based on matched string httpx -match-string XXX filter-regex Filter results based on filtered regex httpx -filter-regex XXX match-regex Filter results based on matched regex httpx -match-regex XXX

Running httpx with stdin

This will run the tool against all the hosts and subdomains in hosts.txt and returns URLs running HTTP webserver.

Running httpx with file input

This will run the tool against all the hosts and subdomains in hosts.txt and returns URLs running HTTP webserver.

Running httpx with CIDR input

Running httpx with subfinder

Notes

• As default, httpx checks for HTTPS probe and fall-back to HTTP only if HTTPS is not reachable.
• For printing both HTTP/HTTPS results, no-fallback flag can be used.
• Custom scheme for ports can be defined, for example -ports http:443,http:80,https:8443
• vhost, http2, pipeline, ports, csp-probe, tls-probe and path are unique flag with different probes.
• Unique flags should be used for specific use cases instead of running them as default with other flags.
• When using json flag, all the information (default probes) included in the JSON output.

Thanks

httpx is made with by the projectdiscovery team. Community contributions have made the project what it is. See the Thanks.md file for more details. Do also check out these similar awesome projects that may fit in your workflow:

Probing feature is inspired by @tomnomnom/httprobe work