wget Linux Command

wget

wget [options] [urls]

Perform non-interactive file downloads from the Web. wget works in the background and can be used to set up and run a download without the user having to remain logged on. wget supports HTTP, HTTPS, FTP, as well as downloads through HTTP proxies. wget uses a global startup file that you may find at /etc/wgetrc or /usr/local/etc/wgetrc. In addition, users can define their own $HOME/.wgetrc files.

Options

-a logfile, --append-output=logfile
Append output messages to logfile, instead of overwriting the contents. If logfile doesn't exist, create it.

-A acclist, --accept=acclist
Specify a comma-separated list of filename suffixes or patterns to accept.

-b, --background
Go into the background immediately after startup, writing output to the file specified with -o or to wget-log.

-B url, --base=url
Used with -F to prepend the specified URL to relative links in the input file specified with -i.

--bind-address=address
When making client TCP/IP connections, bind( ) to the specified local address, which can be specified as a hostname or IP address. Useful if your system is bound to multiple IP addresses.

-c, --continue
Continue getting a partially downloaded file. Affects the restarting of downloads from an earlier invocation of wget. Works only with FTP servers and HTTP servers that support the Range header.

--connect-timeout=seconds
Set the timeout for a connection to be established in seconds. The default is never to time out, unless a timeout is implemented by system libraries.

--cut-dirs=num
Ignore the specified number of directory components when creating the local directory structure.

-d, --debug
Turn on debugging. wget must have been compiled with debug support.

-D domainlist, --domains=domainlist
Specify a comma-separated list of domains to be followed. Does not turn on -H.

--delete-after
Delete each retrieved file from the local machine after downloading it. Useful for prefetching pages through a proxy. -k is ignored if specified with --delete-after.

--dns-cache=off
Turn off DNS-lookup caching.

--dns-timeout=seconds
Set the DNS lookup timeout to seconds. The default is to never time out.

-e command, --execute=command
Execute the specified command after the commands in .wgetrc, overriding any .wgetrc commands. Can be included multiple times, once for each command to execute.

--exclude-domains=domain-list
Specify a comma-separated list of names that are never to be followed.

-F, --force-html
When reading input from a file, force the file to be treated as an HTML file.

--follow-ftp
Follow FTP links from HTML documents. The default is to ignore FTP links.

--follow-tags=list
Specify a comma-separated list of tags to be considered, overriding the internal table that wget normally uses during a recursive retrieval.

-h, --help
Display usage information and exit.

-H, --span-hosts
Enable spanning across hosts when doing recursive retrieval.

--header=header
Add an additional header to be passed to the HTTP server. The header must include a colon (:) preceded by at least one nonblank character, and with no newline characters. Can be specified multiple times. If header is an empty string, all user-defined headers are cleared.

--html-extension
Append the suffix .html to the filenames of downloaded files where the URL does not include it (for example, an .asp file).

--http-user=user, --http-password=password
Specify the username and password on an HTTP server.

-i file, --input-file=file
Read URLs from the specified file. URLs specified on the command line are accessed before URLs in the file.

-I list, --include-directories=list
Specify a comma-separated list of directories to follow when downloading. The list elements may contain wildcards.

--ignore-length
Ignore the "Content-Length" header on the HTTP server.

--ignore-tags=list
Specify a comma-separated list of tags to be ignored for recursive retrievals.

-k, --convert-links
Convert document links after the download is complete so they work locally.

-K, --backup-converted
When converting a file, back up the original and add a .orig suffix. Affects the behavior of -N.

--keep-session-cookies
Causes --save-cookies to also save session cookies.

-l depth, --level=depths
For recursive retrievals, specify the maximum recursion depth. The default depth is 5.

-L, --relative
Follow relative links only.

--limit-rate=rate
Set the maximum download speed, The default is to specify the rate in bytes, or add a k suffix for kilobytes or m for megabytes.

--load-cookies=file
Load cookies from the specified file before the first HTTP retrieval.

-m, --mirror
Turn on options suitable for mirroring a remote site. Equivalent to -r -N -l inf --no-remove-listing.

-N, --timestamping
Turn on timestamping.

-nc, --no-clobber
Do not download a file if there is already a copy on the disk. The default is to preserve the original copy and rename successive downloads, adding .1, .2, etc. to their name. May not be specified with -N.

-nd, --no-directories
Do not create a directory hierarchy when doing recursive retrievals.

-nH, --no-host-directories
Disable creation of directories prefixed by the name of the host. The default is to include the hostname.

--no-cache
Disable server-side cache for an HTTP retrieval. The default is for caching to be on.

--no-cookies
Disable the use of cookies.

--no-glob
Turn off FTP globbing to prevent the use of wildcards for multiple file retrievals.

--no-http-keep-alive
Turn off the keep-alive feature for HTTP retrievals.

--np, --no-parent
In recursive retrievals, do not ever go up to the parent directory.

--no-remove-listing
Do not remove the temporary .listing files generated by FTP retrievals.

-nv, --non-verbose
Turn off verbose mode, but don't run completely quietly. Displays error messages and basic information.

-o logfile, --output-file=logfile
Log output messages to logfile, instead of the default standard error.

-O file, --output-documents=file
Concatenate all documents into the specified file. If the file exists, it is overwritten. Specify the file as - to write to standard output.

-p, --page-requisites
Download all files necessary to display an HTML page.

-P prefix, --directory-prefix=prefix
Set the directory prefix to the specified value.

--passive-ftp
Perform a passive FTP retrieval.

--post-data=string, --post-file=file
Use POST as the method for HTTP requests and send the specified data in the request body. Use --post-data to send string as data and --post-file to send the file contents.

--progress=type[:style]
Set the progress indicator to type. Valid types are dot and bar; the default is bar. With --progress=dot, you can also set a style. The default style is for each dot to represent 1K, with 10 dots in a cluster and 50 dots per line. Alternatives are binary, with each dot representing 8K, 16-dot clusters, and 48 dots per line; mega, for downloading very large files, with each dot representing 64K, 8 dots per cluster, and 48 dots per line; and giga, with each dot representing 1M, 8 dots per cluster, and 4 clusters per line.

--protocol-directories
Use the protocol name as part of the local filename.

--proxy-user=user, --proxy-passwd=password
Specify the username and password for authentication on a proxy server.

-q, --quiet
Run quietly; don't produce output.

-Q quota, --quota=quota
Specify download quota for automatic retrievals. The default value is in bytes; add k suffix for kilobytes, or m for megabytes.

-r, --recursive
Turn on recursive retrieving.

-R rejlist, --reject=rejlist
Specify a comma-separated list of filename suffixes or patterns to reject.

--random-wait
Set a random wait time to prevent being identified by web sites that look for patterns in time between requests so they can block access.

--read-timeout=seconds
Set the read (and write) timeout to the specified number of seconds. The default is 900 seconds.

--referer=url
Include a "Referer: url" header in an HTTP request.

--restrict-file-names=mode[,nocontrol]
Restrict the characters found in remote URLs from appearing in local filenames. The value of mode is the operating system—e.g., unix or windows (use unix for Linux). Such characters are escaped with a percent sign (%). The default is to escape characters not valid on your operating system. Appending ,nocontrol turns off escaping of control characters.

--retr-symlinks
When retrieving FTP directories recursively, follow symbolic links and retrieve the linked-to files.

-S, --server-response
Print HTTP server headers and FTP server responses.

--save-cookies=file
Save cookies in the specified file before exiting. Does not save expired cookies, and only saves session cookies if --keep-session-cookies is also specified.

--save-headers
Save the headers sent by an HTTP server to the file, preceding the contents and separated by a blank line.

--spider
Behave like a web spider, checking that pages exist but not downloading them.

--strict-comments
Turn on strict parsing of HTML comments, instead of terminating comments at the first occurrence of -->.

-t num, --tries=num
Set the number of retries to the specified value of num. Set num to 0 or inf to keep trying forever (infinitely) (default is 20 retries), unless there is a fatal error such as "connection refused."

-T seconds, --timeout=seconds
Set network timeout to the specified number of seconds. Equivalent to specifying all of --dns-timeout,--connect-timeout, and --read-timeout.

-U agent,--user-agent=agent
Specify an agent string to the HTTP server to replace the default identification of Wget/version, where version is the current wget version. This string is used in the User-Agent header field.

-v, --verbose
Turn on verbose output, printing all available data. This is the default.

-V, --version
Display version information and exit.

-w seconds, --wait=seconds
Specify the wait in seconds between retrievals. Used to lighten server load. Use the suffix m to specify the wait in minutes, h for hours, or d for days.

--waitretry=seconds
Specify the number of seconds to wait between retries if the download fails. The default in the global configuration file is to not wait.

-x, --force-directories
Create a hierarchy of directories even if one wouldn't otherwise be created.

-X list, --exclude-directories=list
Specify a comma-separated list of directories to exclude from download. List elements may contain wildcards.

-Y on|off, --proxy=on|off
Turn proxy support on or off (default is on).

Linux Commands 9/10/2017 08:21:00 pm