--keep-session-cookies
When specified, causes --save-cookies to also save session cookies. Session cookies are normally not
saved because they are meant to be kept in memory and forgotten when you exit the browser. Saving
them is useful on sites that require you to log in or to visit the home page before you can access
some pages. With this option, multiple Wget runs are considered a single browser session as far as
the site is concerned.
Since the cookie file format does not normally carry session cookies, Wget marks them with an expiry
timestamp of 0. Wget's --load-cookies recognizes those as session cookies, but it might confuse
other browsers. Also note that cookies so loaded will be treated as other session cookies, which
means that if you want --save-cookies to preserve them again, you must use --keep-session-cookies
again.
|
--post-data=string
--post-file=file
Use POST as the method for all HTTP requests and send the specified data in the request body.
--post-data sends string as data, whereas --post-file sends the contents of file. Other than that,
they work in exactly the same way. In particular, they both expect content of the form
"key1=value1&key2=value2", with percent-encoding for special characters; the only difference is that
one expects its content as a command-line parameter and the other accepts its content from a file. In
particular, --post-file is not for transmitting files as form attachments: those must appear as
"key=value" data (with appropriate percent-coding) just like everything else. Wget does not currently
support "multipart/form-data" for transmitting POST data; only "application/x-www-form-urlencoded".
Only one of --post-data and --post-file should be specified.
Please be aware that Wget needs to know the size of the POST data in advance. Therefore the argument
to "--post-file" must be a regular file; specifying a FIFO or something like /dev/stdin won't work.
It's not quite clear how to work around this limitation inherent in HTTP/1.0. Although HTTP/1.1
introduces chunked transfer that doesn't require knowing the request length in advance, a client
can't use chunked unless it knows it's talking to an HTTP/1.1 server. And it can't know that until
it receives a response, which in turn requires the request to have been completed -- a chicken-and-
egg problem.
Note: if Wget is redirected after the POST request is completed, it will not send the POST data to
the redirected URL. This is because URLs that process POST often respond with a redirection to a
regular page, which does not desire or accept POST. It is not completely clear that this behavior is
optimal; if it doesn't work out, it might be changed in the future.
This example shows how to log to a server using POST and then proceed to download the desired pages,
presumably only accessible to authorized users:
# Log in to the server. This can be done only once.
wget --save-cookies cookies.txt \
--post-data 'user=foo&password=bar' \
http://server.com/auth.php
# Now grab the page or pages we care about.
wget --load-cookies cookies.txt \
-p http://server.com/interesting/article.php
If the server is using session cookies to track user authentication, the above will not work because
--save-cookies will not save them (and neither will browsers) and the cookies.txt file will be empty.
In that case use --keep-session-cookies along with --save-cookies to force saving of session cookies.
|
--delete-after
This option tells Wget to delete every single file it downloads, after having done so. It is useful
for pre-fetching popular pages through a proxy, e.g.:
wget -r -nd --delete-after http://whatever.com/~popular/page/
The -r option is to retrieve recursively, and -nd to not create directories.
Note that --delete-after deletes files on the local machine. It does not issue the DELE command to
remote FTP sites, for instance. Also note that when --delete-after is specified, --convert-links is
ignored, so .orig files are simply not created in the first place.
|