Scraping the web is fraught with peril. URLs die; networks get disrupted
and best laid plans for building a corups from links can quickly go awry.
Use this funtion to mitigate some of the pain of retrieving web resoures.
safePOST(url = NULL, config = list(), timeout = httr::timeout(5),
..., body = NULL, encode = c("multipart", "form", "json", "raw"),
handle = NULL)
the url of the page to retrieve
Additional configuration settings such as http
authenticate), additional headers
add_headers), cookies (
config for full details and list of helpers.
a call to
httr::timeout(). Default timeout is
Further named parameters, such as
passed on to
modify_url. Unnamed parameters will be combined
One of the following:
FALSE: No body. This is typically not used with
PATCH, but can be useful if you need to send a
bodyless request (like
NULL: An empty body
"": A length 0 body
upload_file("path/"): The contents of a file. The mime
type will be guessed from the extension, or can be supplied explicitly
as the second argument to
A character or raw vector: sent as is in body. Use
content_type to tell the server what sort of data
you are sending.
A named list: See details for encode.
If the body is a named list, how should it be encoded? Can be
one of form (application/x-www-form-urlencoded), multipart,
(multipart/form-data), or json (application/json).
For "multipart", list elements can be strings or objects created by
upload_file. For "form", elements are coerced to strings
and escaped, use
I() to prevent double-escaping. For "json",
parameters are automatically "unboxed" (i.e. length 1 vectors are
converted to scalars). To preserve a length 1 vector as a vector,
I(). For "raw", either a character or raw vector. You'll
need to make sure to set the
The handle to use with this request. If not
supplied, will be retrieved and reused from the
based on the scheme, hostname and port of the url. By default httr
requests to the same scheme/host/port combo. This substantially reduces
connection time, and ensures that cookies are maintained over multiple
requests to the same host. See
handle_pool for more
This is a thin wrapper for
purrr::safely() that will
either return a
response object or
NULL if there was an error.
If you need the reason for the error (e.g.
Could not resolve host...)
you should write your own wrapper.