summaryrefslogtreecommitdiff
path: root/usr.bin/ftp/fetch.c
AgeCommit message (Collapse)Author
2024-04-23ftp: send Host: headers with CONNECT requests when tunneling TLS over anStuart Henderson
HTTP proxy (i.e. for fetching resources over https). This is required by some proxy servers. Ftom KUWAZAWA Takuya, ok tb@
2024-04-17ftp: send 'Accept */*' headerTheo Buehler
A recent update to filezilla showed a server that would refuse to let us download the distfile without us sending this header. Browsers, curl and wget do so, so it should be safe for us to follow suit. ok deraadt florian phessler sthen
2023-06-28drop needless strcspn in the header parsingOmar Polo
since fetch.c revision 1.211, ftp removes trailingwhitespaces early so there's no need to re-do that when parsing a header. while here, remove an unused variable too. ok tb, millert
2023-06-28fix parsing of the Last-Modified headerOmar Polo
Was overlooked in r1.209. diff from 'a dog' (OpenBSD [at] anthropomorphic [dot] dog) ok tb, sthen
2023-03-08Delete obsolete /* ARGSUSED */ lint comments.Philip Guenther
ok miod@ millert@
2022-12-26spelling fixes; from paul tagliamonteJason McIntyre
amendments to his diff are noted on tech
2022-11-09Before printing the redirect URI pass it through stravis since it isClaudio Jeker
untrusted input. OK tb@ kn@ millert@
2022-11-09Strip spaces at end of header lines and in chunked encoding headers.Claudio Jeker
HTTP standard allows for spaces in too many places OK millert@ tb@
2022-09-15Use non-blocking connect() with ppoll() and timeout instead of alarm().Todd C. Miller
For hosts with multiple IP addrs this makes it possible to fall over from an unresponsive IP to another. This also replaces the other connect(2) + connect_wait() calls with timed_connect() so the -w option now works for more that just http. OK sthen@ deraadt@
2022-09-08Adjust HTTP header parsing to follow RFC more closely.Claudio Jeker
RFC9112 allows any amount of space/tabs between the ':' and the value. Until now this code required exactly one space which works most of the time but is not RFC compliant. OK djm@
2021-11-10Revert previousKlemens Nanni
I overlooked the autoinstall case where "Requesting ..." is used, but those messages that got fixed where omitted in ftp's SMALL version. Noticed the hard way by anton
2021-11-09Print actually requested URLsKlemens Nanni
Encoding URL paths changes the requested URL and therefore may yield different responses (opposed to an unencoded URL), solely depending on how the server implements de/encoding. Always print the encoded URL which actually gets requested in output like "Requesting ..." and erors likes "Error retrieving ....: 404 Not Found" and don't use the original URL provided on the command line. This matches exactly what is seen on the wire, e.g. with tshark(1) and helps debugging URL de/encoding related (server) issues. Feedback OK sthen
2021-11-06Stop URL encoding the tilde characterKlemens Nanni
RFC 1738 Uniform Resource Locators (URL) lists tilde as unsafe character. RFC 2396 Uniform Resource Identifiers (URI): Generic Syntax updates it to The tilde "~" character was added to those in the "unreserved" set, since it is extensively used on the Internet in spite of the difficulty to transcribe it with some keyboards. In theory, this shouldn't make a difference, but some servers do not decode "%7e" and thus erroneously serve a 404. RFC 2396 2.4.2. When to Escape and Unescape says: In some cases, data that could be represented by an unreserved character may appear escaped; for example, some of the unreserved "mark" characters are automatically escaped by some systems. If the given URI scheme defines a canonicalization algorithm, then unreserved characters may be unescaped according to that algorithm. For example, "%7e" is sometimes used instead of "~" in an http URL path, but the two are equivalent for an http URL. Update ftp(1) to RFC 2396 by no longer treating "~" as unsafe character. This is effectively a one-character diff; update comments accordingly as well as the order of characters to ease code-to-standard comparison. This matches curl(1) and wget(1) behaviour wrt. encoding of "~". OK sthen
2021-08-31Spacing. OK tb@Claudio Jeker
2021-03-29in unsafe_char(), handle %NN with array-index inspection rather than weirdTheo de Raadt
ptr++ ok claudio
2021-03-13only try to set timestamps on files; avoids error with ftp -o /dev/nullStuart Henderson
ok jca robert
2021-02-25ftp: prevent double free() in error pathChristian Weisgerber
Reported by bentley@; ok bentley@ jca@
2021-02-16make use of getline(3) in ftp(1)Christian Weisgerber
Replace fparseln(3) with getline(3). This removes the only use of libutil.a(fparseln.o) from the ramdisk. Replace a complicated fgetln(3) idiom with the much simpler getline(3). ok jca@
2021-02-02introduce support for sending the If-Modified-Since header whileRobert Nagy
fetching over http(s) and use the timestamps from the remote server's Last-Modified header if available when saving local files this makes it possible to mirror files better with ftp(1) the new timestamp behaviour can be disabled with the new '-u' flag ok sthen@, input from sthen@ and gnezdo@
2021-01-01Handle Permanent Redirect (RFC 7538)Christopher Zimmermann
ok jca@, kn@
2020-10-18Accommodate POSIX basename(3) that takes a non-const parameter andChristian Weisgerber
may modify the string buffer. improved and ok jca@
2020-07-04Avoid malloc(3) calls in signal handlerkn
Fetch aborts through SIGINT (^C) print a message with fputs(3), but this calls malloc() on its own, which is not supported from interrupt handler context. Fix it by using write(2) which avoids further memory allocations. While here, merge abortfile() into the identical aborthttp() with a more generic "fetch aborted." message for simplicity. Spotted with vm.malloc_conf=SU and ^C on a port's "make fetch" causing ftp(49660) in malloc(): recursive call Abort trap (core dumped) OK jca (who came up with using write(2) independently)
2020-07-04Fix a double free in error pathsJeremie Courreges-Anglas
Consistently disarm the SIGINT handler on error, else a SIGINT can lead to taking twice the cleanup path. Initial report by naddy@, ok tb@
2020-06-20Avoid one uninitialized warning in file_get()Jeremie Courreges-Anglas
2020-02-22Avoid duplication in the code that sends headersJeremie Courreges-Anglas
ok yasuoka@
2020-02-22Respect userinfo even when built with NOSSL (floppy ramdisks)Jeremie Courreges-Anglas
Not handling it is incorrect and can lead to credentials leaks in DNS requests. The resulting growth is reasonable (about 300 bytes on amd64). ok yasuoka@
2020-02-20Backout revision 1.190 partially. That part was an unintended change whichYASUOKA Masahiko
is work in progress.
2020-02-19Fix http (not https) auth combined with proxy auth.Jeremie Courreges-Anglas
First look for userinfo, and overwrite it to make sure it doesn't reappears again later. Then reset the path to fix the fragile mechanism that produces the full request URI for the proxied connection case. ok yazuoka@
2020-02-19Don't put the userinfo in request URI. This also makes accessing aYASUOKA Masahiko
https server with user/password through "http_proxy" environment variable work properly. ok jca
2020-02-13Two longjmp(3) vs volatile fixes and improvementsJeremie Courreges-Anglas
- allocate read buffer before setjmp(3) so that its value is properly defined when longjmp(3) returns - only mark as volatile variables modified after setjmp(3) and used again after a possible return from longjmp(3)
2020-02-13Fixes and tweaks for read/write loop in url_get()Jeremie Courreges-Anglas
Changes already present in file_get() - no need to special case write(2) returning 0 - clearer loop condition - fix read error detection and properly save errno
2020-01-21Fix tls_handshake() usage which was added without checking return valuesBob Beck
correctly. This would break ftp when the handshake doesn't complete in one shot. (noticed when making tls 1.3 connections to cloudflare.cdn) ok jsing@
2020-01-15Move local file handling out of url_get()Jeremie Courreges-Anglas
The code is mostly duplicated already, handling local files here just makes for more complex code. Split it out to its own function. This mechanically prevents redirections to local files. Positive feedback from Hiltjo Posthuma
2019-12-19Prevent redirections to file: URLsJeremie Courreges-Anglas
Report and fix from Hiltjo Posthuma, input from and ok deraadt@
2019-12-09Reinstate ftp_printf to log sent HTTP headersJeremie Courreges-Anglas
On SMALL builds ftp_printf is just a #define to avoid a size increase. ok millert@
2019-12-09With NOSSL let url_get() print a nice error message for https urlsJeremie Courreges-Anglas
Input from deraadt@
2019-12-08For chunked transfers always restore the SIGINFO handler (not just on error)Jeremie Courreges-Anglas
Overlooked when shuffling the HTTP/1.1 code.
2019-12-05Fix #ifndef NOSSL vs SMALL inconsistenciesJeremie Courreges-Anglas
from Hiltjo Posthuma
2019-12-02Tweak inaccurate commentJeremie Courreges-Anglas
2019-11-18various knf and whitespace; ok jcaTheo de Raadt
2019-11-14HTTP/1.1 for ftp(1)Jeremie Courreges-Anglas
Some sites in ports start to reject HTTP/1.0 requests. Let's move on and implement HTTP/1.1. Should fit in ramdisks. ok sthen@ tb@
2019-11-04Apply more 'static' to help the compilerJeremie Courreges-Anglas
Results in better code and a size decrease.
2019-11-03Also use stdio for TLS connectionsJeremie Courreges-Anglas
Set up two wrappers around tls_read/write to be used along with the not-very-portable funopen(). This kills a bunch of local code, always a nice thing for an utility which ends up in bsd.rd. "seems legit" deraadt@, ok kn@
2019-10-23change some error reports to include the failing URL (which we vis, inTheo de Raadt
case it came via a redirect) some help from jca, discussed with aja
2019-10-13Also close the server connection before retrying on a 503.Jeremie Courreges-Anglas
Keeping it around uses both local and remote resources for no good reason. ok job@
2019-10-13Factor out socket cleanup codeJeremie Courreges-Anglas
As a side effect this shuts down the TLS connection before closing the underlying socket for redirectionss. ok job@
2019-10-09On a 503, only retry if "Retry-After: 0" is present.Jeremie Courreges-Anglas
We just bail out if the header is absent or if the server tells us to wait. Prodding from job@, ok sthen@ deraadt@
2019-10-05Retry request once when receiving a 503Jeremie Courreges-Anglas
Basic implementation: we just retry once, and make no attempt (yet) to parse any Retry-After header. The idea is to work around cdn.openbsd.org sometimes replying with a 503 for reasons unknown. According to juanfra@ it sets "Retry-After: 0" so this minimal implementation should be enough. Different diff from espie@, test case from sthen@, input from millert@, ok millert@ deraadt@
2019-06-28When system calls indicate an error they return -1, not some arbitraryTheo de Raadt
value < 0. errno is only updated in this case. Change all (most?) callers of syscalls to follow this better, and let's see if this strictness helps us in the future.
2019-05-16Revert suni'ls ftp rewrite for now.Florian Obser
We are juggling too many things at the moment and we can't deal with the differences in behaviour right now.
generated by cgit v1.2.3 (git 2.25.1) at 2024-12-03 11:57:08 +0000