| Age | Commit message (Collapse) | Author |
|---|
|
is way more optimized than what we had earlier and there's definitely stuff
we want to keep, however it is early optimization that doesn't account for
many features and makes them hard (if not impossible) to write without
ugly workarounds that ruin the purpose of the optimizations.
the backout goes to 30 May's right before the commit and catches up on all
the non-queue related commits that happened since then.
i'll work on reintroducing the ideas from this queue when the basic
features we expect from a MTA are implemented.
suggested on tech@ about a week ago, no objections, several "please make
smtpd move forward" mails from hackers and tech readers.
|
|
ok jacekm@
|
|
|
|
|
|
|
|
Major goals:
1) Fix bad performance caused by the runner process doing full queue
read in 1s intervals. My Soekris can now happily accept >50 msg/s
while having multi-thousand queue; before, one hundred queue would
bring the system to its knees.
2) Introduce Qmail-like scheduler that doesn't write as much to the
disk so that it needs less code for servicing error conditions,
which in some places can be tricky to get right.
3) Introduce separation between the scheduler and the backend; these
two queue aspects shouldn't be too tied too each other. This means
that eg. storing queue in SQL requires rewrite of just queue_backend.c.
4) Make on-disk queue format architecture independent, and more
easily extensible, to reduce number of flag days in the future.
Minor goals:
ENOSPC no longer prevents delivery attempts, fixed session limiting
for relayed mail, improved batching of "relay via" mails, human-readable
mailq output, "show queue raw" command, clearer logging, sending
of single bounce about multiple recipients, exact delay= computation,
zero delay between deliveries while within session limit (currently
1s delay between re-scheduling is enforced), mta no longer requests
content fd, corrected session limit for bounce submissions, tiny
<100B queue files instead of multi-KB, detect loops before accepting
mail, reduce traffic on imsg channels by killing enormous struct
submit_status.
|
|
channels to parent, mda, mta, lka, smtp, and control. This leaves
just the channel to queue, which forwards imsgs on runner's behalf
and redirects any replies back to it.
OK gilles@
|
|
|
|
|
|
logging on runtime.
Based on claudio@'s work on ripd, ospfd, ospf6d, dvmrpd, ldpd, bgpd.
With help/ideas/testing from gilles@ jacekm@ todd@
ok jacekm@
|
|
This restructures the client_* API internals significantly. The code becomes
pipelining in nature. All SMTP commands are put on the output queue and
dequeued as quickly as possible. Once dequeued, they're moved to the receive
queue so that replies can be matched with previous commands.
Dequeuing commands from the output queue halts when the count of commands
currently in-pipeline (``cmdi'') is equal to the command send window (``cmdw'').
There are three cmdw values useful in practice:
0 clear pipeline, ie. inhibit all future sends
1 disable pipelining, ie. use old ``one-request-one-reply`` mode
SIZE_T_MAX enable pipelining, ie. dequeue as many commands as possible
At the beginning of session cmdw is 1. When it is found that peer supports
PIPELINING, it grows to SIZE_T_MAX. After dequeing DATA it is again 1. After
sending QUIT it is 0.
Each command dequeued from the output queue becomes a buf in a msgbuf. The act
of combining multiple commands into a single send operation did not need to be
implemented: buf_write() already combines bufs using iovec and sends them at
once using sendmsg(2).
Tested by todd@ and oga@
"looks good" to gilles@
|
|
|
|
as the remote accepts more data instead of doing one big read into
the memory in the beginning of session.
|
|
fatals instead of passing the error up (kills ~300 lines).
Implement sending of the QUIT command which replaces crude close(2).
tested by gilles@, todd@
|
|
Reported by jsing@
|
|
ok gilles@
|
|
entry in order to avoid the risk of sending duplicate mail.
|
|
logging of successful deliveries until all MXs were tried, plus add logging
of 5yz replies.
tested by todd@, "reads ok" gilles@
|
|
ok jacekm@
|
|
ok gilles@ jacekm@
|
|
The 1yz and 6yz status codes are now removed prior to reporting the status
message in bounce messages, which provides an easy way to distinguish
between local and remote status messages. Initial diff from jacekm@
ok gilles@ jacekm@
|
|
process to relay mails. ok gilles@
|
|
Committing on behalf of gilles@ who doesn't have access to a safe box.
|
|
|
|
|
|
bounce as soon as it hits mx2 in mx1 -> mx2 -> mx3 relay chain.
hint by sthen@, ok gilles@
|
|
- make sure queue_message_update() creates bounces using bounce_record()
- when mta sends update to queue and it sees that batch is flagged with
F_BATCH_PERMFAIL, only update the envelope error message if it doesn't
have F_MESSAGE_PERMFAIL set, otherwise we may lose the real reason why
we failed for that recipient. There's cleanup to do around that old
code, i'm sure we can get it simpler.
this commit fixes a bug pea@ spotted where a bounce message would not
display the reason of a failure when we generated it after failing to
deliver a relayed message.
|
|
|
|
|
|
idea by jacekm@ a few weeks ago, discussed with pyr
|
|
appear in a bounce message, and message_get_errormsg() to retrieve that
message.
- when loop is detected, call message_set_errormsg()
- in mta, call message_set_errormsg() for each recipient failure
- in mta, call message_set_errormsg() to copy batch errors to recipients if
we failed to deliver for a session related error
- when bouncing, add the recipient and error reason to the bounce message
|
|
work for real. As an added bonus, it simplifies it, makes it follow the
same code path as regular messages and kills quite some code from mta,
mda and store. There's still some work needed but the most painful part
is behind us now ;)
ok jacekm@
|
|
From: Josh Elsasser <josh@elsasser.org>
|
|
Currently the receiver fetches an imsg via imsg_get() and if he expects
an fd, he then calls imsg_get_fd() to fetch the next fd queued on the
imsgbuf from which the imsg came.
This changes hides the fd queueing mechanism to the API user. When closing
an imsg with an fd, the message is flagged so that the receiving end knows
it must dequeue the fd in imsg_get() and return it with the imsg structure.
This way there is no (less) possible screw up from imsg_get_fd() not being
called directly after imsg_get() by the user. The retreived imsg is
self-contained.
ok pyr@, "I like that" henning@
|
|
Stop pushing event handling in the imsg framework.
Instead, provide a small glue layer on top of both imsg and libevent.
This finally clearly separates event handling and imsg construction.
Sidetrack bonus: remove the mega-ugly hack of having a dummy imsg_event_add
stub in smtpctl.
ok jaceckm@
|
|
ok gilles@, jacekm@
|
|
|
|
configuration reloading without killing active sessions; ok jacekm@
|
|
to the routing daemons by claudio@; ok gilles@
|
|
when connect timesout and we close the fd, reset session fd to -1
in session_destroy(), only attempt to close session fd if != -1
fixes a fatal in session_destroy() which happened because we closed a fd
after a timeout, but the session still assumed the fd to be opened.
|
|
host which requests client certificates:
accept [...] relay via [...] ssl certificate "mycert"
diff from Josh Elsasser <josh@elsasser.org>, tested and okayed by me with
no change but the addition of status 554 to the state machine to deal with
remote host telling us it doesn't like our certificate.
|
|
|
|
- reorder alphabetically and add mta to the statistics
|
|
|
|
using newly introduced child struct.
Manage process titles centrally in struct smtpd.
ok gilles@
|
|
|
|
ok gilles@, jacekm@
|
|
|
|
- Improve RFC compliance: CNAMEs are resolved, equal preference MXs
are randomized, relaying via MX that has equal/lower preference
than local server is prevented, decision on when to treat domain
name as implicit MX is better.
ok gilles@
|
|
we need to explicitely set success and permanent failures, this way if we
hit any condition that we don't explicitely flag as permanent failure the
delivery will be rescheduled later.
while at it, made various checks more strict to prevent hiding bugs and a
bit of knr cleanups
discussed with and "reads good" jacekm@
|
