| Age | Commit message (Collapse) | Author |
|---|
|
and confusing.
ok gilles@
|
|
with help and ideas from eric@
ok eric@ gilles@
|
|
- rename IMSG and smtpctl pause/resume parameters
- update man page
tested by me, ok chl@, eric@
|
|
ok gilles@ chl@
|
|
simpler and hopefully saner way.
ok gilles@ chl@
|
|
- bring back 'smtpctl schedule' and 'smtpctl remove' to life
Things you should know:
The ramqueue data structure is not finished yet and lacks an envelope tree
for evpid lookups. I wanted to wait until I'm done but too many people are
affected by not being able to reschedule envelopes, this is a quick fix.
So right now there's an O(rrible) complexity as both commands will perform
a (possibly aborted) queue scan leading to O(n). I will make that O(log n)
soon.
Also, smtpctl remove no longer supports removing an entire message, I will
fix that very soon too.
|
|
discussed with and ok gilles@
|
|
a continuous walk on the disk-queue. the implementation differs from what
jacekm@ commited (and I backed out) a while ago in that it uses a queue and
a host tree required for upcoming features.
code will be improved in tree, it requires changes to be done in queue and
bounce API, I just wanted to commit a working version first ...
tested by todd@ and I
|
|
|
|
"ok and no need to keep them for yourself" gilles@
|
|
is way more optimized than what we had earlier and there's definitely stuff
we want to keep, however it is early optimization that doesn't account for
many features and makes them hard (if not impossible) to write without
ugly workarounds that ruin the purpose of the optimizations.
the backout goes to 30 May's right before the commit and catches up on all
the non-queue related commits that happened since then.
i'll work on reintroducing the ideas from this queue when the basic
features we expect from a MTA are implemented.
suggested on tech@ about a week ago, no objections, several "please make
smtpd move forward" mails from hackers and tech readers.
|
|
ok jacekm@
|
|
|
|
|
|
Major goals:
1) Fix bad performance caused by the runner process doing full queue
read in 1s intervals. My Soekris can now happily accept >50 msg/s
while having multi-thousand queue; before, one hundred queue would
bring the system to its knees.
2) Introduce Qmail-like scheduler that doesn't write as much to the
disk so that it needs less code for servicing error conditions,
which in some places can be tricky to get right.
3) Introduce separation between the scheduler and the backend; these
two queue aspects shouldn't be too tied too each other. This means
that eg. storing queue in SQL requires rewrite of just queue_backend.c.
4) Make on-disk queue format architecture independent, and more
easily extensible, to reduce number of flag days in the future.
Minor goals:
ENOSPC no longer prevents delivery attempts, fixed session limiting
for relayed mail, improved batching of "relay via" mails, human-readable
mailq output, "show queue raw" command, clearer logging, sending
of single bounce about multiple recipients, exact delay= computation,
zero delay between deliveries while within session limit (currently
1s delay between re-scheduling is enforced), mta no longer requests
content fd, corrected session limit for bounce submissions, tiny
<100B queue files instead of multi-KB, detect loops before accepting
mail, reduce traffic on imsg channels by killing enormous struct
submit_status.
|
|
channels to parent, mda, mta, lka, smtp, and control. This leaves
just the channel to queue, which forwards imsgs on runner's behalf
and redirects any replies back to it.
OK gilles@
|
|
|
|
|
|
and assuming message is not in processing/scheduled state
- teach smtpctl how to request message removal from runner
discussed with todd@, idea ok jacekm@
|
|
logging on runtime.
Based on claudio@'s work on ripd, ospfd, ospf6d, dvmrpd, ldpd, bgpd.
With help/ideas/testing from gilles@ jacekm@ todd@
ok jacekm@
|
|
This restructures the client_* API internals significantly. The code becomes
pipelining in nature. All SMTP commands are put on the output queue and
dequeued as quickly as possible. Once dequeued, they're moved to the receive
queue so that replies can be matched with previous commands.
Dequeuing commands from the output queue halts when the count of commands
currently in-pipeline (``cmdi'') is equal to the command send window (``cmdw'').
There are three cmdw values useful in practice:
0 clear pipeline, ie. inhibit all future sends
1 disable pipelining, ie. use old ``one-request-one-reply`` mode
SIZE_T_MAX enable pipelining, ie. dequeue as many commands as possible
At the beginning of session cmdw is 1. When it is found that peer supports
PIPELINING, it grows to SIZE_T_MAX. After dequeing DATA it is again 1. After
sending QUIT it is 0.
Each command dequeued from the output queue becomes a buf in a msgbuf. The act
of combining multiple commands into a single send operation did not need to be
implemented: buf_write() already combines bufs using iovec and sends them at
once using sendmsg(2).
Tested by todd@ and oga@
"looks good" to gilles@
|
|
possibility for fd-starvation fatal when under heavy load.
|
|
`OK' claudio
|
|
ok gilles
|
|
remove the smtpd.sock file, without which /usr/sbin/sendmail will fail
causing mails to be queued in offline.
Initially reported by ian@, may explain oga@'s e-mail burst after smtpd
restart.
|
|
control process does not try to handle it.
spotted and reported by martijn@bunix.org
|
|
was not correctly formatted anyway. Recording of sender uid has
been moved to the server process (getpeereid), which means it is not
subject to forgery.
Based on problem report by Michael Lechtermann <michael@lechtermann.net>
|
|
process to relay mails. ok gilles@
|
|
|
|
Currently the receiver fetches an imsg via imsg_get() and if he expects
an fd, he then calls imsg_get_fd() to fetch the next fd queued on the
imsgbuf from which the imsg came.
This changes hides the fd queueing mechanism to the API user. When closing
an imsg with an fd, the message is flagged so that the receiving end knows
it must dequeue the fd in imsg_get() and return it with the imsg structure.
This way there is no (less) possible screw up from imsg_get_fd() not being
called directly after imsg_get() by the user. The retreived imsg is
self-contained.
ok pyr@, "I like that" henning@
|
|
Stop pushing event handling in the imsg framework.
Instead, provide a small glue layer on top of both imsg and libevent.
This finally clearly separates event handling and imsg construction.
Sidetrack bonus: remove the mega-ugly hack of having a dummy imsg_event_add
stub in smtpctl.
ok jaceckm@
|
|
ok gilles@, jacekm@
|
|
to the routing daemons by claudio@; ok gilles@
|
|
paused, configuring, or exiting; ok gilles@
|
|
using newly introduced child struct.
Manage process titles centrally in struct smtpd.
ok gilles@
|
|
process individually whenever stats need to be fetched, keep stats
in MAP_ANON shared memory allocated by parent.
This means control has direct access to stats and can respond very
quickly without troubling any other daemon processes.
ok gilles@
|
|
contains the missing parts + a memleak plug
|
|
the configuration file again and replace current configuration with new one
in all processes. what we don't support yet is graceful restart, clients in
sessions at the moment of the reload will have a temp failure thrown at 'em
which is ok RFC-wise but which we will try to improve anyway.
tested with various setups, "diff reads good" jacekm@
|
|
|
|
ok gilles@, jacekm@
|
|
|
|
ok gilles@ jacekm@
|
|
and -F cmdline args. Also, date and Message-Id headers are added
when missing.
The main trouble with the current enqueue code is that it requires
dealing with problems in the control process that are already solved
in the smtp process, ie. duplicating a lot of code which interacts
with untrusted clients. This diff solves this by making sendmail
obtain a SMTP socket from smtp via smtpd.sock, and using that socket
to deliver the message. For smtpd it looks as if connection was
made from the network, only difference being the F_MESSAGE_ENQUEUED
message flag, handy when differentation between local and remote
deliveries is wanted.
Most of the features come from the femail program, created by henning@.
Additional testing by Nigel J. Taylor.
ok gilles@, henning@ happy with smtpd using femail code
|
|
|
|
|
|
writing a small app that sent out of order imsg's. prevent this by use
of a state machine and read event masking.
issue spotted by jacekm@, temporary fix by me. there are ideas around
this, but we want to experiment them a bit and they are low priority.
|
|
ok gilles@
|
|
spotted by Matthew Haub <matthew.haub@alumni.adelaide.edu.au>
|
|
a token so that it is possible to do: smtpctl schedule <message id/uid>
introduce F_MESSAGE_FORCESCHEDULE which lets the runner schedule a message
even if the retry delay has not been expired.
F_MESSAGE_ENQUEUED is a valid flag for a message and should not cause an\
errx() in smtpctl show queue
|
|
ok gilles@
|
