| Age | Commit message (Collapse) | Author |
|---|
|
couple of load/dump functions to convert to and from a human readable fmt.
while at it kill struct delivery and merge back its fields to the envelope.
this basically means we shouldn't require users to flush their queues every
time we make a change to struct envelope.
work is not done, but we're at a better state than the binary fsqueue so
we'll improve it in-tree.
has been running on my own box for the last 12 hours or so
ok eric@, chl@
|
|
ok gilles@ chl@
|
|
simpler and hopefully saner way.
ok gilles@ chl@
|
|
ok gilles@
|
|
bring a shitload of unnecessary information everywhere. this required many
parts of smtpd to be refactored and more specifically envelope expansion.
in the process lots of code got simplified, and the envelope expansion code
has been isolated to lka_session.c with some longstanding bugs fixed.
Diff has been tested by many with no major regression reported.
armani@ spotted a bug in a setup where a domain is listed a both primary
and virtual, I will fix that in-tree as it's becoming painful to maintain
this diff out.
|
|
discussed with and ok gilles@
|
|
no functionnal change
|
|
struct message ...
|
|
|
|
smtpd now has an evpid associated to each delivery message, the evpid is an
u_int64_t where the upper 32 bits are the msgid, and the 32 bits are the
envelope unique identifier for that message. this results in lots of space
saved in both disk-based and ram-based queues, but also simplifies a lot of
code.
change has been stressed on my desktop, and has ran on my MX for the entire
afternoon without a regression.
|
|
fsqueue_envelope_create(), fsqueue_message_purge()
- kill deprecated functions in queue_shared.c
At this point fsqueue backend is almost complete, all that is left to do is
to move the qwalk() API inside the queue_backend API, then make sure smtpd
is no longer calling anything queue related directly.
|
|
- change a few prototypes to allow bounce messages to use the
queue_backend API until it gets merged in
- kill functions of the queue API that have been deprecated
|
|
obtain a read{-only,/write} descriptor to the message file.
make sure smtpd uses the new API everywhere it needs a fd, and kill the
many functions that were used until now.
|
|
a continuous walk on the disk-queue. the implementation differs from what
jacekm@ commited (and I backed out) a while ago in that it uses a queue and
a host tree required for upcoming features.
code will be improved in tree, it requires changes to be done in queue and
bounce API, I just wanted to commit a working version first ...
tested by todd@ and I
|
|
|
|
|
|
|
|
is way more optimized than what we had earlier and there's definitely stuff
we want to keep, however it is early optimization that doesn't account for
many features and makes them hard (if not impossible) to write without
ugly workarounds that ruin the purpose of the optimizations.
the backout goes to 30 May's right before the commit and catches up on all
the non-queue related commits that happened since then.
i'll work on reintroducing the ideas from this queue when the basic
features we expect from a MTA are implemented.
suggested on tech@ about a week ago, no objections, several "please make
smtpd move forward" mails from hackers and tech readers.
|
|
attempt to expand the local delivery buffer when relaying mail, it was
kind of ok before but no longer is)
- use the same buffer for local deliveries to files and commands
tested by jmc@ and I
|
|
can be a normal situation and will allow a local/authenticated user to
trigger the fatal on purpose ...
ignore permanently failed bounce deliveries since there's not much smtpd
can do anyway, that's what the previous queue code did.
experienced and reported by pirofti@
|
|
with help from jacekm@
ok gilles@ jacekm@
|
|
ok jacekm@
|
|
|
|
|
|
of the list of next items to try, or near the beginning if the schedule
contains expired mail, which is of highest priority.
|
|
and lasttry args to queue_retry would make it return birth leading
looped scheduling, but that's not true so drop the "+ 1".
|
|
Major goals:
1) Fix bad performance caused by the runner process doing full queue
read in 1s intervals. My Soekris can now happily accept >50 msg/s
while having multi-thousand queue; before, one hundred queue would
bring the system to its knees.
2) Introduce Qmail-like scheduler that doesn't write as much to the
disk so that it needs less code for servicing error conditions,
which in some places can be tricky to get right.
3) Introduce separation between the scheduler and the backend; these
two queue aspects shouldn't be too tied too each other. This means
that eg. storing queue in SQL requires rewrite of just queue_backend.c.
4) Make on-disk queue format architecture independent, and more
easily extensible, to reduce number of flag days in the future.
Minor goals:
ENOSPC no longer prevents delivery attempts, fixed session limiting
for relayed mail, improved batching of "relay via" mails, human-readable
mailq output, "show queue raw" command, clearer logging, sending
of single bounce about multiple recipients, exact delay= computation,
zero delay between deliveries while within session limit (currently
1s delay between re-scheduling is enforced), mta no longer requests
content fd, corrected session limit for bounce submissions, tiny
<100B queue files instead of multi-KB, detect loops before accepting
mail, reduce traffic on imsg channels by killing enormous struct
submit_status.
|
|
- kill noisy log_debug
|
|
channels to parent, mda, mta, lka, smtp, and control. This leaves
just the channel to queue, which forwards imsgs on runner's behalf
and redirects any replies back to it.
OK gilles@
|
|
|
|
|
|
logging on runtime.
Based on claudio@'s work on ripd, ospfd, ospf6d, dvmrpd, ldpd, bgpd.
With help/ideas/testing from gilles@ jacekm@ todd@
ok jacekm@
|
|
to get upset by custom soft/hard ulimit settings.
Suggested by todd@
|
|
|
|
possibility for fd-starvation fatal when under heavy load.
|
|
doesn't take advantage of the new structure yet, but this was a needed
change for upcoming improvements.
- introduce aliasestree_{lookup,insert,remove} to the aliases api
- rename queue_generate_id() to generate_uid() and move it to utils.c as
it is used all over the place and not only in queue
tree idea discussed with jacekm@, if you update rebuild aliases db, make
clean and flush queue
|
|
from Tim van der Molen <tbvdm@xs4all.nl>
|
|
aliases/virtual domains resolution code.
- fix a logic bug which caused virtual domains not to be correctly
handled after one iteration of the aliases resolution code.
- introduce a few helper functions to help clean up and simplify the
lka code.
- simplify the IS_EXT/IS_MAILBOX/IS_RELAY macros so they manipulate a
struct path * instead of the mess of dereferences we were passing them.
|
|
process to relay mails. ok gilles@
|
|
|
|
Stop pushing event handling in the imsg framework.
Instead, provide a small glue layer on top of both imsg and libevent.
This finally clearly separates event handling and imsg construction.
Sidetrack bonus: remove the mega-ugly hack of having a dummy imsg_event_add
stub in smtpctl.
ok jaceckm@
|
|
ok gilles@, jacekm@
|
|
to the routing daemons by claudio@; ok gilles@
|
|
using newly introduced child struct.
Manage process titles centrally in struct smtpd.
ok gilles@
|
|
process individually whenever stats need to be fetched, keep stats
in MAP_ANON shared memory allocated by parent.
This means control has direct access to stats and can respond very
quickly without troubling any other daemon processes.
ok gilles@
|
|
|
|
ok gilles@, jacekm@
|
|
ok gilles@ jacekm@
|
|
|
|
and -F cmdline args. Also, date and Message-Id headers are added
when missing.
The main trouble with the current enqueue code is that it requires
dealing with problems in the control process that are already solved
in the smtp process, ie. duplicating a lot of code which interacts
with untrusted clients. This diff solves this by making sendmail
obtain a SMTP socket from smtp via smtpd.sock, and using that socket
to deliver the message. For smtpd it looks as if connection was
made from the network, only difference being the F_MESSAGE_ENQUEUED
message flag, handy when differentation between local and remote
deliveries is wanted.
Most of the features come from the femail program, created by henning@.
Additional testing by Nigel J. Taylor.
ok gilles@, henning@ happy with smtpd using femail code
|
