| Age | Commit message (Collapse) | Author |
|---|
|
|
|
each .c file, and out of the .h files. To avoid overinclude.
ok gilles, in principle. If this has been done right, -portable should
become easier to maintain.
|
|
ok gilles@
|
|
|
|
process managing message reception, delivery and transfer. Mostly
mechanical, but very intrusive as it required to rewamp all IMSG to
fix ambiguities.
with and ok gilles@
|
|
|
|
- get rid of the whole penalty thing for failed envelopes in the mta and scheduler.
- do not disable routes on smtp errors
- try to schedule all types of envelopes on each scheduler frame.
|
|
|
|
Simplify code and do not impose an order on conditions and rule options.
Format changes that may require smtpd.conf update for some setups:
- SSL certificates are no longer automatically loaded, but must be
explicitely declared using the "pki" keyword.
- "certificate" option becomes "pki" in listener and accept rules.
- "ssl://" becomes "secure://" in relay via rules.
- "helo" becomes "hostnames" in relay rules
New features:
- accept rules do not need an explicit action, in which case alias table
or .forward must provide one.
- new "forward-only" action to force relaying and reject rcpts that expand
as local delivery.
- "!" (negation) modifier on rule matching conditions.
- new "recipient" rule matching condition.
- new "verify" option on listeners and relay rules to reject invalid
certificates.
Other changes:
- remember the helo name advertised on incoming mail and use it for sending
bounces.
- bump envelope version (existing envelopes are updated on-the-fly).
|
|
|
|
- implement suspend/resume scheduling for individual envelopes or message,
with the associated smtpctl commands.
- allow the mta to request immediate scheduling of an envelope.
- on temporary failures a penalty can be given to further delay the next try.
|
|
ok gilles@
|
|
consistency and clarity. Remove useless and confusing extra byte in
a few arrays based on this define.
ok gilles@
|
|
* first bricks of ldap and sqlite support (not finished but both working)
* new table API to replace map API, all lookups are done through tables
* improved handling of temporary errors throughout the daemon
* improved scheduler and mta logic: connection reuse, optimizes batches
* improved queue: more tolerant to admin errors, new layout, less disk-IO
* improved memory usage under high load
* SSL certs/keys isolated to lookup process to avoid facing network
* VIRTUAL support improved, fully virtual setups possible now
* runtime tracing of processes through smtpctl trace
* ssl_privsep.c sync-ed with relayd
* ssl.c no longer contains smtpd specific interfaces
* smtpd-specific ssl bits moved to ssl_smtpd.c
* update mail address in copyright
FLUSH YOUR QUEUE. FLUSH YOUR QUEUE. FLUSH YOUR QUEUE. FLUSH YOUR QUEUE.
smtpd.conf(5) simplified, it will require adaptations
ok eric@
|
|
ok gilles@
|
|
* Log more events (especially client session) and use a better scheme
for that: each messages is prefixed with a token to easily identify
its class:
- info/warn/debug: general server messages
- smtp-in: smtp client connections
- relay: status update for relayed messages
- delivery: status update for local deliveries
* Implement "smtpctl monitor" to display updates of selected internal
counters.
* When reloading the on-disk queue at startup do not commit a message
if no envelope was submitted for that message.
* Remove unused stuff in the config parser.
ok gilles@
|
|
introduce iobuf_xinit() and iobuf_xfqueue(). (idea from eric@)
ok gilles@
|
|
from eric@ input
ok gilles@
|
|
ok gilles@ eric@
|
|
ok eric@ gilles@
|
|
committed, a bounce is put on a runnable list of bounces. This list
is drained to enqueue as much bounces as possible within the limit.
This avoids DoS'ing the server when lots of bounces are enqueued at
startup.
While there, allow new envelopes to be added to a bounce until the
the very last moment (i.e. when the list of recipients is written).
ok gilles@ chl@
|
|
to be grouped into a single bounce message.
The bounce structure keeps a list of envelopes. For now, the list
is constructed by delaying the re-enqueuing of a bounce envelope a
bit, to wait for other bounces from the same message to be part of
the same report.
|
|
|
|
queue backend within the queue process.
The scheduler sends envelope ids to the queue process which loads
the envelope and forward the request to the agent responsible for
the delivery. The result is sent by the agent to the queue which
updates the storage before notifying the scheduler.
Bounces are created and enqueued (from the client side) by the
queue process, rather than the scheduler.
ok gilles@
|
|
New envelopes are pushed into the scheduler through the insert()
commit() rollback() transactional interface functions.
Worklists are pulled from the scheduler through a single batch()
interface function, which returns a list of envelope ids and the
type of processing. Envelopes returned in this batch are said to
be "in-flight", as opposed to "pending". They are supposed to be
processed in some way, and either updated() or deleted() at some
point.
The schedule()/remove() functions are used to alter the internal
state of "pending" envelopes to make them schedulable. The enve-
lopes will be part of a worklist on the next call to batch().
Rewrite the scheduler_ramqueue backend.
The initial queue loading in now done by the queue.
ok gilles@
|
|
longer have them and runner is actually a scheduler so rename.
- introduce scheduler_backend which does the same to scheduler than
queue_backend does to queue and map_backend does to maps
- remove all occurences of RUNNER and runner, replace them with SCHEDULER
and scheduler
ok eric@, ok chl@
|
|
internally in fsqueue backend for now, and let the fsqueue_message()
and fsqueue_envelope() dispatchers do the right thing.
Based on a diff by chl@
ok chl@ gilles@
|
|
a better separation between io and protocol logic. As a side-effect,
it fixes a couple of long-standing issues in the io path, and
hopefully add fresh ones instead. Kill client.c in the process.
ok gilles@
|
|
notify the runner of what happened with an envelope that has been
scheduled. It is not part of the state of the envelope, and it is not
even dumped. So it should only be set by mta/mda, checked by runner
to decide what to do with the envelope, and ignored everywhere else.
ok gilles@
|
|
being only called by bounce sessions, so most of the code there was
actually useless. The envelope is directly deleted or updated at the
relevant place.
ok gilles@
|
|
directly as an envelope of the bounced message, just like "regular"
envelopes.
ok gilles@
|
|
where it belongs.
ok gilles@
|
|
{u_,}int64_t or time_t
While there, cast some time_t to int64_t
These will fix build warnings for portable smptd
ok gilles@ eric@
|
|
couple of load/dump functions to convert to and from a human readable fmt.
while at it kill struct delivery and merge back its fields to the envelope.
this basically means we shouldn't require users to flush their queues every
time we make a change to struct envelope.
work is not done, but we're at a better state than the binary fsqueue so
we'll improve it in-tree.
has been running on my own box for the last 12 hours or so
ok eric@, chl@
|
|
simpler and hopefully saner way.
ok gilles@ chl@
|
|
bring a shitload of unnecessary information everywhere. this required many
parts of smtpd to be refactored and more specifically envelope expansion.
in the process lots of code got simplified, and the envelope expansion code
has been isolated to lka_session.c with some longstanding bugs fixed.
Diff has been tested by many with no major regression reported.
armani@ spotted a bug in a setup where a domain is listed a both primary
and virtual, I will fix that in-tree as it's becoming painful to maintain
this diff out.
|
|
discussed with and ok gilles@
|
|
struct message ...
|
|
smtpd now has an evpid associated to each delivery message, the evpid is an
u_int64_t where the upper 32 bits are the msgid, and the 32 bits are the
envelope unique identifier for that message. this results in lots of space
saved in both disk-based and ram-based queues, but also simplifies a lot of
code.
change has been stressed on my desktop, and has ran on my MX for the entire
afternoon without a regression.
|
|
obtain a read{-only,/write} descriptor to the message file.
make sure smtpd uses the new API everywhere it needs a fd, and kill the
many functions that were used until now.
|
|
- fsqueue->setup() performs the queue initialization;
- fsqueue->message() controls messages;
- fsqueue->envelope() controls envelopes;
This commit brings the following to fsbackend:
fsqueue_setup(), fsqueue_message_delete(), fsqueue_envelope_load(),
fsqueue_envelope_update(), fsqueue_envelope_delete().
It also makes smtpd use the queue_backend API for these operations.
|
|
fd. this shifts responsibility for the fclose to the caller, prevents a
memory leak and makes everyone happy.
diff by Jared Yanovich, thanks !
|
|
|
|
|
|
|
|
|
|
|
|
Major goals:
1) Fix bad performance caused by the runner process doing full queue
read in 1s intervals. My Soekris can now happily accept >50 msg/s
while having multi-thousand queue; before, one hundred queue would
bring the system to its knees.
2) Introduce Qmail-like scheduler that doesn't write as much to the
disk so that it needs less code for servicing error conditions,
which in some places can be tricky to get right.
3) Introduce separation between the scheduler and the backend; these
two queue aspects shouldn't be too tied too each other. This means
that eg. storing queue in SQL requires rewrite of just queue_backend.c.
4) Make on-disk queue format architecture independent, and more
easily extensible, to reduce number of flag days in the future.
Minor goals:
ENOSPC no longer prevents delivery attempts, fixed session limiting
for relayed mail, improved batching of "relay via" mails, human-readable
mailq output, "show queue raw" command, clearer logging, sending
of single bounce about multiple recipients, exact delay= computation,
zero delay between deliveries while within session limit (currently
1s delay between re-scheduling is enforced), mta no longer requests
content fd, corrected session limit for bounce submissions, tiny
<100B queue files instead of multi-KB, detect loops before accepting
mail, reduce traffic on imsg channels by killing enormous struct
submit_status.
|
|
|
|
of forwarding it via queue.
|
