| Age | Commit message (Collapse) | Author |
|---|
|
use pledge and file locking. OK deraadt@
|
|
|
|
ok deraadt@
|
|
each .c file, and out of the .h files. To avoid overinclude.
ok gilles, in principle. If this has been done right, -portable should
become easier to maintain.
|
|
rather than going through the parent process. simplify code in the meantime.
|
|
- Get rid of the scheduler_batch structure. The scheduler can now return
envelopes of different types in a single run, interlacing them to avoid
batch effects.
- Ask for an acknowledgement from the queue when removing or expiring
an envelope to benefit from the inflight envelope limitation mechanism.
This ensures that the scheduler always keeps sending envelopes at a rate
that the queue can sustain in all cases.
- Limit the number of envelopes in a holdq. When a holdq is full,
new envelopes are put back in the pending queue instead, with a
shorter retry time.
- Plumbing for proc-ified schedulers.
imsg version bump. smtpctl stop before updating.
ok gilles@
|
|
- add a "close" hook to the backend API.
- improve the sync() pattern in queue_fs: only sync at commit
time and not for every envelope creation
- various fixes to the experimental external queue API.
|
|
|
|
ok gilles@
|
|
process managing message reception, delivery and transfer. Mostly
mechanical, but very intrusive as it required to rewamp all IMSG to
fix ambiguities.
with and ok gilles@
|
|
|
|
- get rid of the whole penalty thing for failed envelopes in the mta and scheduler.
- do not disable routes on smtp errors
- try to schedule all types of envelopes on each scheduler frame.
|
|
|
|
tempfail for limiting the number of pending deliveries to the same
user. This allows to reach optimal delivery time even in case of
burst, while keeping the number of inflight envelopes low.
|
|
in the scheduler when it has too many tasks for a given relay. The
envelopes are put on a wait queue, and are not scheduled again until
the mta "releases" some envelopes from that queue.
It prevents from having too many inflight envelopes, which are out of reach
for the admin.
|
|
another smtpd instance is running. Close the inherited socket in
every forked process but control.
|
|
- Better transient error handling logic: failing destinations are
automatically disabled for a while. When a destination is active
again, ask the scheduler to retry previous envelopes immediatly.
- More informative error report when all routes fail for a mail.
- Implement a "smtpctl show hoststats" command to get the latest stat
message per MX domain.
- Implement a "smtpctl show routes" command to show the state the
currently known routes to remote MXs.
- Implement a "smtpctl resume route" command to re-enable a route that
has been disabled.
- Do not hardcode limits
- Minor code improvements
|
|
- cleanup the internal queue backend API and get rid of the QOP_* thing.
- implement a queue_proc backend
- rename queue_fsqueue.c to queue_fs
- enable support for queue encryption
- add an envelope cache
- better logging and error reporting
|
|
- implement suspend/resume scheduling for individual envelopes or message,
with the associated smtpctl commands.
- allow the mta to request immediate scheduling of an envelope.
- on temporary failures a penalty can be given to further delay the next try.
|
|
now happens in queue_init(), and backends take the queue passwd as
parameter in their init function.
Remove useless SMTPD_FILTER_USER while there.
|
|
|
|
ok gilles@
|
|
consistency and clarity. Remove useless and confusing extra byte in
a few arrays based on this define.
ok gilles@
|
|
in compressed form. reduce the default size for envelope messages.
ok gilles@
|
|
* first bricks of ldap and sqlite support (not finished but both working)
* new table API to replace map API, all lookups are done through tables
* improved handling of temporary errors throughout the daemon
* improved scheduler and mta logic: connection reuse, optimizes batches
* improved queue: more tolerant to admin errors, new layout, less disk-IO
* improved memory usage under high load
* SSL certs/keys isolated to lookup process to avoid facing network
* VIRTUAL support improved, fully virtual setups possible now
* runtime tracing of processes through smtpctl trace
* ssl_privsep.c sync-ed with relayd
* ssl.c no longer contains smtpd specific interfaces
* smtpd-specific ssl bits moved to ssl_smtpd.c
* update mail address in copyright
FLUSH YOUR QUEUE. FLUSH YOUR QUEUE. FLUSH YOUR QUEUE. FLUSH YOUR QUEUE.
smtpd.conf(5) simplified, it will require adaptations
ok eric@
|
|
a simple QOP_WALK queue operation. Some knf and formating fixes while
there.
ok gilles@
|
|
is running. The scheduler sends the runtime state of each envelope to
the queue process which loads the envelope, fills the runtime bits and
sends the envelope back to the client. Iteration over the envelope set
happens in small chunks to make the request interruptible and to allow
the server to keep doing its job in the meantime.
Adpat "smtpctl schedule-all" to schedule the messages one by one using
the same iteration mechanism.
Document "smtpctl monitor" and "smtpctl show queue".
ok gilles@
|
|
ok gilles@
|
|
* Log more events (especially client session) and use a better scheme
for that: each messages is prefixed with a token to easily identify
its class:
- info/warn/debug: general server messages
- smtp-in: smtp client connections
- relay: status update for relayed messages
- delivery: status update for local deliveries
* Implement "smtpctl monitor" to display updates of selected internal
counters.
* When reloading the on-disk queue at startup do not commit a message
if no envelope was submitted for that message.
* Remove unused stuff in the config parser.
ok gilles@
|
|
pushing the fd with the envelope. This allows the mda to deal itself with
session limits. Envelopes are sent at full rate to the mda, which buffers
them on per-user queues, or sends them back for rescheduling if it already
has too many pending envelopes. Delivery sessions are created (within per-
user and global limits) to drain the queues.
This makes the server handle envelope bursts more efficiently.
ok gilles@
|
|
exhaustion situation that kills the server. For now, put a safe limit on
the number of envelopes sent by the queue process to the mda.
ok gilles@ chl@
|
|
ok gilles@ eric@
|
|
It automagically adds an rcpt=<user@domain> field if "dest" differs from
the original "rcpt". The function takes an "extra" parameter that allows
to add some specific info depending on the context.
ok gilles@
|
|
Ignore it in existing envelopes until it gets completely dropped.
Change "smtpctl show queue" to display the address family of the
envelope source instead of the ENQUEUE flag.
ok gilles@
|
|
and put it out of profiling, so it's not accounted.
While there, for PROC_PARENT:
- set smtpd_process for PROC_PARENT
- use setproctitle() like other processes
ok gilles@
|
|
ok gilles@
|
|
there in the last few years ;-)
- get rid of availdesc(): getdtablecount() is so much more reliable
- get rid of env->sc_maxconn, we can be much smarter with getdtablecount()
and getdtablesize()
- disable accept when we hit the control process fd reserve
- disable accept when we fail
- enable accept when we're back below the limit
this is not the full fd exhaustion diff, i'll merge changes from relayd
tomorrow, this was only required to get rid of the env->sc_maxconn and
availdesc() mess
"reads alright" eric@
|
|
- statistics can now have a type (counter, timestamp, timeval, timespec and
possibly others in the future)
- stat_increment() / stat_decrement() now take an increment/decrement value
and are at the moment only of type counter
- stat_set() now takes a stat_value
- provide helpers to convert raw values to stat_value
ok eric@, ok chl@
while at it fix a rq_queue_dump() call using a bogus timestamp in scheduler
ramqueue.
|
|
suggested by Jan Stary.
move queue loading notification to log_debug() while there.
ok gilles@
|
|
queue, also notify the scheduler so it can rollback the current update.
Send only the msgid while there.
ok gilles@
|
|
the mta/mda.
ok gilles@
|
|
ok gilles@
|
|
ok gilles@
|
|
> statistics are no longer static structures in shared memory
> statistics are only set, smtpd never uses them in its logic
> each statistic is a key/value where key can be any (dynamic) string
- convert all uses of the former API to use the new one
- implement stat_ramstat that keeps non-persistent stats in ram structure
ok eric@, ok chl@
|
|
Add missing header needed by time()
ok eric@
|
|
to be grouped into a single bounce message.
The bounce structure keeps a list of envelopes. For now, the list
is constructed by delaying the re-enqueuing of a bounce envelope a
bit, to wait for other bounces from the same message to be part of
the same report.
|
|
queue backend within the queue process.
The scheduler sends envelope ids to the queue process which loads
the envelope and forward the request to the agent responsible for
the delivery. The result is sent by the agent to the queue which
updates the storage before notifying the scheduler.
Bounces are created and enqueued (from the client side) by the
queue process, rather than the scheduler.
ok gilles@
|
|
New envelopes are pushed into the scheduler through the insert()
commit() rollback() transactional interface functions.
Worklists are pulled from the scheduler through a single batch()
interface function, which returns a list of envelope ids and the
type of processing. Envelopes returned in this batch are said to
be "in-flight", as opposed to "pending". They are supposed to be
processed in some way, and either updated() or deleted() at some
point.
The schedule()/remove() functions are used to alter the internal
state of "pending" envelopes to make them schedulable. The enve-
lopes will be part of a worklist on the next call to batch().
Rewrite the scheduler_ramqueue backend.
The initial queue loading in now done by the queue.
ok gilles@
|
|
longer have them and runner is actually a scheduler so rename.
- introduce scheduler_backend which does the same to scheduler than
queue_backend does to queue and map_backend does to maps
- remove all occurences of RUNNER and runner, replace them with SCHEDULER
and scheduler
ok eric@, ok chl@
|
|
incoming messages are now always stored in /incoming, whatever the queue_backend is.
remove QOP_FD_RW and fsqueue_message_fd_rw().
while there check return value of generated paths before calling rmtree()
with advice from gilles@ and eric@
ok gilles@ eric@
|
