Age | Commit message (Collapse) | Author |
|
this allows an admin to inspect the queue without having to manually
extract bucket and find the path to an envelope or message.
diff by Sunil Nimmagadda <sunil@poolp.org>
ok eric@, chl@ and I
|
|
requested by gilles@
|
|
Ignore it in existing envelopes until it gets completely dropped.
Change "smtpctl show queue" to display the address family of the
envelope source instead of the ENQUEUE flag.
ok gilles@
|
|
- remove SHOW_RUNQUEUE it was a noop since runqueues have been removed
years ago from smtpd
|
|
- statistics can now have a type (counter, timestamp, timeval, timespec and
possibly others in the future)
- stat_increment() / stat_decrement() now take an increment/decrement value
and are at the moment only of type counter
- stat_set() now takes a stat_value
- provide helpers to convert raw values to stat_value
ok eric@, ok chl@
while at it fix a rq_queue_dump() call using a bogus timestamp in scheduler
ramqueue.
|
|
ok eric@
|
|
> statistics are no longer static structures in shared memory
> statistics are only set, smtpd never uses them in its logic
> each statistic is a key/value where key can be any (dynamic) string
- convert all uses of the former API to use the new one
- implement stat_ramstat that keeps non-persistent stats in ram structure
ok eric@, ok chl@
|
|
ok gilles@ chl@
|
|
ok gilles@
|
|
longer have them and runner is actually a scheduler so rename.
- introduce scheduler_backend which does the same to scheduler than
queue_backend does to queue and map_backend does to maps
- remove all occurences of RUNNER and runner, replace them with SCHEDULER
and scheduler
ok eric@, ok chl@
|
|
Add a command-line option to specify the backend to use at runtime.
ok gilles@
|
|
internally in fsqueue backend for now, and let the fsqueue_message()
and fsqueue_envelope() dispatchers do the right thing.
Based on a diff by chl@
ok chl@ gilles@
|
|
"smtpctl show" output, and use errx rather than err for the unknown
message error.
ok gilles
|
|
Found by LLVM/Clang Static Analyzer.
ok gilles@
|
|
- introduce the scheduler_ramqueue backend
- remove all occurences of ramqueue outside of the ramqueue backend
- teach runner how to use the new API
it is now possible to write custom schedulers !
ok eric@, ok chl@
|
|
call is issued by smtpd or smtpctl. In the latter case, only perform
sanity checks and do not touch directories. A running server no
longer lose its "incoming/" directory each time smtpctl is called...
ok gilles@
|
|
notify the runner of what happened with an envelope that has been
scheduled. It is not part of the state of the envelope, and it is not
even dumped. So it should only be set by mta/mda, checked by runner
to decide what to do with the envelope, and ignored everywhere else.
ok gilles@
|
|
and confusing.
ok gilles@
|
|
ok eric@
|
|
to smtpctl.c
ok gilles@
|
|
Bug reported by Mark Patruck <mark at wrapped.cx>
ok gilles@ eric@
|
|
queue_backend into account and assumed a filesystem with a specific layout.
This commit does plenty of things:
- make qwalk an abstraction in the queue_backend API, and impose queue
drivers to implement qwalk_open(), qwalk() and qwalk_close();
- move previous qwalk_open(), qwalk() and qwalk_close() to the fsqueue
driver since they were fsqueue specific ...
- make qwalk API work with msgid/evpid instead of pathnames since we're
going to use the queue_backend API to load envelopes by evpid anyway;
- makes smtpd use *solely* the queue_backend API when manipulating the
queue. pathnames were removed from smtpd.h and moved into the fsqueue
which means we can now store a queue anywhere ... as long as we write
the ten functions or so required for a queue driver ;-)
ok eric@, ok chl@
|
|
- rename IMSG and smtpctl pause/resume parameters
- update man page
tested by me, ok chl@, eric@
|
|
- introduce 'smtpctl schedule-all'
ok eric@
|
|
couple of load/dump functions to convert to and from a human readable fmt.
while at it kill struct delivery and merge back its fields to the envelope.
this basically means we shouldn't require users to flush their queues every
time we make a change to struct envelope.
work is not done, but we're at a better state than the binary fsqueue so
we'll improve it in-tree.
has been running on my own box for the last 12 hours or so
ok eric@, chl@
|
|
simpler and hopefully saner way.
ok gilles@ chl@
|
|
ok gilles@
|
|
ok gilles@
|
|
ok gilles@
|
|
for developers to see the impact of structure changes on memory and disk
usage, and useful for users to better understand 'smtpctl show stats'
|
|
improve scheduling and general ramqueue operations. unused yet
|
|
- bring back 'smtpctl schedule' and 'smtpctl remove' to life
Things you should know:
The ramqueue data structure is not finished yet and lacks an envelope tree
for evpid lookups. I wanted to wait until I'm done but too many people are
affected by not being able to reschedule envelopes, this is a quick fix.
So right now there's an O(rrible) complexity as both commands will perform
a (possibly aborted) queue scan leading to O(n). I will make that O(log n)
soon.
Also, smtpctl remove no longer supports removing an entire message, I will
fix that very soon too.
|
|
discussed with and ok gilles@
|
|
no functionnal change
|
|
a continuous walk on the disk-queue. the implementation differs from what
jacekm@ commited (and I backed out) a while ago in that it uses a queue and
a host tree required for upcoming features.
code will be improved in tree, it requires changes to be done in queue and
bounce API, I just wanted to commit a working version first ...
tested by todd@ and I
|
|
ok gilles@
|
|
|
|
|
|
|
|
"ok and no need to keep them for yourself" gilles@
|
|
is way more optimized than what we had earlier and there's definitely stuff
we want to keep, however it is early optimization that doesn't account for
many features and makes them hard (if not impossible) to write without
ugly workarounds that ruin the purpose of the optimizations.
the backout goes to 30 May's right before the commit and catches up on all
the non-queue related commits that happened since then.
i'll work on reintroducing the ideas from this queue when the basic
features we expect from a MTA are implemented.
suggested on tech@ about a week ago, no objections, several "please make
smtpd move forward" mails from hackers and tech readers.
|
|
|
|
|
|
|
|
Major goals:
1) Fix bad performance caused by the runner process doing full queue
read in 1s intervals. My Soekris can now happily accept >50 msg/s
while having multi-thousand queue; before, one hundred queue would
bring the system to its knees.
2) Introduce Qmail-like scheduler that doesn't write as much to the
disk so that it needs less code for servicing error conditions,
which in some places can be tricky to get right.
3) Introduce separation between the scheduler and the backend; these
two queue aspects shouldn't be too tied too each other. This means
that eg. storing queue in SQL requires rewrite of just queue_backend.c.
4) Make on-disk queue format architecture independent, and more
easily extensible, to reduce number of flag days in the future.
Minor goals:
ENOSPC no longer prevents delivery attempts, fixed session limiting
for relayed mail, improved batching of "relay via" mails, human-readable
mailq output, "show queue raw" command, clearer logging, sending
of single bounce about multiple recipients, exact delay= computation,
zero delay between deliveries while within session limit (currently
1s delay between re-scheduling is enforced), mta no longer requests
content fd, corrected session limit for bounce submissions, tiny
<100B queue files instead of multi-KB, detect loops before accepting
mail, reduce traffic on imsg channels by killing enormous struct
submit_status.
|
|
channels to parent, mda, mta, lka, smtp, and control. This leaves
just the channel to queue, which forwards imsgs on runner's behalf
and redirects any replies back to it.
OK gilles@
|
|
and assuming message is not in processing/scheduled state
- teach smtpctl how to request message removal from runner
discussed with todd@, idea ok jacekm@
|
|
logging on runtime.
Based on claudio@'s work on ripd, ospfd, ospf6d, dvmrpd, ldpd, bgpd.
With help/ideas/testing from gilles@ jacekm@ todd@
ok jacekm@
|
|
are now controlled.
|
|
|