Search the FAQ Archives

3 - A - B - C - D - E - F - G - H - I - J - K - L - M
N - O - P - Q - R - S - T - U - V - W - X - Y - Z
faqs.org - Internet FAQ Archives

Mailing list management software FAQ
Section - 2.04 Performance and system-load issues related to server activity

( Single Page )
[ Usenet FAQs | Web FAQs | Documents | RFC Index | Cities ]


Top Document: Mailing list management software FAQ
Previous Document: 2.03 Do you need to modify the MLM's source code?
Next Document: 2.05 Performance and system-load issues related to mail delivery
See reader questions & answers on this topic! - Help others by sharing your knowledge
The system load an MLM imposes is affected by how it organizes its work in
command processing.  Just about any method is OK if your MLM isn't going to
have much activity (in both postings and server requests), but when you get
into bigger systems the MLM's efficiency turns into a real concern.  "Big,"
here, means a server processing dozens of messages, tens of thousands of
recipients, hundreds of requests -- numbers like that, or bigger.  Really big
LISTSERV systems, for example (I use them as an example because their
statistics are posted monthly on LSTSRV-L) may send out an average of half a
million messages a day, meaning they may do a million on a busy day.

A decent guideline for evaluating an MLM's potential load on your system is,
how many processes does it start under light, moderate, and heavy loads?
Let's look first at the minimum load imposed by each Majordomo, SmartList,
ListProc, LISTSERV, and TULP, and then at what happens when they're busy.

"Low load": Majordomo and SmartList are started by Sendmail only when a
message arrives, so that in a baseline state they occupies no resources at
all.  ListProc, LISTSERV, and TULP each keep a server daemon running at all
times, so that even idle they increase the baseline load on your system by
some amount (the penalty is mostly in swap space).

"Heavy load": the tables turn.  Majordomo starts up a Perl process, and
possibly a couple of Sendmail processes, for every message that arrives.
Because Perl is interpreted, and because each new Majordomo and Sendmail
process must interpret its configuration files from scratch, there is
substantial overhead involved in processing each message.  There is no
mechanism to serialize requests, either: if several messages come in at once,
Sendmail fires up an independent Majordomo for each one, and the load
multiplies -- even more so because the independent Majordomos don't organize
to manage file locks, and many of them will sit around for a while (occupying
system resources) as they wait for some other process to release a lock.  In
a message onslaught, your poor system will start to thrash, and you'll be
distressed :-).  SmartList starts up a Procmail process, and possibly a
couple of Sendmail processes, for each message that arrives.  Although
Procmail also uses an interpreted language and like Majordomo needs to read
its configuration files from scratch, SmartList has a lower impact on system
resources because the Procmail interpreter is smaller and (for its job) more
efficient than Perl.  SmartList, by default, handles list submissions in
parallel and administrative requests serially per mailing list, or per
arbitrary group of mailing lists.  In contrast to Majordomo and SmartList,
the ListProc, LISTSERV, and TULP daemons serialize all requests without
multiplying.  Your outgoing mail queue may build up, but even that's not a
problem in most cases because Sendmail (or any other MTA you may use) can be
configured to queue mail when the system is heavily loaded.  As a final
optimization, LISTSERV even maintains an open connection to your system's
port 25 for delivery of outgoing mail; its assumption is that requests will
come in fairly regularly, and that the average load of maintaining an open
Sendmail process is less than that of starting one, then tearing it down for
each new request that arrives.  The bottom line with a daemon-running MLM is
that there is a decreased load on busy systems, and an increased load for
lightly-used systems, but the biggest benefit is that the load is
*predictable*.

Of course, just looking at process count is simplistic, but it gives you an
idea of the things the higher-end MLM's do to reduce system load.  When you
get into really big lists, other features start to make a difference.  As
another small example, without getting too deep into these issues
(salespeople can do that if you're running a gigantic server): LISTSERV
maintains hashed indices to all of its lists so that it can guarantee quick
name lookups even on very large lists.  Think: when a list has 25,000
subscribers, even a grep through it takes a non-trivial time, and that
database of names needs to be checked *every* time a request comes in.  If
you're running a list with a LOT of activity (say, as some people have, you
visualize tens of thousands of people subscribing within a single day), you
can't afford to have a single subscription taking more than a second or two
of CPU time, in total.

The point is, if you are going to run big lists, it's worth your while to
check out what each system does to manage it efficiently.  Most approaches
don't scale well, when pressed.

User Contributions:

Comment about this article, ask questions, or add new information about this topic:




Top Document: Mailing list management software FAQ
Previous Document: 2.03 Do you need to modify the MLM's source code?
Next Document: 2.05 Performance and system-load issues related to mail delivery

Single Page

[ Usenet FAQs | Web FAQs | Documents | RFC Index ]

Send corrections/additions to the FAQ Maintainer:
naleks@Library.UMMED.EDU





Last Update March 27 2014 @ 02:11 PM