Saturday, May 31, 2008

Install Multimedia Codecs (libdvdcss2,w32codecs) in Ubuntu 8.04 (Hardy)

original from http://onlyubuntu.blogspot.com/2007/11/install-mplayer-and-multimedia-codecs.html

make sure these src are in /etc/apt/sources.list


deb http://archive.ubuntu.com/ubuntu gutsy universe multiverse
deb-src http://archive.ubuntu.com/ubuntu gutsy universe multiverse


Installing libdvdcss2 and w32 video codecs in Ubuntu Gutsy Gibbon

Support for WMV, RealMedia and other formats has been bundled into the w32codecs package. This package is not available from the Ubuntu repositories due to licensing and legal restrictions.

For Ubuntu Gutsy Gibbon Users run the following command

sudo wget http://www.medibuntu.org/sources.list.d/hardy.list -O /etc/apt/sources.list.d/medibuntu.list

Now you need to copy the key using the following command

wget -q http://packages.medibuntu.org/medibuntu-key.gpg -O- ¦ sudo apt-key add -

Update the source list using the following command

sudo apt-get update

Install Codecs using the following command

sudo apt-get install w32codecs libdvdcss2

Using above download locations you can install most of the mutimedia codecs for ubuntu.

sudo apt-get update

Mplayer Plugin for Firefox

If you want to install Mplayer with plug-in for Mozilla Firefox run the following command

sudo apt-get install mozilla-mplayer


element to bash shell 1

To get a sorted listing of all users on the system, type:

$ cut -d: -f1 < /etc/passwd | sort

nice: change job's priortity

Special Characters Character Meaning
~Home directory1
`Command substitution (archaic)4
#Comment4
$Variable expression3
&Background job1
*String wildcard1
(Start subshell8
)End subshell8
\Quote next character1
|Pipe1
[Start character-set wildcard1
]End character-set wildcard1
{Start command block7
}End command block7
;Shell command separator3
'Strong quote1
<">Weak quote1
<Input redirect1
>Output redirect1
/Pathname directory separator1
?Single-character wildcard1
!Pipeline logical NOT5

quoting and backslash escaping

$ echo '2 * 3 > 5' is a valid inequality
$ echo 2 \* 3 \> 5 is a valid inequality.
$ echo The Caterpillar and Alice looked at each other for some \ > time in silence: at last Caterpillar took the hookah out of its \> mouth, and addressed her in a languid, sleepy voice.

Basic shell command
erase  kill   werase rprnt  flush  lnext  susp   intr   quit   stop   eof
^?     ^U     ^W     ^R     ^O     ^V     ^Z/^Y  ^C     ^\     ^S/^Q  ^D

Control Keystty NameFunction Description
CTRL-C   intr      Stop current command
 CTRL-D  eof       End of input
CTRL-\    quit      Stop current command, if CTRL-C doesn't work
CTRL-S   stop      Halt output to screen
CTRL-Q  Restart output to screen
DEL or   CTRL-?  eraseErase last character
CTRL-U  kill         Erase entire command line
CTRL-Z   susp     Suspend current command
CTRL-A  head   head of current command line
CTRL-H  backspace
CTRL-S and CTRL-Q are basically a nuisanc

The resulting output will include this information:
intr = ^c; quit = ^|; erase = DEL; kill = ^u; eof = ^d; eol = ^`;swtch = ^`; susp = ^z; dsusp <undef>;



Thursday, May 29, 2008

netstat

netstat -nat
w

netstat -tun
netstat -tun | grep ":80"

netstat   -atnp  
-p means process

Sunday, May 25, 2008

static inline VS extern inline

to be honest, i dont know the difference between extern inline and static inline. however, there is an article.
=====

http://tree.celinuxforum.org/CelfPubWiki/ExternVsStaticInline

This page describes how 'extern inline' is used in the kernel vs. 'static inline', and some of the tradeoffs involved.

At one point, the kernel could not function properly when you forced the compiler to not inline functions. There are functions in the kernel which do not work properly when they are expressed as functions (with function prolog and epilog code) rather than as inline code.

In 2001, Linus said:

- "static inline" means "we have to have this function, if you use it
but don't inline it, then make a static version of it in this
compilation unit"

* "extern inline" means "I actually _have_ an extern for this function,
but if you want to inline it, here's the inline-version"

... we should just convert
all current users of "extern inline" to "static inline".

see http://www.uwsg.indiana.edu/hypermail/linux/kernel/0107.3/0466.html and whole thread at: http://www.uwsg.indiana.edu/hypermail/linux/kernel/0107.3/index.html#440

However, there are exceptions to this rule: For example http://www.uwsg.indiana.edu/hypermail/linux/kernel/0107.3/0519.html says:

[conversion from 'extern inline' to 'static inline']
Doesn't work for the ones in include/linux/parport_pc.h, which have
extern versions in drivers/parport/parport_pc.c. Gives build errors.

See http://www.greenend.org.uk/rjk/2003/03/inline.html for a good discussion on inlining in GCC in general.

The following are some reasons to NOT change 'extern inline' to 'static inline' in order to preserve code correctness:

  • if function pointers are taken for functions marked 'extern inline', then they will all be to the same function, and will match. However, if function pointers are takend for static inline functions, then they won't match and code which compares function addresses will behave differently.
  • kernel developers sometimes use 'extern inline' to mark function which

    MUST not be inlined. Although the compiler may choose to not inline a function so marked, this rarely occurs in practice. By setting a function as 'extern inline', and then NOT providing as associated extern non-inline function to back it up, if the compiler fails to inline the function a linker error will be generated. This guarantees that the code will either run with the function inlined, or that it cannot be run at all.

So... when a kernel developer uses 'extern inline' without a backing extern function, it is an indication of a function that MUST be inlined.

There are other uses of 'extern inline' WITH backing extern functions (grep for EXTERN_INLINE - it will show up in the alpha architecture), but these are used for the traditional 'extern inline' reasons.

Saturday, May 24, 2008

how to use the robot.txt file

http://www.dwfaq.com/tutorials/Miscellaneous/robot_txt.asp

Q. How can I control which pages are indexed by the Search Engines?

A. By adding a robots.txt file to the root directory of your website, you can help control the indexing of your site by robots that ignore the <META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW"> convention.

The mystery of the robots.txt file revealed

Control which of your pages are NOT indexed with a robots.txt file

You should add a robots.txt file to the root directory of all your websites to help control the indexing of your site by robots that ignore the <META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW"> convention. In this file you specifically list any pages that you DO NOT want walked and indexed (such as password protected folders and folders which contain only images, etc.). The robots.txt file is very simple yet very powerful and every website should have a robots.txt file on the root directory.

The Terminology

Create a new file with Notepad and call it robots.txt

The two conventions used in robots.txt file are User-agent: and Disallow: /

User-agent: * By using the * or wild card you are addressing ALL robots. If you wish to address individual robots you need to list each robot separately with an individual User-agent: statement. They must be listed by their specific name or IP Address, along with a separate Disallow: / statement listing the folders and files you DO NOT want the specified robot to index.

Tip: Use the * wild card to address all robots..... it is the safest way

Disallow: / List any folders that you do not want to have indexed by robots.

Warning: Disallow: / used without any folder name tells the robot do not index ANY page of the website.

ALL Files and folders in the directory named in the Disallow: / statement as well as all of those under it will NOT be indexed by robots.

Sample of Folders that could be in this website that we would not like the spiders to index with the search engines:

Disallow: /tutorials/meta/
Disallow: /tutorials/images/
Disallow: /tutorials/assets/
Disallow: /tutorials/404redirect/

Example: Disallow: /tutorials/
Results: All files and sub folders located within the folder tutorials which includes all the folders listed in the above example as well as any other sub folders of the tutorials directory will not be indexed by the robots if you use this statement.

This would mean that the /meta, /images, /assets, /404redirect, AND any other folders as well as all of the files in those foldes will not be seen by indexing robots.

You may also list specific files that you do not want indexed in a robots.txt file.

Sample of Specific Files that could be in this website that we would not like the spiders to index with the search engines:

Disallow: /tutorials/meta_tags.html
Disallow: /tutorials/custom_error_page.html

# Comments can be placed in a robots.txt file by starting the line with #

::back to top::

The Examples

Download a sample robots.txt or see below for an example.

###############################
#
# sample robots.txt file for this website
#
# addresses all robots by using wild card *
#
User-agent: *
# list folders robots are not allowed to index

Disallow: /tutorials/meta/
Disallow: /tutorials/images/
Disallow: /tutorials/assets/
Disallow: /tutorials/404redirect/
#
# list specific files robots are not allowed to index
#
Disallow: /tutorials/meta_tags.html
Disallow: /tutorials/custom_error_page.html
#
# End of robots.txt file
#
###############################

Related Tutorials

Introduction to Meta Tags
by turtle
URL: http://www.dwfaq.com/Miscellaneous/intro_to_metas.asp

Related Reference and Resources

You can read more about spiders (a.k.a. robots), META tags and what they do, as well as search engine optimization at the following URLs:

J.K. Bowman's Spider Food
URL: http://spider-food.net/handling-robots-b.html

Search Engine World
URL: http://www.searchengineworld.com/robots/robots_tutorial.htm

Search Engine Guide
URL: http://www.searchengineguide.com/1stsearchranking/2001/robots.html

Search Tools
URL: http://www.searchtools.com/robots/robots-txt.html

ZDNet
URL: http://www.zdnet.com/devhead/stories/articles/0,4413,1600632,00.html


Tuesday, May 20, 2008

Mount ISO, CUE/BIN, NRG, IMG, MDF files in ubuntu

I known that to mount iso file
pnix@pnix-a7n:~$ sudo mount -o loop file.iso mountpoint
or
pnix@pnix-a7n:~$ sudo mount -o loop -t iso9660 file.iso mountpoint

and for cue/bin file ,I convert it to iso first use bchunk
pnix@pnix-a7n:~$ bchunk file.bin file.cue file.iso

but how about the others. After some search, I collect tips to handle many types of image file in linux.
For nrg[ nero image ], img [ clone cd] and mdf [ alcohol 120% ] files, We need nrg2iso, ccd2iso and mdf2iso to convert those image files to iso image. Luckily, All are in Feisty repos.
pnix@pnix-a7n:~$ sudo aptitude install nrg2iso ccd2iso mdf2iso
Reading package lists... Done
Building dependency tree
Reading state information... Done
Reading extended state information
Initializing package states... Done
Building tag database... Done
The following NEW packages will be installed:
ccd2iso mdf2iso nrg2iso
0 packages upgraded, 3 newly installed, 0 to remove and 0 not upgraded.
Need to get 19.1kB of archives. After unpacking 193kB will be used.
Writing extended state information... Done
Get:1 http://th.archive.ubuntu.com feisty/universe ccd2iso 0.3-1 [7004B]
Get:2 http://th.archive.ubuntu.com feisty/universe mdf2iso 0.3.0-0ubuntu2 [7066B]
Get:3 http://th.archive.ubuntu.com feisty/universe nrg2iso 0.4-1.1 [4996B]
Fetched 19.1kB in 1s (17.8kB/s)
Selecting previously deselected package ccd2iso.
(Reading database ... 107513 files and directories currently installed.)
Unpacking ccd2iso (from .../ccd2iso_0.3-1_i386.deb) ...
Selecting previously deselected package mdf2iso.
Unpacking mdf2iso (from .../mdf2iso_0.3.0-0ubuntu2_i386.deb) ...
Selecting previously deselected package nrg2iso.
Unpacking nrg2iso (from .../nrg2iso_0.4-1.1_i386.deb) ...
Setting up ccd2iso (0.3-1) ...
Setting up mdf2iso (0.3.0-0ubuntu2) ...
Setting up nrg2iso (0.4-1.1) ...
pnix@pnix-a7n:~$

then to covert to iso use command
pnix@pnix-a7n:~$ mdf2iso file.mdf file.iso
pnix@pnix-a7n:~$ ccd2iso file.img file.iso
pnix@pnix-a7n:~$ nrg2iso file.nrg file.iso


for nrg file, we can mount it directly by
pnix@pnix-a7n:~$ sudo mount -o loop,offset=307200 file.nrg mountpoint


Other choices
cdemu[a kernel module for mounting Cue/Bin files directly] with nautilus script.more
AcetoneISO gui app to mount all image file.

Monday, May 19, 2008

imagemagick

PHP running on your server does not support the GD image library, check with your webhost if ImageMagick is installed



#aptitude install imagemagick
#aptitude install php5-imagick

ImageMagick installed
Check to see if ImageMagick is installed so that Gallery can manipulate images (rotate, resize, make thumbnails, etc.). You only need to have either ImageMagick or Netpbm installed.

Jhead installed
Check to see if jhead is installed so that Gallery can examine EXIF headers embedded in images created with most digital cameras.

jpegtran installed
Check to see if jpegtran is installed so that Gallery can perform lossless rotations on JPEG images.


Monday, May 12, 2008

howto: flush postfix mail queue

Traditional "sendmail -q" command flushes mail queue. Under Postfix, just enter the following to flush the mail queue
# postfix flush
OR
# postfix -f

To see mail queue, enter:
# mailq

To remove all mail from the queue, enter:
# postsuper -d ALL

To remove all mails in the deferred queue, enter:
# postsuper -d ALL deferred

postfix-delete.pl script

Following script deletes all mail from the mailq which matches the regular expression specified as the first argument (Credit: ??? - I found it on old good newsgroup)

#!/usr/bin/perl
 
$REGEXP = shift || die "no email-adress given (regexp-style, e.g. bl.*\@yahoo.com)!";
 
@data = qx</usr/sbin/postqueue -p>;
for (@data) {
if (/^(\w+)(\*|\!)?\s/) {
$queue_id = $1;
}
if($queue_id) {
if (/$REGEXP/i) {
$Q{$queue_id} = 1;
$queue_id = "";
}
}
}
 
#open(POSTSUPER,"|cat") || die "couldn't open postsuper" ;
open(POSTSUPER,"|postsuper -d -") || die "couldn't open postsuper" ;
 
foreach (keys %Q) {
print POSTSUPER "$_\n";
};
close(POSTSUPER);
 

For example, delete all queued messages from or to the domain called fackspamdomain.com, enter:
./postfix-delete.pl fackspamdomain.com
Delete all queued messages that contain the word "xyz" in the e-mail address:
./postfix-delete.pl xyz

Updated for accuracy.

Want to stay up to date with the latest Linux tips, news and announcements? Subscribe to our free e-mail newsletter or full RSS feed to get all updates. You can Email this page to a friend.

You may also be interested in...

Discussion on This Article:

  1. Matt Says:

    The command for "To remove all bounced mail from Queue, enter:" is incorrect. That command will completely delete all the contents of your Mail Queue.

    When I read this I thought it would eliminate only the bounced back emails that were sitting in my queue. I was wrong. It deleted all 6,000 of the emails sitting in my queue.

    If this post is any sign of the type of information provided on this website, the readers are in for a big surprise…and some pink slips from their employer..

  2. vivek Says:

    Matt,

    thanks for the heads up. It was a typo. The post has been updated

  3. Henk Says:

    I think matt deserves a pink slip from his employer, because he uses commands found on random sites without even checking what the effects are!

  4. Jim Says:

    I've found this code on two different blogs and have received compilation errors both times similar to:

    Unmatched right curly bracket at ./delete-queue line 9, at end of line
    (Might be a runaway multi-line ;; string starting on line 5)
    syntax error at ./postfix-delete.pl line 9, near "}"
    Execution of ./postfix-delete.pl aborted due to compilation errors.

    I'm more of a PHP person so Perl's not my thing. I know syntax is really important and I've double and triple checked to make sure things are correct, including composing in vi, pico and bbedit editors.
    ??

  5. Manuel Says:

    Wow This is a seriously crappy coding job. Who writes this stuff? I cleaned this up to not only make sure this was scoped correctly, but also try to have a novice perl user understand it.

    #!/usr/bin/perl

    my $REGEXP = shift(@ARGV) || die "no email-adress given (regexp-style, e.g. bl.*\@yahoo.com)!";

    my %Q;
    my $queue_id;
    my @data = `/usr/sbin/postqueue -p`;
    foreach my $line (@data) {
    if ($line =~ /^(\w+)(\*|\!)?\s/) {
    $queue_id = $1;
    }
    if($queue_id) {
    if ($line =~ /$REGEXP/i) {
    $Q{$queue_id} = 1;
    $queue_id = "";
    }
    }
    }

    open(POSTSUPER,"|postsuper -d -") || die "couldn't open postsuper" ;

    foreach my $key (keys %Q) {
    print POSTSUPER "$key\n";
    }
    close(POSTSUPER);

    Hopefully this helps.. I cannot confirm that this works, just cleaned it up so that it makes sense.

Sunday, May 11, 2008

simple setting SPF DNS

http://old.openspf.org/dns.html

Pick a default.

SPF domains have to publish at least two directives: a version identifier and a default mechanism.

 singnet.com.sg. TXT "v=spf1 -all"

This is the simplest possible SPF record: it means your domain singnet.com.sg never sends mail.

It makes sense to do this when a domain is only used for web services and doesn't do email.

But most domains will want to designate permitted hosts using one or more mechanisms.

If your MX servers send mail, designate them.

 singnet.com.sg. TXT "v=spf1 mx -all"

singnet.com.sg has 8 MX records, pointing to 8 IP addresses. Those hosts are now designated mailers.

If other machines in the domain also send mail, designate them.

 singnet.com.sg. TXT "v=spf1 mx ptr -all"

This designates all the hosts whose PTR hostname match singnet.com.sg.

If any other machines not in the domain also send mail from that domain, designate them.

 singnet.com.sg. TXT "v=spf1 a:singnet.com.sg mx ptr -all"

singnet.com.sg's IP address doesn't show up in its list of MX servers. So we add an "a" mechanism to the directive set to match it.

 singnet.com.sg. TXT "v=spf1 a mx ptr -all"

This is shorthand for the same thing.

Each of your mail servers should have an SPF record also.

When your mail servers create a bounce message, they will send it using a blank envelope sender: <>. When an SPF MTA sees a blank envelope sender, it will perform the lookup using the HELO domain name instead. These records take care of that scenario.

  mx11.singnet.com.sg. TXT "v=spf1 a -all"
mx12.singnet.com.sg. TXT "v=spf1 a -all"
mx13.singnet.com.sg. TXT "v=spf1 a -all"
mx14.singnet.com.sg. TXT "v=spf1 a -all"
mx15.singnet.com.sg. TXT "v=spf1 a -all"
mx16.singnet.com.sg. TXT "v=spf1 a -all"
mx17.singnet.com.sg. TXT "v=spf1 a -all"
mx18.singnet.com.sg. TXT "v=spf1 a -all"

Consider creating an SPF record for every other machine in your domain.

Spammers can forge hostnames as well as domain names: to SMTP there is no difference between the two. If they start forging the hostnames of web servers, unix servers, even workstations, you'll want to create SPF records for those machines also.

Notes

If you send mail through another organization's servers, you should use an Include directive to point to their servers. If they do not have SPF records, maybe they don't know about SPF. Tell them about it!

(optional) use Include records to share these hosts

If other domains use exactly the same set of hosts, you can set up redirects for them. "Redirect" aliases point to other domains which themselves publish SPF records. This aliasing mechanism makes it possible to easily consolidate multiple domains that share the same set of designated hosts.

That's it. You're done.

Note: The above examples are good for a simple case, but do read the Mechanisms page or the SPF draft RFC to see how to configure complex cases. You can have multiple includes for a given domain.

Once you've set up records, try them out.


howto: define a SPF DNS record

http://www.zytrax.com/books/dns/ch9/spf.html

This section defines HOWTO configure a Sender Policy Framework (SPF) record for a domain and its mail servers.

<grovelling apology> The macro feature of SPF was incorrectly documented using parenthesis, these should have been braces ({}).</grovelling apology>

SPF was initiated by Meng Weng Wong of pobox.com and is now an IETF standard (RFC 4408) to enable validation of legitimate sources of email for a domain.

Briefly the design intent of the SPF record is to allow a receiving MTA (Message Transfer Agent) to interrogate the Name Server of the domain which appears in the email (the sender) and determine if the originating IP of the mail (the source) is authorized to send mail for the sender's domain.

The SPF information SHOULD be defined in a standard TXT RR and MAY now be defined in an SPF RR type (BIND releases from 9.4.0 support the SPF RR type - see also RFC 4408).

If a SPF (TXT) RR exists and authorizes the source IP address the mail can be accepted by the MTA. If the SPF (TXT) RR does not authorize the IP address the mail can be bounced - it did not originate from an authorized source for the sender's domain. If the domain does not have an SPF RR the situation is no worse than before.

Many Open Source MTAs have already been modified to use the SPF record and there is no down-side and plenty of potential up-side to implement the proposed record format now.

We use the following terminology to try and simplify the descriptions below:

  1. sender - the full email address of the originator of the mail item (typically uses return-path in the actual SPF checks)
  2. source-ip - the IP address of the SMTP server trying to send this message
  3. sender-domain the domain name part of the sender's email address e.g. assume the sender is info@example.com the sender-domain is example.com.

The SPF record defines one or more tests to carry out to verify the sender. Each test returns a condition code (pre below). The first test to pass will terminate SPF processing.

TXT RR Format

The standard TXT and SPF record format is defined as:

name  ttl  class   TXT     text
name ttl class SPF text

The SPF RR is functionally identical to a TXT record with SPF data. BIND 9.4+ supports the SPF RR type, however previous versions, and most other DNS software (as of July 2007), do not yet support the SPF RR type. Thus the RFC's recommendation is to always provide a TXT based SPF RR and, if your DNS software supports the SPF RR type, duplicate the information from the TXT version of the SPF RR in a native SPF RR. The reason for this procedure is simply because while the master/slave may support the SPF RR, querying name servers - such as name servers used by receiving MTAs - may not. Some, but not all examples, below have been updated to reflect the use of both record types to illustrate usage. In all cases the TXT and SPF RRs are shown with a comment line between containing the word AND as a reminder of the currently policy recommendation. It is safe to assume for the foreseeable future that only using a TXT version of the SPF will always work.

The SPF data is entirely contained in the text field (a quoted string). SPF defines the contents of the quoted string as follows:

v=spf1 [[pre] type ] ... [mod]

Where:

v=spf1

Mandatory. Defines the version being used. Currently the only version supported is spf1.

pre

Optional (defaults to +). pre defines the code to return when a match occurs. If a test is conclusive either add + or omit (defaults to +). If a test might not be conclusive use "?" or "~" (tilde). "-"(minus) is typically only used with -all to indicate that if we have had no previous matches - fail.
Value Description
+ Default. Pass.
- Fail.
~ Softfail.
? Neutral.

type

Defines the mechanism type to use for verification of the sender. May take one of the following values:

Basic Mechanisms

These types do NOT define a verification mechanism but affect the verification sequence.

  1. include - Recurse (restart) testing using supplied domain. The sender-domain is replaced with the included domain name. Example:
    ; spf record for example.com
    example.com. IN TXT "v=spf1 include:example.net -all"
    ; AND
    example.com. IN SPF "v=spf1 include:example.net -all"
    ; use the SPF details for example.net
    ; in the above case to replace example.com's SPF
    ; or
    example.com. IN TXT "v=spf1 mx include:example.net -all"
    ; additive - use MX RR for example.com
    ; AND if that fails use example.nets's SPF
  2. all - The all type terminates processing (but may be optionally followed by a mod value). It is defined to be optional but it is a Good Thing™ to include it. It is normally present in the form -all to signify that if processing reaches this point without a prior match the result will be fail. But if you are not sure that the tests are conclusive you could use ?all which would allow mail to be accepted even if all previous checks failed.

Sender Mechanisms

These types define a verification mechanism.

  1. ip4 - use IP Version 4 addresses e.g. 192.168.3.0 for verification
  2. ip6 - use IP Version 6 addresses for verification
  3. a - use DNS A RRs for verification
  4. mx - use DNS MX RRs for verification
  5. ptr - use DNS PTR RRs for verification
  6. exists - test for existence of domain
Value Description

a
a:domain
a:domain/cidr
a/cidr

In its base form this uses the sender-domain to find an A RR(s) to verify the source. This form relies on an A RR for the domain e.g.

; fragment for example.com
$ORIGIN example.com.
example.com. IN TXT "v=spf1 a -all"
; AND
example.com. IN SPF "v=spf1 a -all"
; needs domain A record
@ IN A 192.168.0.3
; functionally the same as
example.com. IN A 192.168.0.3

The form a/cidr applies the test to the cidr (or IP refix or slash) range of the sender-domain's A RR.

The form a:domain replaces sender-domain with domain's A RR for verification. This does NOT use domain's SPF record(s) (use include for that). The domain form may use macro-expansion features. Example:

; fragment for example.net
$ORIGIN example.net.
@ IN TXT "v=spf1 a:example.com -all"
; AND
@ IN SPF "v=spf1 a:example.com -all"
; will use a single A query to example.com
; which may not yield the result expected unless
; example.com has an A record as below
@ IN A 192.168.0.3
; functionally the same as
example.com. IN A 192.168.0.3

can take a host name format as shown below:

; fragment for example.net
$ORIGIN example.net.
@ IN TXT "v=spf1 a:mail.example.com -all"
; will use a single A query for mail.example.com

The form a:domain/cidr applies the cidr range to the IP address obtained from the A query e.g.

; fragment for example.net
$ORIGIN example.net.
@ IN TXT "v=spf1 a:mail.example.com/27 -all"
; AND
@ IN SPF "v=spf1 a:mail.example.com/27 -all"
; will use a single A query for mail.example.com

Any of the 32 IP addresses that contain mail.example.com will pass. e.g. if the source-ip is 192.168.0.25 and the A RR for mail.example.net is 192.168.0.2 then the test will pass.

mx
mx:domain
mx:domain/cidr
mx/cidr

This basic form without any extensions uses the MX RR of the sender-domain to verify the mail source-ip. The MX record(s) return a host name from which the A record(s) can be obtained and compared with the source-ip. The form mx/cidr applies the IP Prefix or slash range to the A RR address. With any of the domain extensions the MX record of the designated (substituted) domain is used for verification. The domain form may use macro-expansion features.

Warning Remember the MX RR defines the receiving MTA. If this is not the same host(s) as the sending (SMTP) MTA tests based on an mx type will fail. We have also received a report that mx on its own is rejected by certain SPF libraries. We regard this as an error and are trying to contact the library developer to clarify issues.

Examples:

; fragment for example.com
$ORIGIN example.com.
IN TXT "v=spf1 mx:example.net -all"
; AND
IN SPF "v=spf1 mx:example.net -all"
; verify sender using example.net MX and A RRs
; fragment for example.com
$ORIGIN example.com.
IN TXT "v=spf1 mx:/26 -all"
; AND
IN SPF "v=spf1 mx:/26 -all"
; verify sender using example.com MX and A RRs
; and use 16 IP address range
ptr
ptr/domain

Use the source-ip's PTR RR and a reverse map query. The AA RR for the host is then obtained. If this IP matches the sender-ip AND the sender-domain is the same as the domain name of the host obtained from the PTR RR then the test passes. The form ptr:domain replaces the sender-domain with domain in the final check for a valid domain name. The domain form may use macro-expansion features. The PTR record is the least preferred solution since it places a load on the IN-ADDR.ARPA (IPv4) or IPV6.ARPA reverse-map domains which generally have less capacity than the gTLD and ccTLD domains. Examples:

; fragment for example.com
$ORIGIN example.com.
@ IN TXT "v=spf1 ptr -all"
; the effect is to allow any host which is
; reverse mapped in the domain to send mail
ip4:ipv4 ip4:ipv4/cidr In its basic form defines an explicit ipv4 address to verify the mail source-ip. If the source-ip is the same as ipv4 the test passes. May optionally take the form ipv4/cidr to define a valid IP address range. Since this type incurs the least additional load on the DNS the current draft of the proposed RFC recommends this format. Examples:
; fragment for example.com
$ORIGIN example.com.
@ IN TXT "v=spf1 ip4:192.168.0.2 -all"
; AND
@ IN SPF "v=spf1 ip4:192.168.0.2 -all"
; if source-ip is 192.168.0.2 test passes
; cidr format
@ IN TXT "v=spf1 ip4:192.168.0.2/27 -all"
; AND
@ IN SPF "v=spf1 ip4:192.168.0.2/27 -all"
; if source-ip is in range 192.168.0.1
; to 192.168.0.31 test passes
ip6:ipv6
ip6:ipv6/cidr
In its basic form defines an explicit ipv6 address to verify the mail source-ip. If the source-ip is the same as ipv6 the test passes. May optionally take the form ipv6/cidr to define a valid IP address range. Since this type incurs the least additional load on the DNS the RFC recommends this format. Examples:
; fragment for example.com
$ORIGIN example.com.
@ IN TXT "v=spf1 ip6:2001:db8::10 -all"
; AND
@ IN SPF "v=spf1 ip6:2001:db8::10 -all"
; if source-ip is 2001:db8:0:0:0:0:0:10 test passes
; cidr format
@ IN TXT "v=spf1 ip6:2001:db8::10/120 -all"
; if source-ip is in range 2001:db8:0:0:0:0:0:0
; to 2001:db8:0:0:0:0:0:FF test passes
exists:domain

The existence (any valid A RR) of the specified domain allows the test to pass. Domain may use macro-expansion features.

mod

Two optional record modifiers are defined. If present they should follow the last type directive i.e. after the all. The current values defined are as follows:

Modifier Description
redirect=domain Redirects verification to use the SPF records of the defined domain. Functionally equivalent to include but can appear on its own (without a terminating all) or can placed after the all which means "if all the previous test fail try this redirect". Examples:
; fragment for example.com
$ORIGIN example.com.
@ IN TXT "v=spf1 ip4:192.168.0.2 -all redirect=example.net"
; if source-ip is 192.168.0.2 test passes
; if it fails redirect to example.net
; OR single redirect
@ IN TXT "v=spf1 redirect=example.net"
; AND
@ IN SPF "v=spf1 redirect=example.net"
; only use example.net SPF record
exp=txt-rr

The exp record if present should come last in a SPF record (after the all if present). It defines a DNS name whose TXT record's text may be returned with any failure message. Example:

; domain SPF record
IN TXT "v=spf1 mx -all exp=bad.example.com"
; AND
IN SPF "v=spf1 mx -all exp=bad.example.com"
; the getlost TXT record
bad IN TXT "Not allowed to send mail for domain"

The syntax allowed by this record is significantly more complex (see macro-expansion below.

Macro-Expansion

SPF defines a number of macro-expansion features as defined below:

Note: all macro-expansion delimiters use braces {}.

Modifier Description
%{c}Only allowed in TXT records referenced by the exp field. The IP of the receiving MTA.
%{d}The current domain, normally the sender-domain %{o} but replaced by the value of any domain argument in the type above.
%{h}The domain name supplied on HELO or EHLO, normally the hostname of the sending SMTP server.
%{i}sender-ip The IP of SMTP server sending mail for user info@example.com.
%{l}replace with local part of sender e.g. if sender is info@example.com, the local part is info.
%{o}The sender-domain e.g. if email address is info@example.com the sender-domain is example.com.
%{p}The validated domain name. The name obtained using the PTR RR of the sender-ip. Use of this macro will require an additional query unless a ptr type is used.
%{r}Only allowed in TXT records referenced by the exp field. The name of the host performing the SPF check. Normally the same as the receiving MTA.
%{t}Only allowed in TXT records referenced by the exp field. Current timestamp.
%{s} Replace with sender email address e.g. info@example.com
%{v} Replaced with "in-addr" if sender-ip is an IPv4 address and "ipv6" if an IPv6 address. Used to construct reverse map strings.

The above macros may take one or more additional arguments as follows:

  1. r - Indicates reverse the order of the field, for instance, %{or} would display example.com as com.example and %{ir} would display 192.168.0.2 as 2.0.168.192. The normal split uses "." (dot) as the separator but any other character may be used to define the split but a "." (dot) is always used when rejoining so, for instance, %{sr@} would display info@example.com as example.com.info.

  2. digit - the presence of a digit (range 1 to 128) limits the number of right most elements displayed, for instance, %{d1} displays only com only from example.com but %{d5} would display five right hand elements up to the maximum available, in this case it will display example.com since that is all that is available.

Examples

Example 1

Example 1: Assumes a single mail server which both sends and receives mail for the domain.

; zone file fragment for example.com
$ORIGIN example.com.
IN MX 10 mail.example.com.
....
mail IN A 192.168.0.4
; SPF stuff
; domain SPF
example.com. IN TXT "v=spf1 mx -all"
; AND
example.com. IN SPF "v=spf1 mx -all"
; mail host SPF
mail IN TXT "v=spf1 a -all"
; AND
mail IN SPF "v=spf1 a -all"

Notes:

  1. the domain SPF is returned from a sender-domain query using the sender's email address e.g. sender = info@example.com sender-domain = example.com. The SPF record only allows the MX host to send for the domain.
  2. the mail host SPF is present in case the receiving MTA uses a reverse query to obtain the source-ip host name and then does a query for the SPF record of that host. The SPF record states that the A record of mail.example.com alone is permitted to send mail for the domain.

If the domain contains multiple MX servers the domain SPF would stay the same but each mail host should have a SPF record.

Example 2

Example 2: Assumes the domain will send mail through an offsite mail server e.g. an ISP:

; zone file fragment for example.com
$ORIGIN example.com.
IN MX 10 mail.offsite.com.
....
; SPF stuff
; domain SPF
example.com. IN TXT "v=spf1 include:offsite.com -all"
; AND
example.com. IN SPF "v=spf1 include:offsite.com -all"
; WARNING: offsite.com MUST have a valid SPF definition

Notes:

  1. This format should be used IF AND ONLY IF you know that offsite.com has a valid SPF configuration.
  2. include recurses (restarts) verification using the SPF records for offsite.com. Mail configuration changes are localised at offsite.com which may simplify administration.
  3. include could have been replaced with redirect.

Example 3

Example 3: Assumes we are the host for a number of virtual mail domains and that we can send mail from any host in our subnet.

Zone file fragment for one of the virtual mail domains:

; zone file fragment for vhost1.com
$ORIGIN example.com.
IN MX 10 mail.example.com.
....
; SPF stuff
; domain SPF
vhost1.com. IN TXT "v=spf1 include:example.com -all"
; AND
vhost1.com. IN SPF "v=spf1 include:example.com -all"

Notes:

  1. the domain SPF is returned from a sender-domain query using the sender's email e.g. sender = info@vhost1.com, sender-domain = vhost1.com. The SPF record recurses to the DOMAIN example.com for verification.

Zone file for example.com

; zone file fragment for example.com
IN MX 10 mail.example.com.
....
; SPF stuff
; domain SPF - any host from
; 192.168.0.1 to 192.168.0.30 (32 - bcast and mcast = 30)
; can send mail
example.com. IN TXT "v=spf1 ip4:192.168.0.3/27 -all"
; AND
example.com. IN SPF "v=spf1 ip4:192.168.0.3/27 -all"
; mail SPF
mail IN TXT "v=spf1 ip4:192.168.0.3/27 -all"
; AND
mail IN SPF "v=spf1 ip4:192.168.0.3/27 -all"

Notes:

  1. the domain SPF is returned from a sender-domain query using the sender's email e.g. sender = info@example.com sender-domain = example.com. The SPF record allows any host in the 32 address subnet which contains 192.168.0.3 to send mail for this and any host virtual domain e.g virtual1.com in the above example. NOTE: while /27 allows 32 IP addresses subnet rules remove 192.168.0.0 and 192.168.0.31 as the multicast and broadcast addresses respectively. [read more about IPv4 Classes]
  2. In the above scenario we could have used a slightly shorter version such as:
    example.com. IN  TXT    "v=spf1 mx/27 -all"
    ; AND
    example.com. IN SPF "v=spf1 mx/27 -all"
    This record has the same effect as a:192.168.0.3/27 above but will cost a further DNS look up operation whereas the IP is already available.
  3. The above scenario relies on the fact that customers will only send mail via the domain example.com i.e. they will NOT send via another ISP at home or when travelling. If you are not sure if this is the case you can terminate the sequence with ?all which says kinda pass (soft fail) and let the mail go through - perhaps logging the incident to capture statistics.

If the domain contains multiple MX servers the domain SPF would stay the same but each mail host should have a SPF record.

Example 4

Example 4: Assumes that the domain never sends mail from ANY location - ever. Typically you would do this to prevent bogus mail for everyone else - it is a supreme act of self-sacrifice!

; zone file fragment for example.com
; zone does NOT contain MX record(s)
...
; SPF stuff
; domain SPF
example.com. IN TXT "v=spf1 -all"
; AND
example.com. IN SPF "v=spf1 -all"

Notes:

  1. This SPF test will always fail since the only condition it tests is the all which results in a fail.

Example 5

Example 5: Illustrates various macro expansion features:

; zone file fragment for example.com
$ORIGIN example.org.
IN MX 10 mail.example.com.
....
; SPF records
; domain SPF
@ IN TXT "v=spf1 exists:%{ir}.%{v}.arpa -all exp=badguy.example.com"
; AND
@ IN SPF "v=spf1 exists:%{ir}.%{v}.arpa -all exp=badguy.example.com"
badguy IN TXT "The email from %{s} using SMTP server at %{i}
was rejected by %{c} (%{r}) at %{t} because it failed
the SPF records check for the domain %{p}.
Please visit http://abuse.example.com/badguys.html
for more information"

Notes:

  1. The badguy TXT above is split across multiple lines for presentation reasons only and should appear on a single line in the zone file.
  2. The exists:%{ir}.%{v}.arpa tests is a great example BUT IT WILL NOT WORK because the exists type uses an A RR. But it's a great example to show the power of macro-expansion and we can't think of a better one. Sorry about that.

Postfix Virtual Mail Server

https://help.ubuntu.com/community/PostfixCompleteVirtualMailSystemHowto

Abstract

There are many howtos in the Internet about setting up mail servers and various people has various choice of MTAs. Some like, Qmail, while some like Postfix or Exim. I have been using Qmail for a long time and it is an excellent MTA. The way the Qmail is licensed and distributed that there are no binary packages so that users can easily setup with their favorite distribution, and installing basic Qmail setup even is not that difficult but users need applied various patches and tweaks etc to get a complete setup done. With all these issues Qmail is the preferred choice of many geeks, since it won't give you head ache once up and running. The drawback is that this system is difficult to upgrade since users need to compile the source code and install.

The decision behind this guide is to use Postfix,an equally secure and fast MTA like Qmail, it is easy to configure and setup a Basic System in any Linux distribution. Postfix has many add-ons and support Maildir format, PostgreSQL and MySQL backend for storing and managing virtual domains very easily. This setup will be a complete virtual mail domain systems , with anti-virus and spam filtering for ISPs , hosting companies, and individual corporations who wish to use Ubuntu Linux as there preferred server platform.

System Overview

It is important to know how our system works before going to install. A virtual mail system needs to be able to handle email for numerous domains with multiple users over a variety of interfaces. When you handle multiple domains within the same mail system it presents you some management issues. We have to answer these issues using our technology. Assume ,for example you may have following questions to answer.

  • What will you do if you have two users which require same username for different domains?

  • If you are providing imap access and smtp-auth, how do combine the various authentication daemons into a single system?

  • How do you provide security for the numerous components that comprise the system?

  • What we can do, if users are asking their own spam filtering policies.

  • Individual domain administrators are asking a web-based interface to manage their mail domains

  • How do you setup a web-based management system for Postfix?

  • Each user needs a web-base interface to change his mail account's password

  • How you are going backup user account database and disaster recovery?.

  • Utilizing ssl for transport layer security

  • Handle mailing lists for any domain

How do you manage all these issues together?

Don't panic, I will answer all these questions one by one. Be happy and continue to read this howto.

What You Get

  • Web based system administration

  • Unlimited number of domains

  • Virtual mail users without the need for shell accounts

  • Domain specific user names

  • Mailbox quotas

  • Web access to email accounts

  • Web base interface to change user passwords

  • IMAP and POP3 support

  • Auto responders

  • SMTP Authentication for secure relaying

  • SSL for transport layer security

  • Strong SPAM filtering

  • Anti-Virus filtering

  • Log Analysis

Packages Required

The following packages are need to implement our system and most of these packages are in APT repositories. In our installation section you will learn how to install and configure each.

The Big Picture

The following figure shows the big picture of our setup. If you look at it carefully the figure itself is self explanatory.

CompleteSetupOverview.png

In our setup:

  • Postfix Mail Transfer Agent receives emails via the SMTP protocol and delivers them to different places on your hard disk.

  • MySQL database server stores the information to control the behavior of postfix. It knows about users, domains, email forwarding and passwords.

  • Courier is a standalone mail server just like Postfix but we just use its POP3/IMAP server component to let users access the mailboxes.

  • SASL, the Cyrus library is using to authenticate your users who are dialed in at another ISP while they are on the road they get an IP address outside of your network. Your mail server however only trusts local IP addresses. The SASL ,Simple Authentication and Security Layer, adds authentication to SMTP and makes your mail server trust them.

How Postfix Mappings Work?

It is very important to understand how Postfix mapping works. Heart of our system is Postfix mapping. Let's discuss it here. Don't skip this section.

The generic literal meaning of mapping is assign one value to another. What we have to map in Postfix is email user accounts or email address. One example is /etc/aliases, the local aliases or local system users mapping file used by Postfix. The syntax of this file is:

postmaster: root

This makes all the mails which are coming to postmaster@yourdomain.tld are redirected to root@yourdomain.tld. We can divide the above syntax to Left Hand Side LHS and Right Hand Side RHS. This RHS and LHS are common abbreviations which we usually used in mappings. The following table will make this idea even more clear.

LHS

RHS

postmaster:

root

IconHint.png

Usually we do not use colon(:) in LHS for Postfix and this has been done for backward compatibility with historical reasons. The local alias file is a special file that is compiled with newaliases command but not with usual Postfix mapping command postmap

With a basic default Postfix installation we use text file for mappings. We write the mappings into this file and then convert it into a hash file using postmap command so Postfix can look up items quickly. For example, assume that we need to map our virtual mailboxes in a file called /etc/Postfix/virtual_mailboxes. The syntax of this file look like:

info@domain1.com sigiri
info@domain2.com kala

You may have noticed that we don't have colon(:) in the LHS of the mappings file

Then you need to run:

postmap /etc/postfix/virtual_mailboxes

You can access this mappings in the Postfix configuration file by including the following line:

virtual_mailbox_maps = hash:/etc/postfix/virtual_mailboxes

In our setup, we will replace this text mapping files with MySQL tables. Our intention is to make data handling lot more flexible, robust, and scalable. Since database tables can and usually contain more than just two columns you will need to tell Postfix which database column is meant to be the LHS and which is the RHS. This is accomplished by creating a configuration file which will look something like this:

user = postfix
password = YJiNLQtubgnOE
hosts = 127.0.0.1
dbname = postfix
table = mailbox
select_field = maildir
where_field = username
#additional_conditions = and active = '1'

For the purposes of this discussion, lets assume this is saved in a file called /etc/postfix/mysql_virtual_mailbox_maps.cf. You would then be able to use this mapping in postfix using the following entry in main.cf file.

virtual_mailbox_maps = mysql:/etc/postfix/mysql_virtual_mailbox_maps.cf 

The fields in this configuration file are the user that needs to connect to the MySQL database, password of that MySQL user, dbname, the name of the MySQL database, table ,the name of the table in MySQL database and hosts, the name of the server that MySQL runs on.

Postfix uses the this configuration file as a guide on how to use the database table as a mapping similar to the mapping file with two fields described above. The LHS of the mapping is defined as where_field and the RHS is defined as select_field. In this example we will map the maildir column to the username column. Using the configuation, Postfix constructs a SQL query something like select maildir from postfix.mailbox where username='johndoe' to lookup the maildir for a given username. The following table breaks this distiction out:

LHS

RHS

where_field

select_field

username

maildir

How Postfix Virtual Domains Work?

Understanding how virtual domains work is very important to understand how our virtual mail setup works.

There are two types of domains in Postfix.

  • Local domains: All domains which are listed as mydestination are treated as local domains by Postfix. Emails for local domains are delivered to system users which are listed in /etc/passwd file and these mails are spooled in /var/mail directory.

  • Virtual domains: In addition to the local domains Postfix can deliver mails for virtual domains. Not like local domains, Postfix will let us to handle virtual domains in a very flexible manner. The good thing is with virtual domains is that we do not need system account in /etc/passwd for each and every mail account. This provides us a way to handle thousands of mail accounts very easily in our mail server system. The mapping which we discussed above is used to handle mail account information. You can use MySQL, PostgreSQL , or LDAP for the user account management. In our setup we use MySQL backend to manage user accounts of virtual domains.

    Postfix handles virtual domains as two different categories and need to understand how they work.

    • Virtual alias domains: These domains are used to forward or alias mails from one email address to another email address. Such domains can be used receive mailboxes and store on your hard disk. You do not necessarily need to use virtual alias domains in your setup. Instead we can user virtual_alias_maps even if the domains are not listed as virtual alias domains. The virtual_alias_maps is a general-purpose redirection mapping that works for everything that passes your system including local domains.

    • Virtual mailbox domains: Postfix uses these domains to receive mails for users under the domains and store them in the mailboxes on the hard disk. This is parameter which will simply tell Postfix to receive the mails on behalf of the domain and store them in each user's mailbox. The virtual_mailbox_maps mapping is used by Postfix to determine the location of the mailbox on your hard disk. Please remember that you can still use the virtual_alias_maps mapping to forward email to other mailboxes or external email addresses so not every user on that domain must actually have a mailbox but can also just have the email forwarded somewhere else.

IconHint.png

It is important to understand that a domain is either a virtual alias domain or a virtual mailbox domain or a local domain. If you make a domain a virtual alias domain you will not be able to receive email for that domain on your server. On the contrary you can use the virtual_alias_maps to forward/alias email for both types of domain. So the virtual mailbox domains are generally the more flexible to use.

IconWarning.png

A domain can either be virtual or local and you can not use one domain in the both of these roles. Please never both. So if you decide you want your default domain be a virtual domain then remove it from the mydestination definition. Just leave it blank or set it to mydestination=localhost. Email addresses like root@localhost would then be delivered to the local root user.

Installing a Ubuntu Server System

In this howto I assume that you already know how to setup a Ubuntu server system. I would suggest you to do a Ubuntu server installation without a GUI. If you want to know more about installation look [WWW] here

DNS Setup

This how to assumes that you already have at least one valid registered domain with an MX record setup for that domain. Setting up a BIND DNS server can be learned [WWW] here

Package Installation and Configuration

You will to have install and configure all packages which I have mentioned in the "Package Required" section. Let's deal with one by one.

Postfix Base Server Installation

Here we install Postfix and other packages which Postfix requires connect with a MySQL backend.

Installing Postfix

When you install Postfix I would suggest to select "Internet Site" option.

To install Postfix, you need to install the postfix package. For Postfix documentation, you need the postfix-doc package.

Installing MySQL map support for Postfix

We need to add MySQL mapping support for Postfix. Hope that you have already read How Postfix Mappings Work? section in this howto.

To install postfix-mysql, install the postfix-mysql package.

To install MySQL client program, install the mysql-client package.

To install MySQL server, install the mysql-server package.

Installing Packages for Client Access and Authentication

We will need to offer our user IMAP/POP3 access. Our setup will offer those using the following packages.

To provide client authentication, install the courier-authdaemon package.

To add MySQL support for courier-authdaemon, install the courier-authmysql package. On Ubuntu 7.10, courier-authmysql seems deprecated, install courier-authlib-mysql instead.

To provide unencrypted POP3 access, install the courier-pop package.

To provide SSL-encrypted POP3 access, install the courier-pop-ssl package.

To provide unencrypted IMAP access, install the courier-imap package.

To provide SSL-encrypted IMAP access , install the courier-imap-ssl package.

Installing package for SMTP authentication

Our system will allow road-warriors to send email through our server using authenticated SMTP. This will basically stop unauthorized relaying through our mail server. Not only we are authenticating our user's when they are retrieving mails but also we authenticate them when they are sending mails as well.

To provide encrypted authenticated SMTP, install the postfix-tls package.

This may have already installed with postfix. If so,leave it.

To install Cyrus SASL library, install the libsasl2 package.

To add authentication mechanisms for the SASL library, install the libsasl2-modules package.

To add MySQL support authentication mechanisms with the SASL library, install the libsasl2-modules-sql package.

To create certificates, install the openssl package.

Setting MySQL Backend

Now that we have installed packages required to prepare a MySQL backend for Postfix. In this section we will configure MySQL database for Postfix.

Setting MySQL root Password

Please note that MySQL root is different from Linux root account. It is the account which which users can use to get administrative previlege with MySQL database sever. Initial installation of MySQL has no password fro root account. If you have not setup a password for MySQL root yet, this is the time for you to do so. There are few ways to do the same and I will present two of them here. Please user either one of them.

To set MySQL root in the command prompt:

$ sudo mysqladmin -u root password rootpassword

or you can use the following steps to do the same

$ mysql -u root

Then in the mysql prompt type:

mysql> SET PASSWORD FOR 'root'@'localhost' = PASSWORD('rootpassword');

Don't forget set a good password rootpassword

Setting MySQL Database for Impatient Users

The schema is based on the postfixadmin MySQL schema. If you are impatient you can driectly create the tables using following steps.

$ editor postfixadmin-mysql.sql

Then copy and paste the following script to the above file and save it. Replace postfixpassword on the INSERT INTO user line with a password of your choosing for the postfix user. Do the same for the postfixadmin password.

To create the database, type in a terminal.

$ mysql -u root -p < postfixadmin-mysql.sql
#------------------------------------Start copy------------------------------------- 
#
# Postfix Admin
# by Mischa Peters <mischa at high5 dot net>
# Copyright (c) 2002 - 2005 High5!
# License Info: http://www.postfixadmin.com/?file=LICENSE.TXT
#

# This is the complete MySQL database structure for Postfix Admin.
# If you are installing from scratch you can use this file otherwise you
# need to use the TABLE_CHANGES.TXT or TABLE_BACKUP_MX.TXT that comes with Postfix Admin.
#
# There are 2 entries for a database user in the file.
# One you can use for Postfix and one for Postfix Admin.
#
# If you run this file twice (2x) you will get an error on the user creation in MySQL.
# To go around this you can either comment the lines below "USE MySQL" until "USE postfix".
# Or you can remove the users from the database and run it again.
#
# You can create the database from the shell with:
#
# mysql -u root [-p] < DATABASE_MYSQL.TXT

#
# Postfix / MySQL
#
CREATE DATABASE postfix;

GRANT SELECT ON postfix.* TO postfix@localhost IDENTIFIED BY 'postfixpassword';
GRANT SELECT, INSERT, DELETE, UPDATE ON postfix.* TO postfixadmin@localhost IDENTIFIED BY 'postfixadmin';

USE postfix;
#
# Table structure for table admin
#
CREATE TABLE admin (
username varchar(255) NOT NULL default '',
password varchar(255) NOT NULL default '',
created datetime NOT NULL default '0000-00-00 00:00:00',
modified datetime NOT NULL default '0000-00-00 00:00:00',
active tinyint(1) NOT NULL default '1',
PRIMARY KEY (username),
KEY username (username)
) COMMENT='Postfix Admin - Virtual Admins';

#
# Table structure for table alias
#
CREATE TABLE alias (
address varchar(255) NOT NULL default '',
goto text NOT NULL,
domain varchar(255) NOT NULL default '',
created datetime NOT NULL default '0000-00-00 00:00:00',
modified datetime NOT NULL default '0000-00-00 00:00:00',
active tinyint(1) NOT NULL default '1',
PRIMARY KEY (address),
KEY address (address)
) COMMENT='Postfix Admin - Virtual Aliases';

#
# Table structure for table domain
#
CREATE TABLE domain (
domain varchar(255) NOT NULL default '',
description varchar(255) NOT NULL default '',
aliases int(10) NOT NULL default '0',
mailboxes int(10) NOT NULL default '0',
maxquota int(10) NOT NULL default '0',
transport varchar(255) default NULL,
backupmx tinyint(1) NOT NULL default '0',
created datetime NOT NULL default '0000-00-00 00:00:00',
modified datetime NOT NULL default '0000-00-00 00:00:00',
active tinyint(1) NOT NULL default '1',
PRIMARY KEY (domain),
KEY domain (domain)
) COMMENT='Postfix Admin - Virtual Domains';

#
# Table structure for table domain_admins
#
CREATE TABLE domain_admins (
username varchar(255) NOT NULL default '',
domain varchar(255) NOT NULL default '',
created datetime NOT NULL default '0000-00-00 00:00:00',
active tinyint(1) NOT NULL default '1',
KEY username (username)
) COMMENT='Postfix Admin - Domain Admins';

#
# Table structure for table log
#
CREATE TABLE log (
timestamp datetime NOT NULL default '0000-00-00 00:00:00',
username varchar(255) NOT NULL default '',
domain varchar(255) NOT NULL default '',
action varchar(255) NOT NULL default '',
data varchar(255) NOT NULL default '',
KEY timestamp (timestamp)
) COMMENT='Postfix Admin - Log';

#
# Table structure for table mailbox
#
CREATE TABLE mailbox (
username varchar(255) NOT NULL default '',
password varchar(255) NOT NULL default '',
name varchar(255) NOT NULL default '',
maildir varchar(255) NOT NULL default '',
quota int(10) NOT NULL default '0',
domain varchar(255) NOT NULL default '',
created datetime NOT NULL default '0000-00-00 00:00:00',
modified datetime NOT NULL default '0000-00-00 00:00:00',
active tinyint(1) NOT NULL default '1',
PRIMARY KEY (username),
KEY username (username)
) COMMENT='Postfix Admin - Virtual Mailboxes';

#
# Table structure for table vacation
#
CREATE TABLE vacation (
email varchar(255) NOT NULL default '',
subject varchar(255) NOT NULL default '',
body text NOT NULL,
cache text NOT NULL,
domain varchar(255) NOT NULL default '',
created datetime NOT NULL default '0000-00-00 00:00:00',
active tinyint(1) NOT NULL default '1',
PRIMARY KEY (email),
KEY email (email)
) COMMENT='Postfix Admin - Virtual Vacation';
#------------------------------------End copy-------------------------------------

Step by Step Database Setup

The users who are wish to setup database step by step can use the following steps and understand what table is using what purpose.

Setting up Database, Users, and Privileges

Connect to MySQL database as root

$ mysql -u root -p

Then execute the following SQL commands.

To create and use the database:

mysql> CREATE DATABASE postfix;
mysql> USE postfix;

To create Postfix user & set password (replace postfixpassword with a password of your choosing):

mysql> GRANT SELECT ON postfix.* TO postfix@localhost IDENTIFIED BY 'postfixpassword';

To create Postfix Admin user & set password (replace postfixadmin with a password of your choosing):

mysql> GRANT SELECT, INSERT, DELETE, UPDATE ON postfix.* TO postfixadmin@localhost IDENTIFIED BY 'postfixadmin';

Create the Table Admin

This table is used create the administrators for our virtual mail system. The admin user will be able create, modify, and delete virtadomain administrators, mailboxes and other administrative tasks in the mail system. Postfix is not using this table.

Copy and paste the sql statement to your mysql> prompt.

CREATE TABLE admin (
username varchar(255) NOT NULL default '',
password varchar(255) NOT NULL default '',
created datetime NOT NULL default '0000-00-00 00:00:00',
modified datetime NOT NULL default '0000-00-00 00:00:00',
active tinyint(1) NOT NULL default '1',
PRIMARY KEY (username),
KEY username (username)
) COMMENT='Postfix Admin - Virtual Admins';

Create the Alias table

Postfix is using the "address" and "goto" column. Courier is not using this table.

IconHint.png

This table can be used for virtual .forward files. This table is nothing more than /etc/aliases that you will find on any *nix OS. Multiple destination email addresses need to be separated by a "," (comma).

Following is the table structure for table alias. Copy and paste the sql statement to your mysql> prompt

CREATE TABLE alias (
address varchar(255) NOT NULL default '',
goto text NOT NULL,
domain varchar(255) NOT NULL default '',
created datetime NOT NULL default '0000-00-00 00:00:00',
modified datetime NOT NULL default '0000-00-00 00:00:00',
active tinyint(1) NOT NULL default '1',
PRIMARY KEY (address),
KEY address (address)
) COMMENT='Postfix Admin - Virtual Aliases';

Create the Domain table

Postfix is using the "domain" and "description" column. Courier is not using this table.

Copy and paste the sql statement to your mysql> prompt

CREATE TABLE domain (
domain varchar(255) NOT NULL default '',
description varchar(255) NOT NULL default '',
aliases int(10) NOT NULL default '0',
mailboxes int(10) NOT NULL default '0',
maxquota int(10) NOT NULL default '0',
transport varchar(255) default NULL,
backupmx tinyint(1) NOT NULL default '0',
created datetime NOT NULL default '0000-00-00 00:00:00',
modified datetime NOT NULL default '0000-00-00 00:00:00',
active tinyint(1) NOT NULL default '1',
PRIMARY KEY (domain),
KEY domain (domain)
) COMMENT='Postfix Admin - Virtual Domains';

Create the Domain Admin Table

Table structure for table domain_admins. This table is used to create individual administrators for each virtual domain. Postfix or Courier is not using this table.

Copy and paste the sql statement to your mysql> prompt.

CREATE TABLE domain_admins (
username varchar(255) NOT NULL default '',
domain varchar(255) NOT NULL default '',
created datetime NOT NULL default '0000-00-00 00:00:00',
active tinyint(1) NOT NULL default '1',
KEY username (username)
) COMMENT='Postfix Admin - Domain Admins';

Create the Mailbox Table

Postfix is using the "username" and "maildir" column while Courier is using the "username", "password", "name" and "maildir" column.

Copy and paste the sql statement to your mysql> prompt.

CREATE TABLE mailbox (
username varchar(255) NOT NULL default '',
password varchar(255) NOT NULL default '',
name varchar(255) NOT NULL default '',
maildir varchar(255) NOT NULL default '',
quota int(10) NOT NULL default '0',
domain varchar(255) NOT NULL default '',
created datetime NOT NULL default '0000-00-00 00:00:00',
modified datetime NOT NULL default '0000-00-00 00:00:00',
active tinyint(1) NOT NULL default '1',
PRIMARY KEY (username),
KEY username (username)
) COMMENT='Postfix Admin - Virtual Mailboxes';

Create the Log Table

Postfix or Courier is not using this table. Instead this table is used to log the activities of domain administrators and mailbox users.

Copy and paste the sql statement to your mysql> prompt.

CREATE TABLE log (
timestamp datetime NOT NULL default '0000-00-00 00:00:00',
username varchar(255) NOT NULL default '',
domain varchar(255) NOT NULL default '',
action varchar(255) NOT NULL default '',
data varchar(255) NOT NULL default '',
KEY timestamp (timestamp)
) COMMENT='Postfix Admin - Log';

Create the Vacation Table

Virual Vacation is done with a local shell account that can receive email. The email is then handled by a Perl script which sends the Vacation message back to the sender.

Copy and paste the sql statement to your mysql> prompt.

CREATE TABLE vacation (
email varchar(255) NOT NULL default '',
subject varchar(255) NOT NULL default '',
body text NOT NULL,
cache text NOT NULL,
domain varchar(255) NOT NULL default '',
created datetime NOT NULL default '0000-00-00 00:00:00',
active tinyint(1) NOT NULL default '1',
PRIMARY KEY (email),
KEY email (email)
) COMMENT='Postfix Admin - Virtual Vacation';

Setting Postfix MySQL Maps

As specified earlier in this document you need to tell Postfix where the control information is stored in the database. You need to create the following four text files in /etc/postfix for that reason.

Note that in the files we create below, we specify 127.0.0.1 for the hosts field instead of localhost. This is because Postfix is run in a chroot environment, and if you specify localhost Postfix will try to connct to the MySQL deamon using a unix socket in the directory /var/run/mysql, to which it will not have access. Using 127.0.0.1 forces Postfix to connect using a TCP/IP socket, which will work in the chroot environment.

IconWarning.png

In all files below we use postfixpassword for the password field. You will want to change this password to one of your own choosing for security reasons. You will also not want to post these configuration files when asking for help on the forums without changing the password field for obvious reasons.

Creating Virtual Alias Maps

Postfix will use this file for Virtual Alias Maps and it will use The LHS of the mapping is defined as where_field and the RHS is defined as select_field. In this file it would be a mapping of the address column to the goto column.

$ sudo editor /etc/postfix/mysql_virtual_alias_maps.cf

Then add the following code segment to the above file.

user = postfix
password = postfixpassword
hosts = 127.0.0.1
dbname = postfix
table = alias
select_field = goto
where_field = address

Virtual Domain Maps

Posfix is only using domain field from this table. For domains we do not need to map LHS and RHS.

$ sudo editor /etc/postfix/mysql_virtual_domains_maps.cf

Then add the following code segment to the above file.

user = postfix
password = postfixpassword
hosts = 127.0.0.1
dbname = postfix
table = domain
select_field = domain
where_field = domain
#additional_conditions = and backupmx = '0' and active = '1'

Virtual Mailbox Maps

Postfix will map username column with maildir querying mailbox table.

$ sudo editor /etc/postfix/mysql_virtual_mailbox_maps.cf

Then add the following code segment to the above file.

user = postfix
password = postfixpassword
hosts = 127.0.0.1
dbname = postfix
table = mailbox
select_field = maildir
where_field = username
#additional_conditions = and active = '1'

Virtual Mailbox Quota Maps

Postfix will this maps to handle the quota for virtual mailboxes. Username column will be mapped with the quota column in the mailbox table.

$ sudo editor /etc/postfix/mysql_virtual_mailbox_limit_maps.cf

Then add the following code segment to the above file.

user = postfix
password = postfixpassword
hosts = 127.0.0.1
dbname = postfix
table = mailbox
select_field = quota
where_field = username
#additional_conditions = and active = '1'

Relay Domain Maps

If you are going to use your mail system only for hosting backup MX for some virtual domains then you need this mapping to tell the Postfix to enable the relaying for these domains.

$ sudo editor /etc/postfix/mysql_relay_domains_maps.cf

Then add the following code segment to the above file.

user = postfix
password = postfixpassword
hosts = 127.0.0.1
dbname = postfix
table = domain
select_field = domain
where_field = domain
additional_conditions = and backupmx = '1'

IconWarning.png

For security concerns you should let all these to be read by root only. If you let other users in the system to read these files, very body on your system can read the postfix database access password which is written in plain text and find out information about user accounts and alter them in the database. Dangerous!!!.

Execute the following commands to make these file secure from others.

To set the group of these files to postfix:

$ sudo chgrp postfix /etc/postfix/mysql_*.cf

To make the file readable by the group:

$ sudo chmod 640 /etc/postfix/mysql_*.cf

Create a vmail user

Our system can hold mailboxes for thousands of users. All of these users are virtual users and none of them is a Linux system user, hence these users can not store their mail in our system's hard disk. You probably do not want to assign a unique UID (user ID) to every user, so let's create a Linux user who will become the owner of all mailboxes.

$ sudo groupadd -g 5000 vmail
$ sudo useradd -m -g vmail -u 5000 -d /home/vmail -s /bin/bash vmail

Configuring Postfix with MySQL maps

We have already created our MySQL maps config files and now the time is to setup Postfix main.cf file so that Postfix can query MySQL database for virtual mailboxes and domains.

Open the main.cf file.

$ sudo editor /etc/postfix/main.cf

Then add the following code segment to main.cf

# Virtual Mailbox Domain Settings

virtual_alias_maps = mysql:/etc/postfix/mysql_virtual_alias_maps.cf
virtual_mailbox_domains = mysql:/etc/postfix/mysql_virtual_domains_maps.cf
virtual_mailbox_maps = mysql:/etc/postfix/mysql_virtual_mailbox_maps.cf
virtual_mailbox_limit = 51200000
virtual_minimum_uid = 5000
virtual_uid_maps = static:5000
virtual_gid_maps = static:5000
virtual_mailbox_base = /home/vmail
virtual_transport = virtual

# Additional for quota support

virtual_create_maildirsize = yes
virtual_mailbox_extended = yes
virtual_mailbox_limit_maps = mysql:/etc/postfix/mysql_virtual_mailbox_limit_maps.cf
virtual_mailbox_limit_override = yes
virtual_maildir_limit_message = Sorry, the your maildir has overdrawn your diskspace quota, please free up some of spaces of your mailbox try again.
virtual_overquota_bounce = yes

Setting up Postfix

Postfix has several hundred configuration parameters that are controlled via the main.cf file. Fortunately, all parameters have sensible default values. We only have to define the following parameters

$ sudo editor /etc/postfix/main.cf
#The host name where your MX for virtual domains will point to
myhostname = mail.domain.com
mydestination = #Remains blank since we are going to host virtual domains
relayhost = #Remains blank unless you are going to use your ISP's SMTP server mail sending out mails. In which case it would be set to the host name of the ISP's SMTP server

Leave the following to their default values

alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname
mynetworks = all
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all

Enhanced Mail Services

Postfixadmin

dpkg -i postfixadmin_2.2.0-1rc1_all.deb  
  • Warning: this package does not contains mysql scripts. You could use one from the svn version. I hope this will be fixed soon in released 2.2.

  • setup databases (warning: it could overlapped databases created manualy before)

    • do not forget to change passwords in DATABASE_MYSQL.TXT

 cat DATABASE_MYSQL.TXT|mysql -u root -p
  • restart apache2

/etc/init.d/apache2 restart
  • edit /usr/share/postfixadmin/config.inc.php and setup database password and set the flag "configured"=true

  • browse to [WWW] http://localhost/postfixadmin

  • now you can add domains and mailboxes

**TODO** rewrite this part with release 2.2

Courier-IMAP and Authentication Services

  • In /etc/courier/authmysqlrc

MYSQL_SERVER 127.0.0.1
MYSQL_USERNAME postfix
MYSQL_PASSWORD thepassword
MYSQL_DATABASE postfix
MYSQL_USER_TABLE mailbox
MYSQL_LOGIN_FIELD username
MYSQL_NAME_FIELD name
MYSQL_CRYPT_PWFIELD password
#MYSQL_CLEAR_PWFIELD password
MYSQL_MAILDIR_FIELD maildir
MYSQL_QUOTA_FIELD concat(quota,'S')
MYSQL_HOME_FIELD '/home/vmail'
MYSQL_UID_FIELD '5000'
MYSQL_GID_FIELD '5000'
  • Restart daemons and checks logs:

 /etc/init.d/courier-authdaemon restart ; /etc/init.d/courier-imap restart; /etc/init.d/courier-pop restart
tail -f /var/log/mail*

SMTP Authentication

A good documentation : [WWW] http://www.starbridge.org/spip/spip.php?article1

In /etc/postfix/main.cf add

smtpd_recipient_restrictions = reject_unauth_pipelining, permit_mynetworks, permit_sasl_authenticated, reject_non_fqdn_recipient, reject_unauth_destination, check_policy_service inet
# modify the existing smtpd_sender_restrictions
smtpd_sender_restrictions = permit_sasl_authenticated, permit_mynetworks, reject_non_fqdn_sender, reject_unknown_sender_domain, reject_unauth_pipelining, permit
# then add these
smtpd_sasl_auth_enable = yes
broken_sasl_auth_clients = yes
smtpd_sasl_path = /etc/postfix/sasl:/usr/lib/sasl2
smtpd_sasl_security_options = noanonymous
smtpd_sasl_local_domain =

In /etc/postfix/sasl/smtpd.conf

pwcheck_method: auxprop
auxprop_plugin: sql
mech_list: plain login cram-md5 digest-md5
sql_engine: mysql
sql_hostnames: 127.0.0.1
sql_user: postfix
sql_passwd: yourpassword
sql_database: postfix
sql_select: select passwd from mailbox where username='%u@%r' and active = 1

Web Access

SquirrelMail is a standards-based webmail package written in PHP. It includes built-in pure PHP support for the IMAP and SMTP protocols, and all pages render in pure HTML 4.0 (with no JavaScript required) for maximum compatibility across browsers. It has very few requirements and is very easy to configure and install. SquirrelMail has all the functionality you would want from an email client, including strong MIME support, address books, and folder manipulation. SUORCE: SquirrelMail home page

$ sudo apt-get install squirrelmail

Refining the Setup

Anti-Spam Configuration

Installing Amavisd and Spamassassin

Quarantine and Spam Management

Auto and Per-Recipient White/Black Lists

Amavis/Spamassassin UI

GreyListing

Distributed and Collaborative Networks

Anti-Virus Configuration

Configuring for ClamAV

Log Analyzer

Install and Configure AWStats

Wrapping it Up

Final Changes and Troubleshooting

Other links

Until this article will be finished, you can find very similiar one here:

Howto created by: ChinthakaDeshapriya.