Difference between pages "MIMEDefang" and "MySQL"

From Wikislax
(Difference between pages)
Jump to: navigation, search
(Created page with "{{RightTOC}} == What is MIMEDefang ? == [http://www.mimedefang.org MIMEDefang] is a plugin specific to the Sendmail Milter interface and affords scanning modifying f...")
 
(Created page with "{{RightTOC}} == What is MySQL ? == [http://www.mysql.com MySQL] is the traditional Database software companion to Apache and PHP. MySQL was previously a property of MySQL AB...")
 
Line 1: Line 1:
 
{{RightTOC}}
 
{{RightTOC}}
  
== What is MIMEDefang ? ==
+
== What is MySQL ? ==
  
[http://www.mimedefang.org MIMEDefang] is a plugin specific to the [[Sendmail]] [[Milter]] interface and affords scanning modifying filtering or bouncing mails while they are being received by sendmail.
+
[http://www.mysql.com MySQL] is the traditional Database software companion to Apache and PHP. MySQL was previously a property of MySQL AB but was acquired by Oracle Corporation in 2010. This created some worries in the OpenSource community, which led to create a fork, [https://mariadb.org/ MariaDB], which is a plug-in replacement for MySQL and the one included in the Slackware distribution.
  
MIMEDefang consist of four major components: mimedefang, mimedefang-multiplexor, mimedefang.pl, and mimedefang-filter. MIMEDefang, written in C, splits incoming messages into parts. Mimedefang-multiplexor creates and manages a pool of perl processes that execute mimedefang.pl, a perl script.
+
== Installing MySQL ==
  
== Installing MIMEDefang ==
+
The quickest way to install MySQL is to NOT install it and use instead [https://mariadb.org/ MariaDB], included with Slackware. If this is your choice, please note that '''/usr''' will be the root directory of the software instead of '''/usr/local/mysql''', and proceed to [[MySQL#Running SQL|Running SQL]].
  
MIMEDefang requires a few perl modules that can be installed as below :
+
'''apr''' and '''apr-util''', used by apache, refer to MySQL so the latter must be installed first. [http://dev.mysql.com/downloads/mysql/ Download] the latest stable version, choosing "Source Code" when asked to Select platform. [http://dev.mysql.com/doc/ Documentation] is available from the web site. Here MySQL is compiled with OpenSSL support to be able to encrypt MySQL connections on port 3306 (just in case, as we don't intend to open port 3306 to the external world at the moment).
  
  # perl -MCPAN -e shell
+
  # groupadd mysql
  . . .
+
  # useradd -g mysql mysql
cpan> install Digest::SHA1
+
  # tar -C /usr/local -xvf mysql-x.y.z.tar.gz
cpan> install IO::Stringy
+
  # cd /usr/local/mysql-x.y.z
cpan> install MailTool
+
  # chown -R root:root .
cpan> install MIME::Tools
+
  # less INSTALL-SOURCE
cpan> install MIME::Base64
+
# rm CMakeCache.txt
cpan> install Unix::Syslog
+
  # make clean
cpan> install Net::DNSBL::Client
+
  # cmake . -DCMAKE_INSTALL_PREFIX=/usr/local/mysql -DINSTALL_LIBDIR=lib64 \
quit
+
  -DINSTALL_PLUGINDIR=lib64/plugin -DMYSQL_DATADIR=/var/lib/mysql \
 
+
-DDEFAULT_CHARSET=latin1 -DDEFAULT_COLLATION=latin1_swedish_ci \
[http://www.mimedefang.org/download Download MIMEDefang] untar and install as below. Mickey Hill's [http://www.mickeyhill.com/mimedefang-howto MIMEDefang HOWTO] includes detailed installation information and tips.
+
-DMYSQL_UNIX_ADDR=/var/run/mysql/mysql.sock -DWITH_SSL=yes
 
 
  # tar -C /usr/local -xvf mimedefang-x.y.tar.gz
 
  # cd /usr/local
 
  # chown -R root:root mimedefang-x.y
 
  # cd mimedefang-x.y
 
  # ./configure --help | less
 
  # ./configure --libdir=/usr/local/lib64 \
 
  --mandir=/usr/local/man --with-user=milter
 
 
  # make
 
  # make
 +
# removepkg /var/log/packages/mysql-5.1.46-x86_64-2
 
  # make install
 
  # make install
 
  # make clean
 
  # make clean
 +
# cp -R /usr/local/mysql/man /usr/local
 +
# cp support-files/my-default.cnf /etc/my.cnf
 +
# chmod u+x scripts/mysql_install_db
 +
# scripts/mysql_install_db --user=mysql --basedir=/usr/local/mysql --datadir=/var/lib/mysql
  
== Configuring MIMEDefang ==
+
== Configuring MySQL ==
  
MIMEDefang uses a single configuration file in '''/etc/mail/sa-mimedefang.cf'''. Add the lines below for DCC Pyzor and Bayes. The '''bayes_path''' seems required for SpamAssassin to find the Bayes database :
+
MySQL ('''5.6.24''') works correctly out of the box so there is nothing to do really. In the past we had to edit '''/etc/my.cnf''' to uncomment line '''#skip-networking''' to prevent MySQL network connections. We also had to include in this file, after the '''[client]''' tag a line '''default-character-set=''charset_name''''' (for instance '''utf8''' or '''latin1''') to eliminate this warning when launching a client :
  
use_dcc 1
+
  # mysql -u root -p
dcc_timeout 10
+
  Enter password:
dcc_path /usr/local/bin/dccproc
+
  '''mysql: Unknown OS character set 'ISO-8859-15'.'''
+
  '''mysql: Switching to the default character set 'latin1'.'''
use_pyzor 1
+
  Welcome to the MySQL monitor. Commands end with ; or \g.
pyzor_timeout 10
+
  . . .
pyzor_path /usr/local/bin/pyzor
 
 
# Set Bayes parameters
 
  #
 
  use_bayes 1
 
  use_learner 1
 
use_bayes_rules 1
 
bayes_auto_learn 1
 
bayes_auto_expire 1
 
bayes_file_mode 700
 
bayes_expiry_max_db_size 150000
 
bayes_path /var/spool/MD-Quarantine/.spamassassin/bayes
 
   
 
  #  Set headers which may provide inappropriate cues to the Bayesian classifier
 
  #
 
  bayes_ignore_header X-Scanned-By
 
bayes_ignore_header X-Spam-Check
 
bayes_ignore_header X-Spam-Score
 
bayes_ignore_header X-Probably-Spam-Tag
 
  
'''/etc/mail/mimedefang-filter''' is a perl fragment read by mimedefang.pl where customization can take place. Be sure to check '''man mimedefang-filter'''. Global variable $SALocalTestsOnly has a default value of 1 to to disable Receive, RBL and Razor tests. Add line $SALocalTestsOnly = 0; at the beginning of mimedefang-filter to enable these tests. The code snipet below extracted from '''filter_end''' in mimedefang-filter calls for a few interesting comments:
+
This is no longer necessary, so let us directly update '''/etc/rc.d/rc.mysqld''' with new paths and options :
  
* Messages of 100K or more are unlikely to be spam so are not scanned to avoid a waste of resources.
+
# Start mysqld:
* The number of stars in X-Spam-Score is easy to use in your MUA rules to classify mail as ham or spam.
+
mysqld_start() {
* We disabled the spamassassin report to collect spam unmodified for the Bayes module initialization.
+
  if [ -x /usr'''/local/mysql'''/bin/mysqld_safe ]; then
* We added line action_sm_quarantine("$hits ($score) $names"); to test the Sendmail quarantine feature.
+
    # If there is an old PID file (no mysqld running), clean it up:
 
+
    if [ -r /var/run/mysql/mysql.pid ]; then
    # Spam checks if SpamAssassin is installed
+
      if ! ps axc | grep mysqld 1> /dev/null 2> /dev/null ; then
    if ($Features{"SpamAssassin"}) {
+
        echo "Cleaning up old /var/run/mysql/mysql.pid."
        if (-s "./INPUTMSG" < 100*1024) {
+
        rm -f /var/run/mysql/mysql.pid
            # Only scan messages smaller than 100kB.  Larger messages
+
      fi
            # are extremely unlikely to be spam, and SpamAssassin is
+
    fi
            # dreadfully slow on very large messages.
+
    /usr'''/local/mysql'''/bin/mysqld_safe '''--character_set_server=utf8''' \
            my($hits, $req, $names, $report) = spam_assassin_check();
+
        '''--basedir=/usr/local/mysql''' --datadir=/var/lib/mysql \
            my($score);
+
         --pid-file=/var/run/mysql/mysql.pid $SKIP &
            if ($hits < 40) {
+
  fi
                $score = "*" x int($hits);
+
  }
            } else {
 
                $score = "*" x 40;
 
            }
 
            # We add a header which looks like this:
 
            # X-Spam-Score: 6.8 (******) NAME_OF_TEST,NAME_OF_TEST
 
            # The number of asterisks in parens is the integer part
 
            # of the spam score clamped to a maximum of 40.
 
            # MUA filters can easily be written to trigger on a
 
            # minimum number of asterisks...
 
            if ($hits >= $req) {
 
                action_change_header("X-Spam-Score", "$hits ($score) $names");
 
                md_graphdefang_log('spam', $hits, $RelayAddr);
 
                # If you find the SA report useful, add it, I guess...
 
                action_add_part($entity, "text/plain", "-suggest",
 
                                "$report\n",
 
                              "SpamAssassinReport.txt", "inline");
 
                # action_quarantine_entire_message("$hits ($score) $names");
 
                # action_sm_quarantine("$hits ($score) $names");
 
                # send_quarantine_notifications();
 
                # action_discard();
 
            } else {
 
                # Delete any existing X-Spam-Score header?
 
                action_delete_header("X-Spam-Score");
 
            }
 
        }
 
    }
 
 
 
There is nothing special to configure in Sendmail to use the quarantine feature. '''mailq -qQ''' affords viewing the content of the quarantine queue, '''sendmail -qQ -qI<u>substr</u> -Q''' affords unquarantining a specific mail.
 
 
 
<u>Note</u> : if anything goes wrong it is possible to get all SpamAssassin debug information in the maillog file by adding a '''debug => 1,''' parameter when creating the SpamAssassin object in function spam_assassin_init of '''/usr/local/bin/mimedefang.pl''' :
 
 
 
#***********************************************************************
 
# %PROCEDURE: spam_assassin_init
 
# %ARGUMENTS:
 
#  config -- optional spamassassin config file
 
# %RETURNS:
 
#  A Mail::SpamAssassin object.
 
# %DESCRIPTION:
 
#  Scans message using SpamAssassin (http://www.spamassassin.org)
 
#***********************************************************************
 
sub spam_assassin_init (;$) {
 
    my($config) = @_;
 
    my $LOCAL_RULES_DIR = '/etc/mail/spamassassin';
 
    my $LOCAL_STATE_DIR = '/var/lib';
 
 
    . . .
 
 
         my $sa_args = {
 
                '''debug              => 1,'''
 
                local_tests_only  => $SALocalTestsOnly,
 
                dont_copy_prefs    => 1,
 
                userprefs_filename => $config,
 
                user_dir          => $Features{'Path:QUARANTINEDIR'},
 
        };
 
 
 
== Configuring Milter ==
 
 
 
Add the '''INPUT_MAIL_FILTER''' macro to sendmail.mc to send all mails trough the specified filter :
 
 
 
# cd /usr/local/sendmail-*/cf/cf
 
# vi sendmail.mc
 
a
 
INPUT_MAIL_FILTER(`mimedefang', `S=unix:/var/spool/MIMEDefang/mimedefang.sock, F=T, T=S:5m;R:5m')dnl
 
<esc>
 
  :x
 
# m4 ../m4/cf.m4 sendmail.mc > sendmail.cf
 
# cp sendmail.mc /etc/mail
 
# cp sendmail.cf /etc/mail
 
 
 
== Reconfiguring rc.sendmail ==
 
  
<u>MIMEDefang must be started before and stopped after sendmail</u>. Copy the init-script provided under /etc/rc.d then call it from rc.sendmail.
+
== Running SQL ==
  
# '''cp examples/init-script /etc/rc.d/rc.mimedefang'''
+
Give execution rights to the rc.mysqld script and launch the daemon :
# '''vi /etc/rc.d/rc.sendmail'''
 
. . .
 
# Start sendmail:
 
sendmail_start() {
 
  /etc/rc.d/rc.mimedefang start
 
  sleep 2
 
  if [ -x /usr/sbin/sendmail ]; then
 
    echo "Starting sendmail MTA daemon: /usr/sbin/sendmail -L sm-mta -bd -q25m"
 
    /usr/sbin/sendmail -L sm-mta -bd -q25m
 
    echo "Starting sendmail MSP queue runner:  /usr/sbin/sendmail -L sm-msp-queue -Ac -q25m"
 
    /usr/sbin/sendmail -L sm-msp-queue -Ac -q25m
 
  fi
 
}
 
 
# Stop sendmail:
 
sendmail_stop() {
 
  /etc/rc.d/rc.mimedefang stop
 
  killall sendmail
 
  sleep 6
 
}
 
. . .
 
<esc>
 
:x
 
  
== Running Sendmail with MIMEDefang ==
+
# chmod u+x /etc/rc.d/rc.mysqld
 +
# /etc/rc.d/rc.mysqld start
  
As we have installed and configured SpamAssassin, ClamAV and MIMEDefang previously, all the pieces are now in place and ready to interlock. Just restart sendmail :
+
== Using MySQL ==
  
# /etc/rc.d/rc.sendmail start
+
For a quick introduction on MySQL refer to this [http://www3.ntu.edu.sg/home/ehchua/programming/sql/MySQL_Intermediate.html overview]. Otherwise, if you have time to invest, MySQL comes with a very exhaustive [http://dev.mysql.com/doc documentation], including a tutorial. A few quick notes however :
Starting mimedefang-multiplexor:                            [  OK  ]
 
Starting mimedefang:                                        [  OK  ]
 
Starting sendmail MTA daemon:  /usr/sbin/sendmail -L sm-mta -bd -q25m
 
Starting sendmail MSP queue runner:  /usr/sbin/sendmail -L sm-msp-queue -Ac -q25m
 
root@inner:/etc/rc.d#
 
  
Check behaviour in file '''/var/log/maillog'''.
+
Mysql defines a root user without an initial password. '''mysqladmin''' makes is possible to specify one by typing '''/usr(/local/mysql)/bin/mysqladmin -u root -p password ‘new-password’'''. The old password is then requested. Type return (the old password is empty). From that point, the password will need to be specified for every  client invocation, for example : '''"/usr(/local/mysql)/bin/mysql -u root -p mysql_db"'''. '''"-p"''' with no value means that the password will be requested interactively. '''mysql_db''' is the database name. Not specifying the password on the command line affords avoiding potential disclosure to other users using '''ps'''.
  
== A MIMEDefang issue ==
+
You can use '''mysql''' to configure itself. Refer to the [http://dev.mysqm.com/doc/ documentation] for the '''SQL''' syntax, or use [[phpMyAdmin]], a PHP application to manage MySql from your browser.
  
Sometimes the message '''Milter (mimedefang): local socket name /var/spool/MIMEDefang/mimedefang.sock unsafe''' can be found in file '''/var/log/maillog''' for each incoming mail, in this case not processed. As the socket is created automatically by MIMEDefang at run time, the error message itself is mistaken. What actually happens is that the socket is not created at all.
+
To invoke MySQL, use '''mysql -u user''' then enter queries at the mysql> prompt, '''quit''' to terminate the invocation. Commands can be entered uppercase or lowercase. It is possible to enter formulas, for example '''select sin(pi()/4), (4+1)*5, current_date;. "\c"''' at the end of a command affords cancelling.
  
The root cause is unclear. Buggy perl modules have been mentionned on some websites. A list of the perl modules used by MIMEDefang can be obtained by "'''mimedefang.pl -features'''" and upgraded with the '''upgrade''' command of perl. Another likely cause is that MIMEDefang seems to need some time to properly wrap-up on termination. To work around this, we added as above some '''sleep''' commands in file '''/etc/rc.d/rc.sendmail'''.
+
Queries may be entered on several lines. When awaiting input on a new line, mysql displays '''->''' if a command completion is awaited, '''<nowiki>'></nowiki> for a string that began with a '''<nowiki>'</nowiki>, '''“>''' for a string that began with a '''“''', '''`>''' for an identifier that began with a '''`''', '''/*>''' for a comment that began with a '''/*'''.
  
The presence of the socket can be tested with the command :
+
'''show databases;''' to view a list of existing databases. '''use database''' (without a semicolon) to use a specific database. GRANT ALL ON database.* TO 'your_mysql_name'@'your_client_host'; must have been used first by the admin to grant access. '''CREATE DATABASE database;''' to create a database then '''USE database'''.
  
# '''ls -al /var/spool/MIMEDefang/'''
+
Alternatively it is possible to specify the database name as the last parameter on the mysql command eg '''mysql -h host -u user -p database. select database();''' affords viewing which database is in use.
total 16
 
drwxr-x---  2 milter root  4096 Oct 10 14:08 ./
 
drwxr-xr-x 18 root  root  4096 Oct  7 08:22 ../
 
srw-rw-rw-  1 milter milter    0 Oct  9 12:48 clamd.sock=
 
-rw-------  1 milter milter    6 Oct 10 14:08 mimedefang-multiplexor.pid
 
srwx------  1 milter milter    0 Oct 10 14:08 mimedefang-multiplexor.sock=
 
-rw-------  1 milter milter    6 Oct 10 14:08 mimedefang.pid
 
srwx------  1 milter milter    0 Oct 10 14:08 mimedefang.sock=
 
  
<br>
+
<br/>
  
{{pFoot|[[Milter]]|[[Main Page]]|[[MySQL]]}}
+
{{pFoot|[[Milter]]|[[Main Page]]|[[Apache]]}}

Latest revision as of 23:33, 6 December 2017

What is MySQL ?

MySQL is the traditional Database software companion to Apache and PHP. MySQL was previously a property of MySQL AB but was acquired by Oracle Corporation in 2010. This created some worries in the OpenSource community, which led to create a fork, MariaDB, which is a plug-in replacement for MySQL and the one included in the Slackware distribution.

Installing MySQL

The quickest way to install MySQL is to NOT install it and use instead MariaDB, included with Slackware. If this is your choice, please note that /usr will be the root directory of the software instead of /usr/local/mysql, and proceed to Running SQL.

apr and apr-util, used by apache, refer to MySQL so the latter must be installed first. Download the latest stable version, choosing "Source Code" when asked to Select platform. Documentation is available from the web site. Here MySQL is compiled with OpenSSL support to be able to encrypt MySQL connections on port 3306 (just in case, as we don't intend to open port 3306 to the external world at the moment).

# groupadd mysql
# useradd -g mysql mysql
# tar -C /usr/local -xvf mysql-x.y.z.tar.gz
# cd /usr/local/mysql-x.y.z
# chown -R root:root .
# less INSTALL-SOURCE
# rm CMakeCache.txt
# make clean
# cmake . -DCMAKE_INSTALL_PREFIX=/usr/local/mysql -DINSTALL_LIBDIR=lib64 \
-DINSTALL_PLUGINDIR=lib64/plugin -DMYSQL_DATADIR=/var/lib/mysql \
-DDEFAULT_CHARSET=latin1 -DDEFAULT_COLLATION=latin1_swedish_ci \
-DMYSQL_UNIX_ADDR=/var/run/mysql/mysql.sock -DWITH_SSL=yes
# make
# removepkg /var/log/packages/mysql-5.1.46-x86_64-2
# make install
# make clean
# cp -R /usr/local/mysql/man /usr/local
# cp support-files/my-default.cnf /etc/my.cnf
# chmod u+x scripts/mysql_install_db
# scripts/mysql_install_db --user=mysql --basedir=/usr/local/mysql --datadir=/var/lib/mysql

Configuring MySQL

MySQL (5.6.24) works correctly out of the box so there is nothing to do really. In the past we had to edit /etc/my.cnf to uncomment line #skip-networking to prevent MySQL network connections. We also had to include in this file, after the [client] tag a line default-character-set=charset_name (for instance utf8 or latin1) to eliminate this warning when launching a client :

# mysql -u root -p
Enter password: 
mysql: Unknown OS character set 'ISO-8859-15'.
mysql: Switching to the default character set 'latin1'.
Welcome to the MySQL monitor.  Commands end with ; or \g.
. . .

This is no longer necessary, so let us directly update /etc/rc.d/rc.mysqld with new paths and options :

# Start mysqld:
mysqld_start() {
  if [ -x /usr/local/mysql/bin/mysqld_safe ]; then
    # If there is an old PID file (no mysqld running), clean it up:
    if [ -r /var/run/mysql/mysql.pid ]; then
      if ! ps axc | grep mysqld 1> /dev/null 2> /dev/null ; then
        echo "Cleaning up old /var/run/mysql/mysql.pid."
        rm -f /var/run/mysql/mysql.pid
      fi
    fi
    /usr/local/mysql/bin/mysqld_safe --character_set_server=utf8 \
        --basedir=/usr/local/mysql --datadir=/var/lib/mysql \
        --pid-file=/var/run/mysql/mysql.pid $SKIP &
  fi
}

Running SQL

Give execution rights to the rc.mysqld script and launch the daemon :

# chmod u+x /etc/rc.d/rc.mysqld
# /etc/rc.d/rc.mysqld start

Using MySQL

For a quick introduction on MySQL refer to this overview. Otherwise, if you have time to invest, MySQL comes with a very exhaustive documentation, including a tutorial. A few quick notes however :

Mysql defines a root user without an initial password. mysqladmin makes is possible to specify one by typing /usr(/local/mysql)/bin/mysqladmin -u root -p password ‘new-password’. The old password is then requested. Type return (the old password is empty). From that point, the password will need to be specified for every client invocation, for example : "/usr(/local/mysql)/bin/mysql -u root -p mysql_db". "-p" with no value means that the password will be requested interactively. mysql_db is the database name. Not specifying the password on the command line affords avoiding potential disclosure to other users using ps.

You can use mysql to configure itself. Refer to the documentation for the SQL syntax, or use phpMyAdmin, a PHP application to manage MySql from your browser.

To invoke MySQL, use mysql -u user then enter queries at the mysql> prompt, quit to terminate the invocation. Commands can be entered uppercase or lowercase. It is possible to enter formulas, for example select sin(pi()/4), (4+1)*5, current_date;. "\c" at the end of a command affords cancelling.

Queries may be entered on several lines. When awaiting input on a new line, mysql displays -> if a command completion is awaited, '> for a string that began with a ', “> for a string that began with a , `> for an identifier that began with a `, /*> for a comment that began with a /*.

show databases; to view a list of existing databases. use database (without a semicolon) to use a specific database. GRANT ALL ON database.* TO 'your_mysql_name'@'your_client_host'; must have been used first by the admin to grant access. CREATE DATABASE database; to create a database then USE database.

Alternatively it is possible to specify the database name as the last parameter on the mysql command eg mysql -h host -u user -p database. select database(); affords viewing which database is in use.


Milter Main Page Apache