Difference between pages "Sucknews" and "Compiling Xen"

From Wikislax
(Difference between pages)
Jump to: navigation, search
(Installing Sucknews)
 
 
Line 1: Line 1:
 
{{RightTOC}}
 
{{RightTOC}}
  
== What is Sucknews ? ==
+
== What is Xen ? ==
  
Sucknews affords getting the newsfeeds over a regular NNTP connection with your Internet Service Provider. This comes in handy when you are not a big company and have no agreements with peer Newsgroup servers.
+
[http://wiki.xen.org/wiki/Xen_Overview Xen] is an hypervisor, a thin software layer executing multiple virtual machines, each running its own operating system. Xen is normally used as a server virtualization platform, running on headless servers without graphical console and controlled through the network. However it is also possible to run Xen on graphical desktops, and with proper hardware virtualization, to dedicate the primary graphics card (and keyboard / mouse) to a virtual machine, making it possible to have high performance full 3D and video acceleration in a virtual machine (see [http://wiki.xen.org/wiki/Xen_VGA_Passthrough Xen VGA Passthru]). Xen is otherwise free and open source.
  
== Installing Sucknews ==
+
== A few quirks ==
  
There was a time when sucknews was available from [http://www.sucknews.org sucknews.org] but this site has disappeared and there seems to be no obvious source of sucknews update software so we'll stick with an old [{{SERVER}}/wikislax/download/suck-4.3.2.tar.gz version] (perfectly satisfactory anyway). Untar and install as below :
+
The '''dev86-0.16.21-x86_64-1.txz''' package included with Slackware 14.2 does not afford compiling '''xen-4.9.0''' properly, and must be replaced by '''[{{SERVER}}/wikislax/download/dev86-0.16.17-x86_64-2.txz dev86-0.16.17-x86_64-2.txz]''', that can be found on the Slackware 13.37, 14.0, or 14.1 distribution disks, in directory '''slackware64/d'''. Trying to compile dev86 [http://v3.sk/~lkundrak/dev86/ from source] is not an alternative : dev86 source versions 0.16.18 to 0.16.21 have the same issue compiling xen-4.9.0, and older 0.16.17 does not compile on Slackware 14.2.
  
  # tar -C /usr/local -xvf suck-x.y.z.tar.gz
+
# removepkg /var/log/packages/dev86-0.16.21-x86_64-1.txz
 +
# installpkg dev86-0.16.17-x86_64-2.txz
 +
 
 +
Also, the '''grub-2.00''' included with Slackware 14.2 does not afford booting xen. However, getting the latest version '''grub-2.02''' from source does work.
 +
 
 +
== Hardware requirements ==
 +
 
 +
Xen runs on Intel X86 hardware and requires a processor and motherboard supporting VT-x and optionally VT-d for hardware virtualization. See this [http://www.intel.com/support/motherboards/desktop/sb/cs-030922.htm page] for a list of Intel compatible motherboards and chipsets and this [http://ark.intel.com/ page] for a list of compatible processors. Our system running Xen successfully at the time of this writing (and since June 2012) is based on a DZ77GA70K Intel motherboard, an Intel® Core™ i7-3770 Processor (the overclockable i7-3770 "K" model does not afford virtualization), 32 Gb of PC12800 memory and an MSI GeForce G210 graphics board.
 +
 
 +
== Documentation difficulties ==
 +
 
 +
Although the software itself works well and is pretty straighforward, good quality Internet information is missing. The volume of information on the Xen wiki is plethoric, but mostly irrelevant as pertaining to old versions of everything. Building the big picture requires interpretation of tiny bits in forum messages, a pretty painful process, although I have to recognize that it worked for me in the end. An alternative is to use one of these old-style information repositories named "books". Yes it is pretty old-fashionned ;) but actually there are good ones on the topic. Here is [http://www.amazon.co.uk/The-Book-Xen-Practical-Administrator/dp/1593271867/ref=sr_1_1?s=books&ie=UTF8&qid=1341037874&sr=1-1 the most recent I found], it is a good value but of course you can find more on [http://www.amazon.co.uk/s/ref=nb_sb_noss_2?url=search-alias%3Dstripbooks&field-keywords=xen&x=0&y=0 amazon(.co.uk)].
 +
 
 +
== Software constraints ==
 +
 
 +
To make a long story short, at the time of this writing (and since June 2012) working with nVidia graphic boards on Xen and X11 requires the "nouveau" driver. Other drivers like nv or the nVidia proprietary driver do not support Xen and switch off the screen when launched or do not display properly. "Nouveau" requires a fairly recent version of X11. Slackware 13.37 or newer is required. "Nouveau" is available in kernel 3.4.2 upstream and was previously included as a staging driver. Xen dom0 support was included in kernel 3.0. To benefit from both Xen and "Nouveau", the best is to use kernel 3.4.2 upstream.
 +
 
 +
== Installing acpica ==
 +
 
 +
Xen requires acpica. [https://www.acpica.org/downloads Download] then install as below :
 +
 
 +
  # tar -C /usr/local -xvf acpica-unix-yyyymmdd.tar.gz
 +
# cd /usr/local
 +
# chown -R root:root acpica-unix-yyyymmdd
 +
# cd acpica-unix-yyyymmdd
 +
# make
 +
# make install
 +
 
 +
== Installing yajl ==
 +
 
 +
Xen 4.19.1 requires yajl. [http://lloyd.github.io/yajl/ Download] then install as below. <u>Note</u> : there is no option to specify the target library directory so the files need to be moved manually.
 +
 
 +
# tar -C /usr/local -xvf lloyd-yajl-17b1790.tar.gz
 
  # cd /usr/local
 
  # cd /usr/local
  # chown -R root:root suck-x.y.z.tar.gz
+
  # chown -R root:root lloyd-yajl-17b1790
  # cd suck-x.y.z.tar.gz
+
  # cd lloyd-yajl-17b1790
# ./configure --help | less
+
  # ./configure
  # ./configure --prefix=/usr/local/news --libdir=/usr/local/news/lib64 --mandir=/usr/local/man \
 
--with-inn-lib=/usr/local/news/lib --with-inn-include=/usr/local/news/include --with-perl-exe=/usr/bin
 
 
  # make
 
  # make
 +
# make install
 +
# cd ../lib
 +
# mv libyajl* ../lib64
 +
# ldconfig
 +
 +
== Compiling Xen ==
 +
 +
[http://xen.org/products/xen_source.html Download Xen] from the official [http://www.xen.org xen.org] site. <u>Note</u> : File stubs-32.h is missing in the compiler includes so we add a link to the existing stubs-64.h. Also, some Xen Python scripts are installed in /usr/local/lib64/python-2.7/site-packages which python cannot find so we add links from the standard library as well.
 +
 +
# cd /usr/include/gnu
 +
# ln -s stubs-64.h stubs-32.h
 +
# cd
 +
# tar -C /usr/local -xvf xen-x.y.z.tar.gz
 +
# cd /usr/local
 +
# chown -R root:root xen-x.y.z
 +
# cd xen-x.y.z
 +
# ./configure --libdir=/usr/local/lib64 --with-initddir=/etc/rc.d
 +
# make world
 
  # make install
 
  # make install
 
  # make clean
 
  # make clean
  # cd /usr/local/news/bin
+
  # cd ../lib64/python2.7/site-packages
  # chown <nowiki>news:news</nowiki> lmove rpost suck testhost
+
# ln -s xen /usr/lib64/python2.7/site-packages
 +
  # ln -s xen-3.0-py2.7.egg-info /usr/lib64/python2.7/site-packages
  
== Running Sucknews ==
+
== Adjusting rc.local* ==
  
The below script affords posting the local messages and get entering messages on your Internet Service Provider site :
+
Xen needs a couple of daemons to run to ensure VM management. Add these lines to rc.local and rc.local_shutdown :
  
  # su news
+
  PATH=/usr/local/sbin:/usr/local/bin:$PATH
$ cd /usr/local/news/bin
+
  export PATH
$ '''vi suck.sh'''
 
'''i'''
 
#!/bin/sh
 
 
NNTP_SERVER=news.free.fr
 
NEWS_PATH=/usr/local/news
 
BIN_PATH=/usr/local/news/bin
 
SUCK_PATH=/usr/local/news/bin
 
  BATCH_PATH=/usr/local/news/spool/outgoing/free
 
FILTER_PATH=$SUCK_PATH/filter.sh
 
 
   
 
   
  cd $SUCK_PATH
+
  # start xencommons
+
  if [ -x /etc/rc.d/xencommons ]; then
######################################################
+
     /etc/rc.d/xencommons start
  # posting outgoing articles (localhost->NNTP_SERVER) #
 
######################################################
 
 
echo "Sending articles..."
 
if test -s $BATCH_PATH
 
then
 
$BIN_PATH/rpost $NNTP_SERVER -b $BATCH_PATH \
 
     -f $FILTER_PATH \$\$o=/tmp/filtered \$\$i /tmp/filtered
 
else
 
    echo "No articles to post..."
 
 
  fi
 
  fi
 
   
 
   
  ######################################################
+
  # stop xencommons
# getting incoming articles (NNTP_SERVER->localhost) #
+
  if [ -x /etc/rc.d/xencommons ]; then
######################################################
+
     /etc/rc.d/xencommons stop
 
echo Getting articles...
 
  if [ -e /tmp/newposts ]; then
 
    rm /tmp/newposts
 
fi
 
$BIN_PATH/suck $NNTP_SERVER -AL $NEWS_PATH/db/active -i 0 -n -H -K -br /tmp/newposts -c
 
if [ -e /tmp/newposts ]; then
 
     $BIN_PATH/rnews /tmp/newposts
 
 
  fi
 
  fi
cat /dev/null > $BATCH_PATH
 
'''<esc>'''
 
''':x'''
 
$ chmod u+x suck.sh
 
$ ./suck.sh
 
Sending articles...
 
No articles to post...
 
Getting articles...
 
Attempting to connect to news.free.fr
 
Using Port 119
 
Official host name: news.free.fr
 
Address: 212.27.60.38
 
Address: 212.27.60.39
 
Address: 212.27.60.37
 
Address: 212.27.60.40
 
Connected to news.free.fr
 
200 news-4.free.fr (4-2) NNRP Service Ready - newsmaster@proxad.net (posting ok)
 
No sucknewsrc to read, creating
 
Adding new groups from local active file to sucknewsrc
 
New Group - adding to sucknewsrc: control
 
control - 1 articles 3983-3983
 
New Group - adding to sucknewsrc: control.cancel
 
control.cancel - 349 articles 94271852-94272200
 
New Group - adding to sucknewsrc: control.checkgroups
 
control.checkgroups - 4 articles 5228-5231
 
New Group - adding to sucknewsrc: control.newgroup
 
control.newgroup - 1 articles 73186-73186
 
New Group - adding to sucknewsrc: control.rmgroup
 
control.rmgroup - 1 articles 30996-30996
 
New Group - adding to sucknewsrc: junk
 
junk - 1 articles 38322-38322
 
New Group - adding to sucknewsrc: alt.os.linux.slackware
 
alt.os.linux.slackware - 1907 articles 231211-233117
 
Elapsed Time = 0 mins 0.72 seconds
 
2227 Articles to download
 
Deduping Elapsed Time = 0 mins 0.00 seconds
 
Deduped, 2227 items remaining, 0 dupes removed.
 
Total articles to download: 2227
 
5290946 Bytes received in 1 mins 20.79 secs, BPS = 65489.2
 
Closed connection to news.free.fr
 
Building RNews Batch File(s)
 
Cleaning up after myself
 
news@inner:/usr/local/news/bin$
 
<ctrl>d
 
#
 
  
In the first part, messages are posted to the provider using '''rpost'''. The list of articles is obtained from the information contained in newsfeed file '''/usr/local/news/spool/outgoing/free'''. The '''-f''' option affords applying a filter to the messages so as to expurge certain headers :
+
== Compiling a dom0 Kernel ==
  
#!/bin/sh
+
Domain-0 (dom0 for short) is a special guest (virtual machine) that the Xen hypervisor always loads on host startup. Dom0 is used to control and manage the Xen hypervisor, and provides virtual disks and networks for other unprivileged guests (=domUs). Dom0 support was introduced in Linux kernel 3.0. The kernel generated must include the .config file domU and [http://wiki.xen.org/wiki/Mainline_Linux_Kernel_Configs#Configuring_the_kernel dom0 options]. Here is a minimal example of such a [{{SERVER}}/wikislax/download/config-dom0 .config dom0] file. Feel free to use it as a base, replacing device drivers as required. The rest of the kernel compilation is nominal :
/usr/local/news/bin/sm -R $1 | sed -e "/^X-Trace/d" -e "/^NNTP-Posting-Host/d" \
 
-e "/^Xref/d" -e "/^X-Complaints-To/d" -e "/^NNTP-Posting-Date/d" > $2
 
  
In the second part, suck gets the messages from your Internet Service Provider. The '''-AL''' option affords using the list of groups defined. '''-i 0''' indicates that there is no limit on the number of messages to get, '''-n''' is for the mode « messages identified by their numbers by the provider » '''-c''' is to update the numbers after the end of the operation, '''-br''' defines the output file, '''-H''' and '''-K''' afford skipping the historic and killfile options. The file written is afterwards used by '''rnews'''. The articles are then available.
+
# tar -C /usr/src -xvf linux-4.4.88.tar.bz2
 
+
# cd /usr/local
== Automating Sucknews ==
+
# rm linux
 
+
# ln -s linux-4.4.88 linux
Automate Sucknews execution using '''crontab'''. In the example below, suck.sh is executed at 13:00 daily :
+
# cd linux
 +
# make menuconfig
 +
# make
 +
# make modules_install
 +
# cp arch/x86_64/boot/bzImage /boot/vmlinuz-4.4.88-dom0
 +
# cp System.map /boot/System.map-4.4.88-dom0
 +
# cp .config /boot/config-4.4.88-dom0
  
# su news
+
We're now all set up, Xen is ready to be booted by grub2 !
$ crontab -e
 
# MIN HOUR DAY MONTH DAYOFWEEK COMMAND
 
00 13 * * * /usr/local/news/bin/suck.sh
 
30 13 * * * /usr/local/news/bin/news.daily expireover lowmark
 
  
 
<br/>
 
<br/>
  
{{pFoot|[[INN]]|[[Main Page]]|[[Asterisk]]}}
+
{{ pFoot |[[Compiling from Source]]|[[Main Page]]|[[Using Grub2]]}}

Revision as of 22:45, 8 February 2025

What is Xen ?

Xen is an hypervisor, a thin software layer executing multiple virtual machines, each running its own operating system. Xen is normally used as a server virtualization platform, running on headless servers without graphical console and controlled through the network. However it is also possible to run Xen on graphical desktops, and with proper hardware virtualization, to dedicate the primary graphics card (and keyboard / mouse) to a virtual machine, making it possible to have high performance full 3D and video acceleration in a virtual machine (see Xen VGA Passthru). Xen is otherwise free and open source.

A few quirks

The dev86-0.16.21-x86_64-1.txz package included with Slackware 14.2 does not afford compiling xen-4.9.0 properly, and must be replaced by dev86-0.16.17-x86_64-2.txz, that can be found on the Slackware 13.37, 14.0, or 14.1 distribution disks, in directory slackware64/d. Trying to compile dev86 from source is not an alternative : dev86 source versions 0.16.18 to 0.16.21 have the same issue compiling xen-4.9.0, and older 0.16.17 does not compile on Slackware 14.2.

# removepkg /var/log/packages/dev86-0.16.21-x86_64-1.txz
# installpkg dev86-0.16.17-x86_64-2.txz

Also, the grub-2.00 included with Slackware 14.2 does not afford booting xen. However, getting the latest version grub-2.02 from source does work.

Hardware requirements

Xen runs on Intel X86 hardware and requires a processor and motherboard supporting VT-x and optionally VT-d for hardware virtualization. See this page for a list of Intel compatible motherboards and chipsets and this page for a list of compatible processors. Our system running Xen successfully at the time of this writing (and since June 2012) is based on a DZ77GA70K Intel motherboard, an Intel® Core™ i7-3770 Processor (the overclockable i7-3770 "K" model does not afford virtualization), 32 Gb of PC12800 memory and an MSI GeForce G210 graphics board.

Documentation difficulties

Although the software itself works well and is pretty straighforward, good quality Internet information is missing. The volume of information on the Xen wiki is plethoric, but mostly irrelevant as pertaining to old versions of everything. Building the big picture requires interpretation of tiny bits in forum messages, a pretty painful process, although I have to recognize that it worked for me in the end. An alternative is to use one of these old-style information repositories named "books". Yes it is pretty old-fashionned ;) but actually there are good ones on the topic. Here is the most recent I found, it is a good value but of course you can find more on amazon(.co.uk).

Software constraints

To make a long story short, at the time of this writing (and since June 2012) working with nVidia graphic boards on Xen and X11 requires the "nouveau" driver. Other drivers like nv or the nVidia proprietary driver do not support Xen and switch off the screen when launched or do not display properly. "Nouveau" requires a fairly recent version of X11. Slackware 13.37 or newer is required. "Nouveau" is available in kernel 3.4.2 upstream and was previously included as a staging driver. Xen dom0 support was included in kernel 3.0. To benefit from both Xen and "Nouveau", the best is to use kernel 3.4.2 upstream.

Installing acpica

Xen requires acpica. Download then install as below :

# tar -C /usr/local -xvf acpica-unix-yyyymmdd.tar.gz
# cd /usr/local
# chown -R root:root acpica-unix-yyyymmdd
# cd acpica-unix-yyyymmdd
# make
# make install

Installing yajl

Xen 4.19.1 requires yajl. Download then install as below. Note : there is no option to specify the target library directory so the files need to be moved manually.

# tar -C /usr/local -xvf lloyd-yajl-17b1790.tar.gz
# cd /usr/local
# chown -R root:root lloyd-yajl-17b1790
# cd lloyd-yajl-17b1790
# ./configure
# make
# make install
# cd ../lib
# mv libyajl* ../lib64
# ldconfig

Compiling Xen

Download Xen from the official xen.org site. Note : File stubs-32.h is missing in the compiler includes so we add a link to the existing stubs-64.h. Also, some Xen Python scripts are installed in /usr/local/lib64/python-2.7/site-packages which python cannot find so we add links from the standard library as well.

# cd /usr/include/gnu
# ln -s stubs-64.h stubs-32.h
# cd
# tar -C /usr/local -xvf xen-x.y.z.tar.gz
# cd /usr/local
# chown -R root:root xen-x.y.z
# cd xen-x.y.z
# ./configure --libdir=/usr/local/lib64 --with-initddir=/etc/rc.d
# make world
# make install
# make clean
# cd ../lib64/python2.7/site-packages
# ln -s xen /usr/lib64/python2.7/site-packages
# ln -s xen-3.0-py2.7.egg-info /usr/lib64/python2.7/site-packages

Adjusting rc.local*

Xen needs a couple of daemons to run to ensure VM management. Add these lines to rc.local and rc.local_shutdown :

PATH=/usr/local/sbin:/usr/local/bin:$PATH
export PATH

# start xencommons
if [ -x /etc/rc.d/xencommons ]; then
    /etc/rc.d/xencommons start
fi

# stop xencommons
if [ -x /etc/rc.d/xencommons ]; then
    /etc/rc.d/xencommons stop
fi

Compiling a dom0 Kernel

Domain-0 (dom0 for short) is a special guest (virtual machine) that the Xen hypervisor always loads on host startup. Dom0 is used to control and manage the Xen hypervisor, and provides virtual disks and networks for other unprivileged guests (=domUs). Dom0 support was introduced in Linux kernel 3.0. The kernel generated must include the .config file domU and dom0 options. Here is a minimal example of such a .config dom0 file. Feel free to use it as a base, replacing device drivers as required. The rest of the kernel compilation is nominal :

# tar -C /usr/src -xvf linux-4.4.88.tar.bz2
# cd /usr/local
# rm linux
# ln -s linux-4.4.88 linux
# cd linux
# make menuconfig
# make
# make modules_install
# cp arch/x86_64/boot/bzImage /boot/vmlinuz-4.4.88-dom0
# cp System.map /boot/System.map-4.4.88-dom0
# cp .config /boot/config-4.4.88-dom0

We're now all set up, Xen is ready to be booted by grub2 !


Compiling from Source Main Page Using Grub2