Archive for the ‘linux’ Category

« Older Entries

FreeNAS iSCSI and VMWare ESXi

Friday, September 3rd, 2010

For a long time I’ve been using NFS to serve the diskspace my FreeNas server has, to my ESXi machine, and ‘ve been wondering if iSCSI would be a better solution. NFS has more overhead than iSCSI, so iSCSI should outperform NFS on higher workloads.
I’ve never really taken the time to get to know iSCSI and it alwasy looked very messy and difficult to set up and use. But today, that idea changed!

We’ve been running ESXi at the office for a couple of weeks too in a test environment, and are investigating the necessities for a production installation at one of our clients. They’ll have 2 servers (1 master, 1 ‘spare’) and a shared NAS/SAN solution to store the virtual disks. As performance is necessary there (I didn’t really care for my own setup at home, as long as it is reliable), I was looking for a simple howto description to use iSCSI on the ESXi side, and tried that with my own FreeNAS storage machine.

Following this guide I found a detailed description of the configuration needed in FreeNAS to get it up and running in 10 minutes. It really did it!
An iSCSI datastore is connected now onto my ESXi machine, for testing. I only took 50GB to test with, as I’m thinking of redesigning my FreeNAS box with another motherboard, raid controller and disk setup, as the current Dual AMD Opteron with 12GB RAM is a bit overkill, and too power hungry… As I don’t have a decent motherboard that supports the Raid controllers I currently use (PCI-X based), I need to rethink the setup and get me a decent PCI-e x4/x8 controller capable of holding 8 to 12 disks (2 controller setup is just fine, even 3 controllers might do if it costs less that way.) When I finish that migration and get all the data copied over, I’ll setup the FreeNAS-ESXi connection on a purely iSCSI based setup…

We’ll see…

Posted in Server Management, System Management, Virtualisation, hardware, linux | No Comments »

mc2xml stopped working

Saturday, August 14th, 2010

Apparently, mc2xml, the software package that downloads and xml-formats the electronic program guide information to my MythTV installation, stopped working a couple of days ago… EPG data is empty after tonight :(

The original site is down, and I checked a couple of forums. They all point to a new site where you can simply download the new package, and start downloading again… luckily…

Posted in AV/TV, MythTV, linux | 1 Comment »

Linux Virtualization – OpenVZ/Proxmox

Wednesday, August 11th, 2010

In times of economical troubles, a small company needs to look for money-saving ideas and use them to take their infrastructure to a less costly setup with the same (or at least nearly) power and possibilities.
For that reason, over the last 2 years we were looking into both energy efficient hardware and virtualization solutions to get the most out of our machines in the datacenter. Every single Ampère costs us a lot of money every month. To save energy, we fitted our backupservers with Western Digital Green Power drives instead of the standard Raid Editions we had before, as we did have to expand the Raid volume anyway.

Another way of reducing the power needs is virtualization some of the separate machines, so the total of our servers could be scaled down a little.
We had a couple of servers used for web- and ftp traffic, which weren’t utilized 100%, not even 25%…
After an inside testing setup with OpenVZ, a container based Linux virtualization solution, we looked into expanding it onto our datacenter installations. The big problem here was the VLAN setup we use in the datacenter, to separate all client networks. Apparently OpenVZ doesn’t really cope well with VLAN’s, or at least there is not that much info about the usage and configuration of setting up containers in separate VLAN’s.
Looking into GUI-possibilities, web based management options for OpenVZ, we stumbled upon Proxmox VE which is an all-in-one distribution, supporting OpenVZ, KVM etc as virtualization options, and which has a nice and full-featured web based management console.

Setting up machines in Proxmox is really easy. You can choose which virtualization environment you want to use (KVM or OpenVZ containers). KVM allows you to run different OS’es next to each other, like Windows servers, BSD etc, while OpenVZ uses the same basic kernel, but allows the containers to be run independently, giving a very low overhead, and maximum performance.
OpenVZ containers are ‘template based’, and Proxmox has a list of already defined and downloadable templates like Debian 4, 5 and 6, Ubuntu 8.04, their own Proxmox email security suite (I think you need a license for this), but also predefined WordPress, Drupal, Zenoss monitoring, Joomla, Mediawiki, …
You can download the ones you need or want to deploy, and a couple of minutes later, the template is ready to be used (depending on your internet connection speed, off course)
Once you have one or more templates, defining a container’ed machine is easy as hell: Create a new ‘machine’, choose OpenVZ as the type, select the template, define a root password for accessing the machine, and select network, disk space etc… When clicking the create button, Proxmox will set up your environment, which will be ready for use in minutes.

VLAN integration with the Proxmox web console is really easy: all you need to do is define extra interfaces as ethx.yyyy where x is the ethernet interface number, and yyyy is your VLAN ID.
Once those were all set, we could define a container, and assign a vmbr bridged network connection to it, which holds the VLAN tag info.

We migrated 4 ftp and webservers in a matter of hours instead of days. Most of the time went into syncing the data between the old and the new machines.

An extra option in Proxmox is defining separate storage possibilities, for example local storage, iSCSI, NFS, … for (automated) backup and snapshot storage…

Proxmox hosts can be ‘joined’ into a cluster, so you can migrate containers between hosts, without downtime, interruption, big troubles or manual intervention. That way you can move machines needing more memory, or migrate to heavier hardware, perform hardware maintenance and have the most important machine run on a backup host server, without the long-time interruptions, and stress accompanied with such a migration or maintenance.

Proxmox is, for Linux virtualization, my favorite now, far above any other solution we have been using in the past.

Posted in IT-news, Server Management, System Management, Virtualisation, linux, ubuntu | No Comments »

Open source push email with z-push

Monday, May 3rd, 2010

After having my email being served by Zimbra Open Source Edition for over a couple of months now, the time was there to implement my email/contacts/tasks sync to my Android cell phone again (after years of true ActiveSync with Exchange).
I started looking at possibilities, being z-push and funambol (probably others too, but not by my knowledge or what I have found) and investigating how to implement (one of) them.

A couple of years ago, I already had some trial and error experience with Funambol, back in the days it was still called Sync4j, but I have never gotten it to work stable enough, so I first went with z-push.

Z-push relies on a couple of packages, being php, apache and php-pear, and a couple of others.

The base of my configuration comes from a basic-detailed description I found on a slashdevslashnull page, describing the steps and info needed for an install.

I didn’t have any success, failing on several parts…
First one: running an Apache on the same host as the Zimbra isn’t all that easy, so I installed a separate Ubuntu server on my virtual server, just for the z-push frontend.
Second: the configuration of the z-push config.php isn’t very well documented. Using the IMAP backend, I got to logging in, but I didn’t have any success in real data. The problem lies in the security settings conflicting between the IMAP z-push backend config and the Zimbra IMAP service.

I did a little more research, and stumbled upon Zimbra-backend for z-push, a backend communications module to interface both products.
The config needed in config.php is even less documented, and it took me a while to figure it out.

To enable debugging, you need to create a file called debug.txt in the z-push root dir, and give it the proper file rights for it to be writable.

This block seems to do it fine, that is: it is giving me a good readout in de created debug.txt file:
// ***********************
// Zimbra Backend Settings
// ***********************
define('ZIMBRA_URL', 'https://172.16.248.21');
define('ZIMBRA_SYNC_CONTACT_PICTURES', true);
define('ZIMBRA_VIRTUAL_CONTACTS',true);
define('ZIMBRA_VIRTUAL_APPOINTMENTS',true);
define('ZIMBRA_VIRTUAL_TASKS',true);
define('ZIMBRA_IGNORE_EMAILED_CONTACTS',true);
define('ZIMBRA_HTML',true);

With this config I can enable my T-Mobile G1 Android 1.6 powered smartphone to pick up email, contacts and such through ActiveSync with the Touchdown application.

Posted in gsm-pda, linux, ubuntu | No Comments »

Nagios3 on Ubuntu

Tuesday, March 30th, 2010

I reinstalled a low power Via C3 machine to monitor my network, now almost 90% of my machines run on ESXi… Sure I didn’t want to monitor a virtual host from one of the virtual guest it runs… when the host goes down, what good will do monitoring from within?

The machine was installed using Ubuntu Server x86 9.10, the most recent stable one, and after basic installation, fixing the IP, setting a hostname and apt-get update, apt-get upgrade and apt-get dist-upgrade, I installed nagios3 and all its extras from aptitude.

Configuration was quite easy, I just had to take over the config I already had running on an older box which will be shut down later this week to save energy (it’s an old power-hungry Pentium 4 machine with too many single-disk storage blocks). After logging in, I noticed a lot of the services were ‘muted’… Notification was disabled on all HTTP checks etc…

When trying to enable the notification for a service check on one of the machines, I got the error Error: Could not stat() command file '/var/lib/nagios3/rw/nagios.cmd'!
I checked the rights on the file, owner and group ownership, but nothing wrong was to be seen there.
Googling around, I stumbled upon a short post on Yes! That Works Fine! and tried the 4 command lines.
I had to change it a bit so the original code:


stop nagios2
/etc/nagios2# dpkg-statoverride --update --add nagios www-data 2710 /var/lib/nagios2/rw
/etc/nagios2# dpkg-statoverride --update --add nagios nagios 751 /var/lib/nagios2
start nagios2

became:

/etc/init.d/nagios3 stop
/etc/nagios3# dpkg-statoverride --update --add nagios www-data 2710 /var/lib/nagios3/rw
/etc/nagios3# dpkg-statoverride --update --add nagios nagios 751 /var/lib/nagios3
/etc/init.d/nagios3 start

After retrying to enable the notification for the HTTP check that caused the message above, it worked without any error, and the notification is working again.

Posted in Server Management, System Management, linux, ubuntu | 2 Comments »

Rescanning sata/scsi devices on non-hotswap capable controllers

Tuesday, January 12th, 2010

I wanted to use the hotswap tray I recently installed in my computer case, and easily plug in the 1.5TB disk to make a backup of the backup server, before reinstalling it in a couple of days.
When plugging in the disk, the Ubuntu system didn’t see it appearing, or at least, I didn’t notice or know where to look for it.

Looking around (googling) for a ‘sata rescan’ I found a thread describing how to rescan the scsi/sata bus (as sata is seen by the kernel as scsi).
It quite simple, once you know which bus holds which disk…

It works as follows:

Become root:
sudo su
go to the directory for the host/controller in /sys/class/scsi_host/hostN where N is the number of the host

and issue this command:
root@ubuntu:/sys/class/scsi_host/host0# echo - - - > scan

in dmesg you will see the new devices found on this controller/host appearing, and the appropriate /dev/sdX device directives are created automatically.

Easy, no?

Posted in System Management, hardware, linux, ubuntu | No Comments »

TechnoTrend DVB-S2 S2-3200 on MythTV

Saturday, January 9th, 2010

Well,

after some testing and trying, it looks like the TechnoTrend S2-3200 is working quite well with MythTV 0.22, though I don’t have or use the DVB-S2 stuff yet… no dual LNB, no HD equipment, just testing for the future.

My Backend was running Ubuntu 9.04 with Mythtv 0.22 packages, which did find the card, but couldn’t use it.
I installed a test machine on an similar motherboard/config and used Ubuntu 9.10, and the card was detected. Switching the TVV smartcard over to the S2-3200 CAM/CI, I was able to scan and later tune in to the TVV encrypted channels!
Bottom line: Ubuntu 9.10, MythTV 0.22 and TT S2-3200 with TVV smartcard: working out of the box!

Update: Sander Van den Bosch has the same setup (kind of, also using the S2-3200 with a TVV subscription). He had an issue receiving the encrypted channels, and he asked for a little information (the reason I installed this testsetup and eventually did the upgrade on my ‘production’ backend). I tried, and he ‘copied’ what was successful for me, but he didn’t succeed in getting the same results.
In the end, he found out that his CAM (original TVV CAM) had an ‘old’ firmware, which wasn’t compatible… It seems my 2 CAM’s (also original TVV CAM modules) are ‘newer’… He used a Windows XP laptop to update the firmware, and succeeded in receiving the TVV encrypted channels! He was able to tune into Eén HD successfully!
Another happy MythTV – TV Vlaanderen user :-D

Posted in MythTV, hardware, linux, ubuntu | 24 Comments »

And then there was Zimbra!

Wednesday, December 9th, 2009

Zimbra, the collaboration suite, gives you a complete MS Exchange server replacement, as far as I can see.
Email, Contacts, Calendar, Documents, Tasks, … everything is in there!

There is an Open Source edition, where you can get the basics for free, without support that is, and can install it on a number of systems. According to Wikipedia and the FAQ it consists of a number of Open Source packages such as postfix, clamav, and such (full list is on the wikipedia article), combined to make a solid Groupware suite.

I recently re-started looking to replace the old Exchange 2003 system I’m still running in a VM, but was always confrontend with either a do-it-yourself approach (combining an mta, antivirus and antispam solution, some ical stuff,… myself), or a you’ll-have-to-pay-a-fee for a complete basic solution.

I knew about Zimbra, and had played a bit with it a couple of years ago. In the beginning of this year, I started looking at the Zimbra Desktop as an email client with integrated calendar and task management modules, because of the platform support. It has an installer routine for Windows, Mac OS X and Linux, and could easily replace Outlook and other proprietary email/groupware clients at some of the customers of the firm I work for.

The server side installation was done in a VM’ed Ubuntu Server 8.04 LTS 32 bit running under VMWare Server 2.0. Installation is quite easy for the Open Source edition (and I presume that the installation routine is the same for all editions…?). You run the big installer package you downloaded, and answer a couple of questions regarding desired modules. It came to tell me a couple of dependencies weren’t met, which were easily installed through apt-get.
After the installer finished, it proposed me a list of settings. In this list, the installer required me to set an admin password, which wasn’t all that clear at first. I asked to quit, because I didn’t see what the installer needed. Seeing, after quitting, that it didn’t start the server daemons, I knew I missed something. I quickly did a reinstall of the package, and when the list was proposed again, I noticed the *** before the password line.
After setting the password, I could Activate the settings, which wasn’t possible before.
Once activated, the install routine ended with a start of all the services.

I redirected a browser to http://serverIP:7107 and was prompted for a username and password. I logged in with the details I entered in the installer, and got a nice and complete configuration module.
Setting up a user for myself, and fiddling around with the settings, the first installation was completed.

I changed my MX records to the newly installed Zimbra server, and noticed mail coming in… It was working.

Next up:
-setup another user to check calender sharing and such
-zpush configuration for Activesync based push email and synchronization
-export-import Exchange mailbox into the Zimbra server mailbox
-shutdown Exchange server VM at last ;)

I’ll keep you posted.

Posted in Server Management, System Management, linux, ubuntu | 2 Comments »

MythTV 0.22 on Ubuntu 9.04

Thursday, December 3rd, 2009

I had a major issue at home with my MythTV backend server… and off course, right before I was implementing a decent backup strategy.

I wanted to move to a Raid5 solution to increase speed and disk capacity, but wanted to do an Ubuntu 9.04 reinstall, move all data and config over from the old single-disk install, and keep running on settings and recordings already in the database. At the same time I wanted to upgrade the backend to Ubuntu 9.10, and use the newer MythTV 0.22 packages.

My install went pretty fine, using software Raid 5 on 3 disks I still had lying around (WD Raid edition SATA2), formatting the 500GB usable space as an ext4 volume, and installation of MythTV and other related packages was done without any issues. I started copying from the old single-disk setup to the new soft-Raid volume, and was almost done migrating. The configuration of MythTV through mythtv-setup couldn’t find my Technotrend S-1500 DVB-S card I used for months now, without any issues on Ubuntu 9.04… so I needed to have a look at that… but I even didn’t get to that point!
All of a sudden, the session over ssh was having some big issues, not finding commands, giving errors etc… Returning to the console, a lot of raid error messages were displayed, so a reboot and recover/rebuild was needed… the machine didn’t take any commands anymore… even a ‘halt’ or ‘reboot’ didn’t work…
When powering down and back up, the machine hung at the kernel loading, giving me a serious error not retrieving md0 anymore… Panic!!!

I disconnected my original disk, booted a Live session of the Ubuntu Desktop 9.10 cd (ext4 is better supported in 9.10 than it is in 9.04), and tried almost everything I know and could find about retrieving data from a faulty software raid volume… to no result…
I decided to do a fresh install on ext3 instead of ext4 with the Ubuntu 9.04 distro, to resolve the card-support issues I was having with Ubuntu 9.10, which worked out pretty fine again.
Card was detected, and scanning gave the results I expected.

I hooked up the original single-disk install SATA drive, and wanted to start copying the database, recording and settings files, but I wasn’t able to mount any of the original partitions on the disk. After some looking around on the disk, I found all partitions vanished! The disk was empty! Even more panic!!!!

Testdisk, an open source partition and data recovering program, had me regain access and information of the recordings partition, and I immediately made a copy of the data to my backup-server, but still no /etc or /var/lib/mysql to be retrieved to get access to the original database and settings… So I need to start from scratch, and convert the old recordings to standard mpg or avi files, so I can at least play them without info etc… That’s not the end of the world!

Now, the MythTV 0.22 upgrade part was going to be the hardest…, or so I thought.
In Ubuntu 9.04, the 0.22 MythTV packages aren’t available through the normal repositories…
I searched in google, and eventually found a page on the MythBuntu site that describes the necessary steps and packages needed to upgrade a 9.04 (or even a 8.10) Ubuntu distro with the MythTV 0.22 release…
The MythTV backend server was easily upgraded through the use of the MythBuntu Repositories package
After installing this deb file through dpkg -i, going into aptitude and doing an update/upgrade, the MythTV packages were upgraded to 0.22 automatically.

I tried connecting with my 0.21 based frontend machines (which run Ubuntu 8.04 LTS), but found that the database structure was changed in such a way the 0.21 cannot contact and use the 0.22 database structure… An upgrade to 0.22 on these machines is needed as well.

I installed the same MythBuntu Repositories package package on these 2 machines, and found it could only install the already installed 0.21 version. I needed to upgrade the distro to 8.10 or higher… I decided to go all the way up to 9.10…

The standard AMD based pc which resides in my living room and is hooked onto the beamer, is the easiest… if anything goes wrong, I can just reinstall it with 9.10 from scratch…, so I started with this one before messing up the MythBuntu 8.04 installation on the AppleTV in the bedroom (which is a lot more work to do a complete (re)installation).
Though not supported, I changed over the /etc/apt/sources.list hardy listings to karmic, and did an apt-get update followed by an apt-get upgrade and an apt-get dist-upgrade… I met a lot of errors installing or configuring packages, and needed to uninstall a lot of packages through aptitude, but eventually using apt-get dselect-upgrade and another apt-get upgrade and dist-upgrade, the machine eventually ended up booting fine with Ubuntu 9.10.
I installed the MythTV 0.22 packages through aptitude and ended up configuring the connection to the newly installed database on the backend. Two minutes , and some screen adjustments later, I was looking at LiveTV through the new 0.22 MythTV release!

For the AppleTV, I will follow the same things, but I’m making a full rsync of the complete filesystem first. If anything goes wrong, I can just rsync it back over it, and have the machine working again…

Now I just need to convert all recordings to a viewable format (mpg or avi) and change all the filenames so I know what recording is in what file… that will take another weekend to do so, I think.

First impressions of 0.22: faster, slicker, … but I didn’t see that many adjustments or changes… (except from the new theme stuff, which was the biggest change announced off course)

Next on the list: BACKUPS!!!! I have a backupserver giving me 2,3TB through NFS of which still 1,2TB is free, why wouldn’t I use it??? :p

More to follow later…
At least now, I’m up to date again…

Posted in AV/TV, MythTV, Server Management, System Management, hardware, linux, ubuntu | 6 Comments »

Arkeia free Enterprise Backup solution for Ubuntu

Monday, November 16th, 2009

According to this page found through this article Arkeia is providing a free license for their Enterprise grade Backup solution Arkeia Network Backup version 8.

The license includes 2 agent licenses for Windows, Mac OSX/BSD and Linux machines and the server software is included in the Ubuntu 8.04 repositories.

I’ll install it on my server for sure!!

Posted in IT-news, Server Management, System Management, linux, ubuntu | No Comments »

« Older Entries