I twitted about the sky high ready time on a VM. I've made a small Jing clip allowing you to see for yourself.

You can download dpclat.exe here.

So I'm quite curious what's happening here. Just one VM, affinity to CPU 0, run dpclat.exe: 15 seconds ready time. Put a load in the VM, ready time goes down... Strange!

Hi all,

Last weekend I upgraded my Joomla back-end to version 1.5.12. This was not an easy process. I had issues with removing the old files, template, modules, pictures. Right now I've resolved some of those issues, pictures should be back on every article, template is working again. This night I will be adding pictures to the other blog-posts and restoring the download section.

Using this new version gives me more options. The things I added is: a fresh comment system (you can now comment on every article and blog-post, give my readers more in-depth information, or better yet, correct my information - Gravatar enabled), tweet button (so you can tweet your followers), code highlighter (so you can easy read an copy the code) and most important, better security and stability (including SEF - Search Engine Friendly). I also should be able to add rich content like videos and other media. In the next folowing weeks I'll be updating the site template and layout.

For fellow bloggers and website holders: Some addresses have been changed!!! The blog is now: http://www.jume.nl/blog and the rss feed for the blog is: http://www.jume.nl/blog?format=feed&type=rss. Can you please, please, please update your links?

So have fun reading the blog and articles and leave a comment once in a time.

Cheers,
Bouke

Last week I posted my UDA experience. I was quite satisfied with the results, but for some reason I couldn't enable VMotion and iSCSI.

It took me some time so resolve the issue. Let me explain what I 'think' is happening:

S99

rc.local is NOT the last started script (while it should). This is very important since Pegasus is doing some stuff AFTER the rc.local. This is because VMware made multiple S99 scripts. I think this is a major issue, since all S99 scripts are started in alphabetical order. So in our case the boot is in this order:

S99local
S99pegasus
S99vmware-autostart
S99vmware-vpx (only there after adding ESX to vCenter)
S99wsman

So during %post I rename '/etc/rc.d/rc3.d/S99local' to '/etc/rc.d/rc3.d/S99zlocal' (notice the 'z' in the filename). Now it always starts last during boot. Huge improvement. I think VMware should not be using S99, there are plenty of other options.

Output

When you are able to run a command in rc.local started interactively, but it is not running during boot, the command might need some kind of output device. Now I've been trying to output the command to '/dev/tty1' but that doesn't work (since /dev/tty1 only exists after the last commands in rc3 are finished). But it does work if you output the command to a file, like /tmp/output. Example:

/etc/rc.d/init.d/sshd restart 2>&1 /tmp/output

Using the method I was able to enable VMotion during scripted install! It also works if you put the VMotion enabling stuff in a script and call the script with an output to file.

Environment variables

Some scripts are not using an absolute path to commands. This way you'll get an issue when the correct $PATH variables are not loaded yet. So it makes perfect sense the command runs during interactive start of rc.local, but not during boot. The way to resolve this is using 'su'. Example:

su - root -c "<command>"

Using this method I was able to get the iSCSI stuff going!

Wait a minute!

Right after boot and during rc.local, not all services are running. So I use sleep to wait some time to make sure those services are running (like the VMware management services). Use 'sleep <n>' where <n> is the number of seconds to wait. I sleep for 120 seconds but I guess it can be much shorter, not tested yet.

Echo "DONE"

For some strange issue I had problems with ^M characters, dos files and ? (question marks) after filenames. This is due to a bug I guess, but as I explained earlier in my previous article, Patrick van Rantwijk showed a workaround I used to start commands. It is adding ' | echo "DONE" ' to the end of the command.

So there you go, I'll need some tidying up to do and I'll post my script soon. Will let you know. But using the above methods you should be able to get EVERYTHING working. Goodluck.

I've not been using the UDA for a long time since I didn't needed it. But I wanted to have my home environment scripted and I noticed a new version was out. So I thought: 'Lets give it a try'. The new features are awesome: check Fokkema's post.

I downloaded beta5 from here: http://www.ultimatedeployment.org/uda20beta.html . First disapointment: it wasn't OVF format. No problemo, I used my 'datastore browser' and uploaded the files to VMFS. Added the .vmx to my inventory and... too bad... the .vmdk is in sparse format which my ESX4i didn't support (must be a VMware Workstation file or something). So I used vCenter Converter to import the VM to my ESX4i server. After quite some time I was ready to go. And today I noticed a funny Twitter post:

"@jpiscaer downloading the newly uploaded UDA OVF"

So I asked him where to find it:

"@jpiscaer @boukeg http://bit.ly/M27lN . Issue with backup up ESXi server solved by moving all VM's to a single LUN"

Well thanks! It's a bit too late for me, but it might help you.

Setup is very easy: it asks: ip address, password, and what services (dchp, tftp, smb, nfs, etc...) to enable. Great! I'm using a Windows 2003 DHCP/DNS server so I was able to disable the DHCP server (which is by default deselected).

Next thing I did was adding some settings in my Windows 2003 DHCP Server. Added:

So that is scope options:
066 <to the ip address or host name to uda>
067 pxelinux.0

If you want to use the DHCP server in the UDA, fine, just configure it. The webinterface is very intuitive.

Next thing was connecting an ESX 3.5U4 iso which I have on NFS. And mounted it, this was without any problems.

Then I created a template. Works like a charm. You need to customize yourself, but the layout is quite logical. There are some bugs which I'll point out later. First things first: PXE. I have had some BIG issues with PXE. Getting an IP from DHCP took ages!!! After wondering around in Google, I've found that STP Portfast should be enabled on your switch. Now if you have an unmanaged switch, you do not have to worry, it doesn't have STP most of the time, so you can't set STP Portfast. IF... But I have a managed switch ... Which I bought second handed , and I didn't know the IP address nor Password .

But what I do have is a shitty little 5 port gigabit switch lying around somewhere, so I put it in between my eth0 card and switch...

That seemed to work. But the first question was: On what interface do you want to do the installation... Not very unattended right. Added --ksdevice=eth0 to the Kernel option command-line in the UDA template. That worked.

Yessss, after 6 hours (trust me, time flies when you are rebuilding your lab, putting other switches in between other switches, try to hack into a switch here you don't have the password, not know the IP Address..., and not to forget: rebooting about 650 times or so) I finally had something.

So, the installation started, and all goes well. It's bloody fast, installing the files in 2 minutes. And waiting for the first reboot... Yesss, that works too, great...

But... Where are the things I put into my %post? Nothing happened. I have a clean installation of my ESX server, which is great, but what about those virtual switches I created, and what about those port groups?

Well... They didn't work. And after taking a look at the script, looking at the ESX server, rebooted 10 times... I was tempted to stop. But I found some strange things... ? (question marks) in filenames, ^M (Control M) in script files I created. Looking into Google, trying to find issues with ^M and ?. Couldn't find anything. So I tried the UDA forum (see link below). And guess what, I wasn't the only one.

Patrick van Rantwijk from CDG had *exactly* the same issue. And he had a fantastic idea, tricking pxe to add | echo "Done" (do not forget the | - pipe sign here). I was finally getting somewhere. Script seem to work, virtual switched are created, settings are applied.

Except for two things. Enable iSCSI, rescan - and enable VMotion. This is tricky. A lot of posts on the community about this one, none resolved (or with workarounds). I have found that S99local is started BEFORE everything is finished. So I renamed S99local to S99zlocal. Now the script starts last. Which is the way it supposed to. But while the script runs, some core services are not running yet, and the script isn't waiting for it. So what I did was creating a second script, start it in the background, added a sleep timer and did some other important stuff: read this article. This seems to work. This is my final script:

# VMware ESX3 template Kickstart file

# Installation Method
url --url=http://[UDA_IPADDR]/[OS]/[FLAVOR]

# root Password
rootpw vmware

# Authconfig
auth --enableshadow --enablemd5

# BootLoader ( The user has to use grub by default )
bootloader --driveorder=[DISKTYPE] --location=mbr

# Timezone
timezone Europe/Amsterdam

# X windowing System
skipx

# Install or Upgrade
install

# Text Mode
text

# Network install type
network --bootproto static --ip [IPADDRESS] --netmask 255.255.255.0 --gateway 192.168.2.1 --nameserver 192.168.2.155 --hostname [SUBTEMPLATE].jume.local --addvmportgroup=1 --vlanid=0

# Language
lang en_US

# Langauge Support
langsupport --default en_US

# Keyboard
keyboard us

# Mouse
mouse none

# Reboot after install ?
reboot

# Firewall settings
firewall --disabled

# Clear Partitions
clearpart --all --initlabel --drives=[DISKTYPE]

# Partitioning
part /boot --fstype ext3 --size 250 --ondisk [DISKTYPE] --asprimary
part swap --size 1600 --ondisk [DISKTYPE] --asprimary
part / --fstype ext3 --size 5120 --ondisk [DISKTYPE] --asprimary
part /var --fstype ext3 --size 2048 --ondisk [DISKTYPE]
part /tmp --fstype ext3 --size 2048 --ondisk [DISKTYPE]
part /opt --fstype ext3 --size 2048 --ondisk [DISKTYPE]
part /home --fstype ext3 --size 2048 --ondisk [DISKTYPE]
# part None --fstype vmkcore --size 100 --ondisk [DISKTYPE]
# part /vmimages --fstype ext3 --size 1 --grow --ondisk [DISKTYPE]

# VMware Specific Commands
vmaccepteula
vmlicense --mode=server --server=27000@vm90000.jume.local --edition=esxFull

%packages
@base

%vmlicense_text

%post

# Backup original rc.local file
cp /etc/rc.d/rc.local /etc/rc.d/rc.local.bak | echo "Done"

cat > /tmp/esxcfg.sh.tmp <<EOF
#!/bin/sh
# Configure ESX Server

# Create vSwitch1 with a port group of Internal
esxcfg-vswitch -L vmnic1 vSwitch0
esxcfg-vswitch -L vmnic3 vSwitch0
esxcfg-vswitch -A vMotion vSwitch0
esxcfg-vmknic -a vMotion -i [VMOTION] -n 255.255.255.0

# Create a vSwitch4 with a port group iSCSI/iSCSI-CIS using vmnic3
esxcfg-vswitch -a vSwitch1
esxcfg-vswitch -A iSCSI vSwitch1
esxcfg-vmknic -a iSCSI -i [ISCSIVMK] -n 255.255.255.0
esxcfg-vswitch -A iSCSI-COS vSwitch1
esxcfg-vswif -a vswif1 -p iSCSI-COS -i [ISCSISC] -n 255.255.255.0
esxcfg-vswitch -L vmnic2 vSwitch1

# Create a local user for SSH Access - Default password is password
# useradd -p '$1$Rg69B9QA$JUtqStBrjNFbyzyP9zTsf0' -c "New Account Created By Kickstart" newaccount

# Allow ROOT access using SSH - Warning this is NOT the must secure configuration!!!
sed -e 's/PermitRootLogin no/PermitRootLogin yes/' /etc/ssh/sshd_config > /etc/ssh/sshd_config.new
mv -f /etc/ssh/sshd_config.new /etc/ssh/sshd_config

# Enable the SSH client (Out/From an ESX hosts)
esxcfg-firewall -e sshClient

# Setup ntpd parameters
cat > /etc/ntp.conf << NTP
restrict default kod nomodify notap noquery nopeer
restrict 127.0.0.1
server 0.nl.pool.ntp.org
server 1.nl.pool.ntp.org
server 2.nl.pool.ntp.org
server nl.pool.ntp.org
fudge 127.127.1.0 stratum 10
driftfile /etc/ntp/drift
broadcastdelay 0.008
authenticate yes
keys /etc/ntp/keys
NTP

cat > /etc/ntp/step-tickers << STEP
0.nl.pool.ntp.org
1.nl.pool.ntp.org
2.nl.pool.ntp.org
nl.pool.ntp.org
STEP

/sbin/chkconfig –level 3 ntpd on
esxcfg-firewall -e ntpClient

# Update system clock
/sbin/hwclock –systohc –utc

# The –utc setting in the “timezone” command above eliminates the need for updating the clock file
#cat > /etc/sysconfig/clock << CLOCK
#ZONE=”UTC”
#UTC=true
#ARC=false
#CLOCK
EOF

cat > /tmp/esxcfg2.sh.tmp <<EOF
#!/bin/sh

sleep 60

# Enable vMotion
VMK=\$(esxcfg-vmknic -l |grep vMotion |awk '{print \$1}')
vimsh -n -e "/hostsvc/vmotion/vnic_set \$VMK"

# Set-up iSCSI Software Emulator
su - root -c "esxcfg-swiscsi -e"
su - root -c "vmkiscsi-tool -D -a 10.0.0.100 vmhba32"
su - root -c "esxcfg-swiscsi -s"
su - root -c "esxcfg-rescan vmhba32"

/usr/bin/vimsh -n -e "hostsvc/datastore/nas_create software storage /raid0/data/software 1"
EOF

mv /etc/rc3.d/S99local /etc/rc3.d/S99zlocal | echo "Done"

# Make esxcfg.sh run from rc.local and make rc.local reset itself
cat >> /etc/rc.d/rc.local <<EOF
tr -d '\15\32' < /tmp/esxcfg.sh.tmp > /tmp/esxcfg.sh | echo "Done"
chmod +x /tmp/esxcfg.sh | echo "Done"
/tmp/esxcfg.sh 2>&1 /tmp/errout
tr -d '\15\32' < /tmp/esxcfg2.sh.tmp > /tmp/esxcfg2.sh | echo "Done"
chmod +x /tmp/esxcfg2.sh | echo "Done"
/tmp/esxcfg2.sh 2>&1 /tmp/errout
/etc/rc.d/init.d/sshd restart >> /tmp/servicestart
/etc/rc.d/init.d/mgmt-vmware restart >> /tmp/servicestart
/etc/rc.d/init.d/ntpd start >> /tmp/servicestart
mv -f /etc/rc.d/rc.local.bak /etc/rc.d/rc.local | echo "Done"
EOF

So anyway, there are some great resources out there:

VMware Communities

RTFM website

UDA Forum

Update: Gabe contacted me throught Twitter and he had a great tip for the UDA: countdown timer on the PXE Boot menu.

1. Edit /var/public/cgi-bin/config.pl
2. 2- Search for the subroutine WriteDefaultFile.
3. Behind the following part:

local($result)=open(DEFAULT,”>$DEFAULTFILE”);
 print DEFAULT “default menu.c32\n”;
 print DEFAULT “prompt 0\n”;
 print DEFAULT “menu title Ultimate Deployment Appliance\n”;
 print DEFAULT “menu INCLUDE pxelinux.cfg/graphics.conf\n”;

print DEFAULT “label localboot\n”;
 print DEFAULT “MENU LABEL Boot from first local harddrive\n”;
 print DEFAULT “localboot 0\n”;
 print DEFAULT “timeout 120\n”;

I had a small DIY weekend. I installed watercooling (I had it lying around here and didn't used it) on my home ESX server. This server was running ESX 3.5U3 with NFS loaded. The reason for that was simple: it didn't supported the AHCI controller on the Asus P5K board.

I took the chance installing all this:

And the final result:

The backside:

I overclocked my Q6600@2,4GHz to 3GHz. Took it on the safe side and put the FSB to 333. I have 667DDR2 memory so I didn't want to push it. I also upgraded the system to ESX4i. And I'm quite happy with the results, my real time encoded stream to my Playstation 3 is not stuttering anymore, and the overall feeling is that all VMs perform a bit more smooth. Now ESX4i detects the speed correctly, and didn't had any issues overclocking: