Bloghttp://www.vichong.com/blog/en-CA2010-02-09T10:13:24ZSquarespace Site Server v5.9.1 (http://www.squarespace.com/)NexentaStor 2.2 with LSI MegaRAID SAS 8888ELPhttp://www.vichong.com/blog/2009/12/31/nexentastor-22-with-lsi-megaraid-sas-8888elp.htmlVictor Hong2009-12-31T23:41:21Zhardware lsi megaraid nexenta nexentastor opensolarisFresh install of Nexenta 2.2 running on the following hardware:


We reused a LSI MegaRAID, and had to set each drive connected, to be a single "virtual" RAID0 drive - in the MegaRAID onboard bios configuration (Ctl - H on startup from memory). Then this presents a bunch of individual drives to Nexenta. NexetaStor 2.2 includes all the mega_sas drivers, so once we had the drives recognized by the controller, they then appeared immediately to NexentaStor.

Only after we had flashed both the motherboard and the LSI card (by putting it into a Windows computers to flash) to the latest firmware revisions, were we able to get Nexenta installed at all. Please always flash flash flash to the latest firmware and save yourself a bunch of time!

In the pic below, you can see a pair of 2.5" SSDs which we are using for the Nexenta OS in a RAIDZ mirror config. The mounting kits for these drives on the free PCI slots are from A-Tech Fabrication. I did toy around with running Nexenta off a USB flash stick, and use the SSDs for cache and logs, but will wait until that configuration has been tested by Nexenta.

Finally, would like to wish all the website visitors, and my clients a Happy and Prosperous New Year for 2010! Also proud to have made end of year contribution to Chatswood Computer Consultants charity of choice: MÉDECINS SANS FRONTIÈRES (MSF)

]]>Linksys WAG325N + CISCO VPN issueshttp://www.vichong.com/blog/2009/12/1/linksys-wag325n-cisco-vpn-issues.htmlVictor Hong2009-12-01T20:10:07Zcisco vpn handitax hardware linksys simplefund wag325nI had a long time problem with my client's CISCO VPN connections to the Australian Tax Office - via HandiTax and Simplefund.

Every time they were lodging to the ATO, they had to retry the lodgement several times before it successfully connected and transmitted.

The problem was consistent across two different software applications but there was a common component with the CISCO VPN client. After the light bulb light up above my head, I googled "Linksys WAG325N CISCO VPN problem" and came across the following forum post on the Linksys site:

http://forums.linksysbycisco.com/linksys/board/message?message.uid=113940

I double checked that the IPSec Passthrough was enabled, but the other part that I needed to add to the router configuration was to put entries under Applications & Gaming -> Port Range Triggering. I did another google search for which ports are used by the CISCO VPN, and put in 500, 4500 and 10000.

Here is a screenshot (click it for a larger view) of the relevant page on the router setup.

After I made these changes, lodging returns to the ATO in both HandiTax and Simplefund worked flawlessly each time on first attempt!

]]>VMWare ESXi 4.0 + HP ProLiant DL160 G5http://www.vichong.com/blog/2009/10/26/vmware-esxi-40-hp-proliant-dl160-g5.htmlVictor Hong2009-10-27T06:44:47Zdl160 g5 esxi 4.0 hp hyper-v server 2008 r2 proliant software vmwareConfirmed - VMWare ESXi 4.0 works on a HP ProLiant DL160 G5

Sometimes it is frustrating to find out if your non HCL hardware will work with ESXi before shelling out $ on hardware, but in my case, I don't have to commit this hardware just for VM service, so it was more of a test if anything, and if it works, it may even go into light production duty (probably would advise to go for something on the official HCL for mission critical duty).

For more whitebox and official hardware compatibility info with ESXi, check out:

The following configuration options worked for me:

  • DL160 firmware up to current version as of Oct 2009
  • SATA RAID disabled
  • Installed onto internal SATA hard drive
  • I had to re-order the boot priority of the SATA hard drives (I have 2 installed) for ESXi 4.0 to boot after the installation (via CD - onto one of the hard drives)
  • Both NICs are seen by ESXi 4.0 as well

I also confirmed that this 1U server also worked fine with Microsoft Hyper-V Server 2008 R2. But what I didn't count on, is how locked down and inconvenient it is for lab testing compared to VMWare's bare metal product. Hyper-V does look great at first glance with the free vmotion stuff, but it just takes too long to setup in a non domain lab setup (and can only be managed from a Win2k8, Vista or Windows 7 console!). For now, going to stick with VMWare products. :)

]]>NexentaStor - increasing swap spacehttp://www.vichong.com/blog/2009/10/21/nexentastor-increasing-swap-space.htmlVictor Hong2009-10-21T15:19:45Znexenta nexentastor opensolaris swap file swap spaceHere's another tip I forgot to post yesterday. How to increase the swap space on your NexentaStor server.

When I was trouble shooting our NexentaStore server when it has low available drive space, in the NexentaStor documentation I found that the default swap size is the memory size. For some reason we had a 1 gig swap file, and 4 gig of ram. Perhaps we didn't have all the ram populated on original install, so just to make sure everything was setup correct to help my trouble shooting, I went looking for a way to increase the swap space.

Since there wasn't any option I could find in the web gui, I was certain it was going to be a CLI thang. After some googling, I came across this:

http://www.crypticide.com/dropsafe/article/2649

I ssh-ed into the NexentaStor server as the admin user, and ran:

zfs set volsize=4G syspool/swap

This changed the swap space to the same size as the physical ram in the server.

]]>Deltek Vision - upgrades and service packs - hold your horses!http://www.vichong.com/blog/2009/10/20/deltek-vision-upgrades-and-service-packs-hold-your-horses.htmlVictor Hong2009-10-20T22:35:16Zdeltek upgrade vision windowsI have a client that has moved over to an accounting/erp/crm/do everything for your consulting company software called Deltek Vision - almost 2 years ago.

Working with Deltek's tech support crew has been fine. They provided plenty of information on recommended server configuration etc, and have even remoted in via GoToAssist sessions to help us with SQL database tweaks etc.

With these types of software systems, they are extremely complex, and with this complexity, there are alway little bugs, reports that don't quite work, etc. On top of this problem, there are alway new features in the pipeline. Some of these new features, my client was looking forward to implement (some new reports from memory).

In my client's case, Deltek took the new service pack/point release out from beta and went gold. Accounting was pushing for the upgrade, and we have Deltek support and our backups as a fall back...lets throw down!

Now look what happens - You go ahead, take a quick snapshot of your database, go ahead and install the new patch/software. Everything loads up fine, and you think everything is sweet. Then IT gets a bunch of calls from users, and accounting, that this report isn't working etc etc. One of the coolest things, was the new version of Vision went from a nice convenient web based app, to a full on local running app - *sigh*. Each time a new feature is introduced, and new bug appears, a previous bug fixed, and a new bug fix is promised in the next patch/release - and so on it goes.

Now this has been going on for about a year, and finally, accounting have come to the fact that no longer can they confidently upgrade their primary systems as soon as they go gold. Result - I am in the process of putting together a budget for a virtualized duplicate of their production environment, on which we can run updates, and test as throughly as possible before giving it the green light (or not) on the production boxes.

I guess the long and the short of it is, encourage your clients/account dept etc, to test test test upgrades and new versions first, preferably in a sandbox/test environment. If they don't have one, let them know what best practices are (ie: give me a test environment), give them your best efforts/milage may vary disclaimer, and proceed accordingly. In my clients case, upgrades have been so counterproductive that they have finally decided to do something about it... and what a better way to do it than with VMs! :)

]]>NexentaStor hardware recommendations + Nexenta Support tiphttp://www.vichong.com/blog/2009/10/20/nexentastor-hardware-recommendations-nexenta-support-tip.htmlVictor Hong2009-10-20T21:50:18Zhardware nexenta nexentastor opensolaris recommendation supportI'd like to shout out to Dave who contacted me yesterday. Good to see people excited about this product, and it reminded me to keep posting up stuff on Nexenta and NexentaStor since it is a new product, and there isn't a heap of information out there yet.

I find great tips and solutions to problems daily via blogs and forums, so best I stay ontop of my work related blogging and give back to the community!

So part of this post is to document the hardware that I know works, and or, is in production with NexentaStor.

If you are currently researching the OpenSolaris HCL, it is a good start for picking out your hardware. From here, I would stick to this:

  • Intel mobo and nic chipsets
  • LSI controller cards

Here is a list of what I have in production with NexentaStor:

Motherboards:

SAS/SATA Controllers:

Drive Chassis:

  • Supermicro SC836E1-R800B 16Bay Server Chassis (2 chassis daisy-chained via backplanes)
    • Coupled to Seagate ST31000340NS 1TB SATA drives

Network Cards:

 

Finding the correct cabling for backplanes and controller cards can be a real PITA sometimes, and I highly recommend you talk to these guys to get exactly what you need to hook up all the chassis, backplanes, and controller cards - http://www.cs-electronics.com

Finally, a Nexenta Support tip. If you wish to contact Nexenta support, the quickest and most efficient way to get to them is via the built in support ticketing system from within the web interface on your NexentaStor box (can also send one via command line). What it actually does, it will send Nexenta Support a breakdown of your setup as an attachment, and auto generates a ticket in their system. Don't bother emailing them, it takes much much longer that way - do it via the system they nicely provide coded right in the product.

]]>SharePoint Services WSS 3.0 + Windows 2003 R2 + indexing / searching broken?http://www.vichong.com/blog/2009/10/20/sharepoint-services-wss-30-windows-2003-r2-indexing-searchin.htmlVictor Hong2009-10-20T21:33:58Zindex search sharepoint services windows wss 3.0The problem of search and or indexing that is broken on a Sharepoint site running on a Windows 2003 R2 box, had a bazillion different "solutions". I searched everywhere on the net for a way to fix this annoying problem. Nothing seemed to make it work. I finally lucked out (on my n-th attempted of fixing this) and came across this blog entry:

http://mysharepointblog.com/post/2007/01/Windows-SharePoint-Services-%28WSS%29-30-Search-Setup-Notes.aspx

I have no idea why the indexing died in the first place, but since I'm not a huge fan of the product in the first place (I'm a Google Apps fanboy!), I'm just going to notch it down to inherent complexity of Sharepoint and leave it at that! Lucky for my client, they are just using Sharepoint for their intranet and not their main file store!

]]>Nexenta + NDMP Backup with BackupExec 12.5 + low free space issueshttp://www.vichong.com/blog/2009/10/20/nexenta-ndmp-backup-with-backupexec-125-low-free-space-issue.htmlVictor Hong2009-10-20T21:04:54Zbackupexec ndmp nexenta nexentastor opensolarisNexentaStor has been lots of fun to work with, there are still a few hickups as mentioned earlier in my blog, but it is turning out to be a pretty cool solution for clients that are not wanting to throw down big bucks on boutique NAS/SANes during this recession. Oh, and do I hear de-dupe is around the corner for ZFS?! :)

I was experimenting around with the NDMP Backup options in NexentaStor. I enabled the service, and punched in the details of our Symantec BackupExec 12.5 server, with NDMP option enabled.

Not being really familiar with this option in BackupExec, I just set everything pretty much with the default, and unfortunately, when browsing for backup devices, Nexenta does not appear in the NDMP devices, so off I went and logged a support ticket with Nexenta.

Here is their response, not holding my breath, but it would be seriously kick ass if NDMP backups did work with BackupExec (not to mention would get great speed, and can point backups to "latest" folders within a special backup snapshot for example - nice point in time backups/archives to tape!)

Looks like we do not have many customers using BackupExec. In general, this is our policy to support 3rd party software only if there is enough evidence that this is either a standard, or used by a significant number of users (which would make it de-facto standard). BackupExec is neither. We may be able though to get to it after the next major release 3.0 planned for end of November, early Dec.

Now onto free space in NexentaStor. I do encourage you to keep a close eye on available space on your volumes. We were in a situation where a slight mistake in a rsync script, filled up our Nexenta box. We had on or less than 10% space left, and it ended up with some strange behaviour.

Everything from no access via web interface, the console on the server locking up when trying to remove snapshots via the web interface, and errors and dead drives appearing after scrub jobs were issued (which weren't really dead drives!). After we blew away a bunch of stuff, and snapshots via command line (setup snapshot destroy), things settled down again.

I really like the weekly reports that the system can send to you, surf through them just to make sure available space is in check, and all should be fine.

]]>pfSense - ftp server in DMZ / OPT interface - the userland FTP-Proxy settinghttp://www.vichong.com/blog/2009/8/27/pfsense-ftp-server-in-dmz-opt-interface-the-userland-ftp-pro.htmlVictor Hong2009-08-28T05:25:26Zdmz ftp ftp-proxy opt pfsense userlandHad some fun setting up a FTP server, on a OPT interface on a pfSense router.

On the legacy router setup, we had Virtual IP for the ftp server. With this setup, we would put in a port forward to the ftp server in the DMZ, and everything would play nice.

With pfSense, you need to tweak some settings to get things happening.

After setting up my virtual IPs, and setting up some WAN rules, we couldn't connect to the ftp server via any interface. Problem turns out to be a combination of using the virt IP (as apposed to the primary IP of the WAN interface as the IP you are using to access ftp from the WAN side), and a little pfSense userland ftp-proxy setting

I sifted through some pfSense forum entries, and the following links gave me the clues:

http://forum.pfsense.org/index.php/topic,6218.0.html

http://doc.pfsense.org/index.php/FTP_Troubleshooting

So I went through things step by step, and as suggested:

  1. remove all ftp rules (WAN side) I setup previously
  2. removed the Virtual IP - I HAD to use the WAN interface primary IP address - the Virtual IP would NOT work - had to make an A record change and notify users of the change
  3. went to Interfaces -> WAN -> unticked the Disable the userland FTP-Proxy application (also unticked on the other interfaces)
  4. this auto created a WAN rule: * * * ext_WAN_IP 21 *
  5. then added another WAN rule: * * * ftpserver_opt_IP 21 *
  6. for users in the LAN, I made an internal DNS override for the ftp domain name to point to the ftp server's internal IP address - eg: ftp.blah.com -> 192.168.1.1 (normally on the net, it would point to the WAN interface primary IP address in the A record)

Took a bit of time, but finally sorted it out and now we are back to ftp serving via pfSense!

]]>NexentaStor 2.0/2.1 + Windows 2008 SP2 FAILhttp://www.vichong.com/blog/2009/8/20/nexentastor-2021-windows-2008-sp2-fail.htmlVictor Hong2009-08-20T15:20:00Znexentastor opensolaris sp2 windows windows 2008Last week, I went ahead and upgraded a client's Windows 2008 domain controllers from SP1 to SP2. Reason, I'm always staying ontop keeping their Linux based NAS boxes working nicely with Active Directory etc, and thought it was a quiet time to update stuff across the board.

As it turns out, once Windows 2008 was at SP2 level, the NexentaStor 2.0 box now doesn't work on CIFS shares. The only way to access them is to use the "guest" login (a login box appears when you browse to it from the network).

Viewing the CIFS server log - got this:

Aug 20 13:27:37 server1 smbd[352]: [ID 526780 daemon.notice] Failed to establish NETLOGON credential chain
Aug 20 13:27:37 server1 smbd[352]: [ID 871254 daemon.error] smbd: failed joining domain.lan (UNSUCCESSFUL)
Aug 20 13:27:37 server1 idmap[335]: [ID 694198 daemon.notice] Configuration unchanged

After sending an email to support, they pointed me to:

http://www.nexenta.com/nexentastor-relnotes - Scroll down to v1.1.9 - known problem

and http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6850508

I have been told this will be addressed in September... or we can roll back to SP1 on the domain controllers! WTF, I guess this is Sun that is just slow to move on a pretty big bug. :(

]]>