|
|
| (20 intermediate revisions by 3 users not shown) |
| Line 1: |
Line 1: |
| − | {{Project
| + | #REDIRECT [[VM_server#2014]] |
| − | |State=Active
| |
| − | |Members=Danny Witberg
| |
| − | |Description=Make a mini VMware cluster
| |
| − | }}
| |
| − | == Introduction ==
| |
| − | | |
| − | VMWare and virtual computing in general had many benefits over a physical server farm. One VMWare host can accomodate multiple virtual servers. Shared resources means better use of the actual hardware. In a cluster of hosts, several automation processes can be achieved such as auto restart upon failed VM, and automatic recovery after a failed host. To experiment with such a system, you will have to have a minimum of 3 host servers, and a shared storage system or datapool. My goal with this project is to set up the hardware in a mini 4U cluster system.
| |
| − | | |
| − | [[File:DL360G5.jpg]] | |
| − | | |
| − | == The hardware ==
| |
| − | | |
| − | The cluster is comprised of 4 HP DL360G5 1U servers. They are fairly cheap to come by, can easily be upgraded and are compact. This is an overview of the 4 systems:
| |
| − | | |
| − | 1) DONE
| |
| − | <UL><LI>Hardware platform: HP DL360 G5</LI>
| |
| − | <LI>CPU: 2x Quadcore Xeon E5430 2.66GHz 64 bit 12MB cache</LI>
| |
| − | <LI>Memory: 26GB PC2-5300F ECC Fully buffered memory</LI>
| |
| − | <LI>Harddisk: none installed</LI>
| |
| − | <LI>Network: 2xBCM5705 gigabit Ethernet with offload engine + 2x dual gigabit Intel PRO1000PT</LI>
| |
| − | <LI>Power: 2x 700W hotswap power supply</LI>
| |
| − | <LI>Optical drive: DVD/CD rewriter</LI>
| |
| − | <LI>Management port: ILO2 100Mbit Ethernet</LI>
| |
| − | <LI>todo: Boot disk: 32GB USB drive</LI></UL>
| |
| − | | |
| − | 2) DONE
| |
| − | <UL><LI>Hardware platform: HP DL360 G5</LI>
| |
| − | <LI>CPU: 2x Quadcore Xeon E5430 2.66GHz 64 bit 12MB cache</LI>
| |
| − | <LI>Memory: 26GB PC2-5300F ECC Fully buffered memory</LI>
| |
| − | <LI>Harddisk: none installed</LI>
| |
| − | <LI>Network: 2xBCM5705 gigabit Ethernet with offload engine + 2x dual gigabit Intel PRO1000PT</LI>
| |
| − | <LI>Power: 2x 700W hotswap power supply</LI>
| |
| − | <LI>Optical drive: DVD/CD rewriter</LI>
| |
| − | <LI>Management port: ILO2 100Mbit Ethernet</LI>
| |
| − | <LI>Boot disk: Todo: 32GB USB stick</LI></UL>
| |
| − | | |
| − | 3) DONE
| |
| − | <UL><LI>Hardware platform: HP DL360 G5</LI>
| |
| − | <LI>CPU: 2x Quadcore Xeon E5420 2.5GHz 64 bit 12MB cache</LI>
| |
| − | <LI>Memory: 26GB PC2-5300F ECC Fully buffered memory</LI>
| |
| − | <LI>Network: 2xBCM5705 gigabit Ethernet with offload engine + 2x dual gigabit Intel PRO1000PT</LI>
| |
| − | <LI>Power: 2x 700W hotswap power supply</LI>
| |
| − | <LI>Harddisk: none</LI>
| |
| − | <LI>Optical drive: DVD/CD rewriter</LI>
| |
| − | <LI>Management port: ILO2 100Mbit Ethernet</LI>
| |
| − | <LI>Boot disk: Todo: 32GB USB stick</LI></UL>
| |
| − | | |
| − | 4) Pending...
| |
| − | <UL><LI>Hardware platform: HP DL360 G5 DONE</LI>
| |
| − | <LI>CPU: 1x Quadcore Xeon E5405 2GHz 64 bit 6MB cache DONE</LI>
| |
| − | <LI>Memory: 22GB PC2-5300F ECC Fully buffered memory DONE</LI>
| |
| − | <LI>Network: 2xBCM5705 gigabit Ethernet with offload engine + 2x dual gigabit Intel PRO1000PT DONE</LI>
| |
| − | <LI>Harddisk: 3-5x SATA 500-1000GB 7200RPM 2.5inch drive + 256GB SSD <-- Still open for suggestions</LI>
| |
| − | <LI>Power: 2x 700W hotswap power supply DONE</LI>
| |
| − | <LI>Optical drive: CD rewriter DONE</LI>
| |
| − | <LI>Management port: ILO2 100Mbit Ethernet</LI></UL>
| |
| − | | |
| − | <p>This gives us a combined total of 62,56 GHz CPU power and 78GB of memory for VMWare! The storage unit also has plenty of memory (22GB) for ZFS caching.</p>
| |
| − | | |
| − | [[File:DL360_inside.jpg|800px]]
| |
| − | | |
| − | [[File:Spare_mem.jpg|400px]]
| |
| − | | |
| − | == Upgrades ==
| |
| − | | |
| − | Upgrades: Memory upgrades can be from 4x4GB+4x1GB = 20GB to a configuration of 8x4GB=32GB. CPU upgrade can be to a dual quadcore system. With purchase of a single E5405 CPU, all three VMWare systems can already be equipped with 4xE5420 and 2xE5405 quadcores. I believe it's best to keep the storage server on a fast dual core CPU, like 3,7GHz, because the file server process is mostly single thread. For now it will run on a 2.5GHz quadcore. 16GB should be plenty to run a good ZFS system to host all VM data storage with iSCSI connections.
| |
| − | | |
| − | == Interconnects ==
| |
| − | | |
| − | All four systems are planned with 6x gigabit ethernet hooked up to 2 Dell Powerconnect 5324 switches. Two of the gigabit can be used for the iSCSI connection, two for the VM's connection, and two for the shared vMotion/management interface. The host OS for the iSCSI server is still undetermined, but it has to support ZFS with ZIL and L2ARC capabilities at good speeds. Hopefully an SSD drive will be a positive influence to the fileserver's speed.
| |
| − | | |
| − | [[File:cluster_layout.png]]
| |
| − | | |
| − | == ZFS storage terminology ==
| |
| − | | |
| − | The ZFS file system is an advanced modern file system used when data has to be secure, fast available and efficient. There are a number of things that can speed up read data access that ZFS uses in terms of caching. First of all it tries to use RAM as a cache. This is called L1ARC (level 1 adaptive replacement cache). RAM memory is the fastest available storage in the computer, and the ZFS system will try to use this to speed up read access. If some portion of data is accessed a lot, it places this in RAM to be extremely fast. If data is not cached in RAM, it has to revert to the storage disks. The L2ARC places a layer between these, and can contain very fast storage disks compared to the storage disks, but slower than the RAM. This used to be very fast SCSI disks, but nowadays SSD drives are preffered due to the very low access latency.
| |
| − | | |
| − | If data has to be stored onto a ZFS system, basically is has to write to the disks and will be bound to the speed of those disks. The ZIL or "ZFS intent log" can speed up this by caching the write transactions onto the cache. Often this is a RAM disk, or an SSD drive. If the ZIL is full, the ZFS system commits this data to the storage disks.
| |
| − | | |
| − | With ZFS we can set up sparse volumes. This means a volume can be advertised to the ZFS client at a different capacity than it really is. Lets say, I have a 2TB storage pool available, but I advertise it as a 4TB size volume. When data is filling up the 2TB storage to almost full capacity, additional storage space can be added to the volume without expanding the volume.
| |
| − | | |
| − | Another neat feature is deduplication. When the same data is stored multiple times, the file system can recognise this and only store this data is single time, with multiple references to this. However, this feature can consume a great amount of RAM.
| |
