Thursday, September 18, 2014

VMworld 2014 Recap

VMworld 2014 concluded jst a couple of weeks backThought of doing a quick recap. The following is extracted from numerous sources such as VMware.com and http://vinfrastructure.it/
  • VMware EVO:RAIL is the new VMware’s hyperconverged solution for building Software Defined Data Center. For more information see the related post. Actually is not clear on which version of vSphere will be base (the edition will be the Enterprise Plus one).
EVO: RAIL Software
The EVO: RAIL software bundle is fully loaded onto the EVO: RAIL qualified partner’s hardware. This software bundle is comprised of:
EVO: RAIL Deployment, Configuration, and Management
• vSphere Enterprise Plus, including ESXi for compute
• VMware Virtual SAN™ for storage
• vCenter Server
• vCenter Log Insight™

Configuration
EVO: RAIL Configuration has three options: Just Go! or Customize Me! or Upload Configuration File.
With Just Go!, EVO: RAIL automatically configures a default set of IP addresses and hostnames for extremely fast deployment in a green-field scenario. Configure your TOR switch and click the Just Go! button. All you have to create are two passwords.
With Customize Me!, customers can specify the following configuration parameters:
• Hostnames for vCenter Server and ESXi™ hosts naming scheme
• Networking (IP ranges and/or VLAN ID): ESXi, Virtual SAN, vSphere vMotion®, vCenter Server, VM Networks
• Passwords: ESXi hosts and vCenter Server, optional Active Directory authentication
• Globals: Time zone; your existing NTP, DNS, and Proxy servers; logging: vCenter Log Insight or third-party syslog server
Compute
EVO: RAIL is sized to run approximately 100 average-sized, general-purpose, data center VMs. Actual capacity varies by VM size and workload. There are no restrictions on application type. EVO: RAIL supports any application that a customer would run on vSphere.
General-purpose VM profile: 2 vCPU, 4GB vMEM, 60GB of vDisk, with redundancy
EVO: RAIL is optimized for Horizon View with configuration options that allow up to 250 View VMs on a single EVO: RAIL appliance. Actual capacity varies by desktop size and workload.
Horizon View virtual desktop profile: 2vCPU, 2GB vMEM,32GB vDisk linked clones

From the tecnical point of view the EVO:RAIL “building block” is a 2U system composed by 4-Node unit (just microserver), where each node is an independent physical server within the 2U enclosure (this make microserver different, for example, from blades).
The form factor (initially will be only this) has been choosed for simplify the decision (no choice, just add more node) but also the deployment and the efficiency.
Each of the four nodes in a EVO:RAIL appliance have (at a minimum):
  • Two Intel E5-2620 v2 six-core CPUs
  • 192GB of memory
  • Internal Drive Bays for the entire appliance – up to 24 hot plug 2.5 drives
  • One SLC SATADOM or SAS HDD as the ESXi™ boot device
  • Three SAS 10K RPM 1.2TB HDD for the VMware Virtual SAN™ datastore
  • One 400GB MLC enterprise-grade SSD for read/write cache
  • One Virtual SAN-certified pass-through disk controller
  • Two 10GbE NIC ports (configured for either 10GBase-T or SFP+ connections)
  • One 1GbE IPMI port for remote (out-of-band) management
  • 1 x Expansion Slots PCI-E
  • Dual PSU – rated between 1600W
Each EVO partner could make something different, but those will be the minimum requirements, and the form factor actually is fixed (but I suppose that there will be other form factor, for example for VDI environments where GPU are needed).

Will be possible scale out up to  four Hyper-Converged Infrastructure Appliance (HCIA), for a maximum of 16 nodes in a cluster (considering the vSphere limit, will be possible to have more in next release).
The top of rack (ToR) switch is not included in the EVO:RAIL requirements, but of course will be needed (an possible with good redundancy), so probably some EVO partners will build a complete offer.
Note that storage will be provided by Virtual SAN, as could be expected in an Hyper-Converged Infrastructure.
But the most intesting aspect of EVO:RAIL is how easy is deploy, configure and also manage it:
  • Rapid configuration in minutes
  • Simple management with a pure HTML5 interface
  • Easy non-disruptive upgrades
  • Automatically scales out

  • VMware vCloud Suite 5.8 is announced with:
    • Improved business continuity and disaster recovery – Self service, policy based provisioning of DR tiers, increased scalability of protection and recovery capabilities, improvement on SRM integration with vCloud Automation Center. It’s now possible to offer SRM as a Service in vCAC’s self service portal (see alsoVMware vCenter Site Recovery Manager 5.8 First Look).
    • Enhanced next-generation applications, such as Big Data Extensions for Hadoop 2.
    • Improved interoperability with NSX.

What's New

VMware vCloud Suite is an integrated offering for building and managing a VMware vSphere private cloud based on the Software-Defined Data Center architecture that enables IT organizations to achieve transformational outcomes. vCloud Suite 5.8 is the latest release of the suite and has the following enhancements:
  • Proactive support and accelerated support request resolution - Support Assistant, a new vCenter Server plug-in, is included in vCloud Suite and can be configured to automatically collect and transmit ESXi and vCenter Server log bundles to VMware support. This data is used to identify latent vSphere environment issues and provide customers with proactive alerts to prevent downtime and accelerate mean-time to resolution of issues.

  • Expanded big data support - Big Data Extension now supports Hadoop 2 distributions, which includes YARN cluster resource management, making application deployment, lifecycle management, rapid provisioning and higher availability easier for big data applications. Additionally, the Hadoop Template Virtual Machine now uses CentOS 6.4 as its default operating system, which increases performance and provides native support for all Hadoop distributions for use with Big Data Extensions.

  • Improved integration for provisioning DR tiers - Provision predefined DR protection tiers to new virtual machines when using array-based replication. Site Recovery Manager defines DR protection tiers and makes them accessible through a new vCenter Orchestrator plug-in. vCloud Automation Center's blueprints enable the self-service, policy-based provisioning of those tiers for faster provisioning.

  • Policy-based network provisioning - Ability to dynamically provision NSX firewall and routing services customized to an application's requirements with vCloud Automation Center.

  • Improved business continuity/disaster recovery - New integration between Site Recovery Manager and Virtual SAN via vSphere Replication reduces the DR footprint with hyper-converged infrastructure, simplifies protection with policy-based VM-centric storage, and lowers DR-related CAPEX and OPEX

  • Scalable protection and disaster recovery - Improved Site Recovery Manager scalability for array-based replication.

  • Next-generation user interface for disaster recovery administration - Streamlined workflows and integration with the vSphere Web Client improve efficiency of Site Recovery Manager administration and navigation of its UI.

vCloud Suite 5.8 Components

  • ESXi 5.5 Update 2
  • vCenter Server 5.5 Update 2
  • vCenter Orchestrator 5.5.2
  • vCenter Configurations Manager 5.7.2
  • vCenter Operations Manager 5.8.3
  • vCenter Hyperic 5.8.2
  • vCenter Infrastructure Navigator 5.8.2
  • vCloud Director 5.5.2
  • vCloud Automation Center 6.1
  • vCloud Networking and Security 5.5.3
  • vCenter Site Recovery Manager 5.8
  • vSphere Replication 5.8
  • vSphere Data Protection 5.8
  • vSphere App HA 1.1
  • vSphere Big Data Extensions 2.0
  • vSphere Update Manager 5.5 Update 2
  • vCenter Support Assistant 5.5.1.1
  •  
  • VMware vSphere 6 (now in public beta) announcements:
    • vVOLs were announce in the past year, but of course will be a new important and interesting feature. Will be interesting see how a host will be able to handle a lot of “LUNs” (or vVOLs) without too much delay during bus rescan or VM migration… but can be a big change in the block level storage world.
    • Fault Tolerance in vSphere 6 will support 4 CPUs (finally… considering we are still using FT 1.0 from several years, but remember that it cannot, directly, handle application or OS fault).
    • VMware vSphere 6 will support cross vCenter vMotion as also long distance vMotion is enhanced (this could become really interesting in Hybrid Cloud scenarios).
    • Using VMware NSX, network properties will now be vMotioned as well when using long distance vMotion.
vSphere 6 Technical Deep dive
VMware continues to build out its hypervisor core management application vCenter with more functionality. There are no dramatic architectural changes but VMware is moving slowly to pull apart vCenter into its component parts to be able to run more vCenters at scale and is creating a central services function.
Platform Services Controller (PSC)
VMware is introducing a new component called the VMware Platform Services Controller (which had a previous beta name of Infrastructure Controller)
SSO was the first component to be spun out into what is now being built up as the PSC. SSO was first released in 5.1 and had major issues and was rebuilt as SSO 2.0 for vSphere 5.5
vCenter, vCOPs, vCloud Director, vCloud Automation Center can use the PSC as a shared component.
The PSC now contains the following functionality:
·         SSO
·         Licensing
·         Certificate Authority
·         Certificate Store
·         Service (Product) Registration

The Certificate Authority and Certificate Store are new components to at last tame the wild and woefully inadequate management of vSphere certificates. The new VMware Certificate Authority (VMCA) can act as a root certificate authority either managing its own certificates or handling certificates from an external Certificate Authority. VMCA provisions each ESXi host with a signed certificate when it is added to vCenter as part of installation or upgrade. You can view and manage these certificates from the vSphere Web Client and manage the full certificate lifecycle workflow.
Service (Product) registration is a component that all other services register to and is the lookup service in vSphere. It is the service that will tell you all the services that are running in the system.
Other services will be added to the PSC in future releases.
The PSC is built into vCenter and runs as a vPostgres database so there’s no additional database to worry about and it runs in both the Windows and appliance version. The PSCs self replicate and importantly don’t use ADAM so it can replicate between Windows and appliance vCenters.
You can either have the PSC embedded within vCenter Server or run it as an external component to vCenter Server.
VMware suggests if you are running 8 or less vCenters it is best to use PSC embedded with vCenter. vCenter will then connect only to its internal PSC.
If you have more than 8 vCenters, VMware recommends using an external PSC rather than the embedded one. You can then use the PSC as a highly available and load balanced service shared by your many vCenters. You won’t then be connected to one PSC but a pool of them.


Just to clarify, the PSC is a new service and separate from the Inventory Service which handles inventory management of VMs.
vCenter Server Appliance (VCSA)
The VCSA has also been beefed up. With 5.1 you could manage 100 hosts and 3000 powered on VMs. vSphere 6 now allows 1000 hosts and 10,000 powered on VMs.
Oracle is still the only external database supported as Microsoft doesn’t have an officially supported ODBC Driver for Linux (SLES), the one they do have is only community supported.
vSphere Web Client
Continued performance gains and tagging improvements along with all new functionality. It still uses Flash which won’t please many people, will we ever get a native HTML5 web client?
vSphere Client
You may be happy or disappointed to hear that VMware has decided to keep on the VI Client (C# Client) for one more release, vSphere 6.0. After that they say it will definitely be gone. Although the Web Client continues to progress and speed up, customer feedback has been that they would like to continue to use the familiar older client for now. No new functionality is being added to the C# Client so although it will be supported, it is only able to manage an ever decreasing subset of vSphere functionality.
Install and Upgrade
The vCenter installer for Windows has been streamlined. We are back to one installer now with all input up front. There is more and better pre-check functionality. You can choose between embedded and external (pre-existing) PSCs during installation
The upgrade procedure hasn’t changed since vSphere 5.5, vCenter is upgraded using the standalone installer or vSphere Update Manager and ESXi is updated using the .ISO or Update Manager.

·         Fault Tolerance in vSphere 6 will support 4 vCPUs;
·         vSphere 6 will support cross vCenter vMotion;
·         vSphere 6 long distance vMotion is enhanced;
·         Using VMware NSX, network properties will now be vMotioned as well when using long distance vMotion.
·         Content Library.
·         Virtual Volumes (vVols)

VMware Fault Tolerance
First of all, VMware Fault Tolerance (FT) is going to support multiprocessor  virtual machines. VMware FT allows continuous availability of multiprocessor  virtual machines with literally zero downtime and zero data loss, even surviving server failures, while staying completely transparent to the guest software stack, requiring absolutely no configuration of in-guest software. This feature has been available from VMware vSphere 4 but it was only supported with single processor virtual machines. This has been the primary reason that VMware FT is seldom implemented.
Multiprocessor support for VMware FT has been the number one improvement for VMware FT requested by end-users.
With VMware vSphere 6 VMware FT will support 4 vCPU virtual machines.
VMware vMotion enhancements
vMotion is one the features which pushed VMware VI/vSphere into enterprise IT infrastructures. vMotion is the one feature which instantly shows the power of virtualization. From VMware VI to the latest vSphere 5.5, vMotion was only possible within the boundaries of the same Datacenter and vCenter. So vMotion could be performed in a single cluster and across clusters in the same Datacenter managed by a single vCenter Server.
With vSphere 6.0 it will be possible to perform a vMotion across vCenters Servers, across virtual switches, across long distances and routed vMotion networks aligning vMotion capabilities with larger data center environments. vMotion across vCenters will simultaneously change compute, storage, networks, and management. This leverages vMotion with unshared storage and will support local, metro and cross-continental distances.

When a VM moves across vCenters, HA properties are preserved and DRS anti-affinity rules are honoured. The standard vMotion compatibility checks are executed. Cross vCenter vMotion requires 250 Mbps network bandwidth per vMotion operation.
Before VMware vSphere 6, vMotion required Layer 2 connectivity for the vMotion network. With vSphere 6.0 vMotion can be performed using routed vMotion networks.
Another great addition in vSphere 6.0 is being able to do Long-distance vMotion. The idea is to be able to support cross-continental US distances with up to 100+ms RTTs while still maintaining standard vMotion guarantees. Use cases are:
·         Disaster avoidance
·         SRM and disaster avoidance testing
·         Multi-site load balancing and capacity utilisation
·         Follow-the-sun scenarios
You can also use Long-distance vMotion to live move virtual machines onto vSphere-based public clouds, including VMware VCHS now called vCloud Air.

 Content Library
Content library is a centralized and distributed location to store handy files you need when installing, maintaining your infrastructure. Like when you’re installing a new VM, you need an ISO-file on a central datastore. Most of the time administrators keep their own copy polluting your infrastructure with all versions and duplicates of the same content. The vSphere 6 Content Library should solve this, a single location, which can be replicated to another vCenter Server at the remote site.
Content to place on the content library:
·         ISO-files
·         VM Templates
·         Virtual appliances (OVF)
·         Scripts
Features:
·         One central location to manage all content.
·         Support for other file types as well.
·         This content needs to be stored once, but can be shared many times (vCenter, VCD, vCenter and VCD).
·         Deployment from templates, ISO-file, appliances to host or cluster.
·         Deployment into Virtual Data Center.
Virtual Volumes
Virtual Volumes (vVols) is one the new features in vSphere 6.0. Virtual Volumes is part of VMware’s Software Defined Storage story which is split between the control plane with Virtual Data Services which is all policy driven and the data plane with Virtual Data Plane which is where the data is actually stored. This completely changes the way storage is presented, managed and used by vSphere/ESXi, making the storage system virtual machine-centric. With current versions of VMware vSpher all storage is LUN-centric or volume-centric.
With VAAI in vSphere 5, storage operations can be offloaded to the storage arrays. vVols takes this a step further and makes storage arrays aware of individual VMDK files.


  • VMware Integrated OpenStack (VIO), a new virtual appliance for simple deployment of OpenStack integrated with vSphere.
    • VMware + OpenStack = Better Together. OpenStack offers already integration with vCenter/ESX, NSX, vSphere Datastores and VSAN, using OpenStack Nova, Neutron, Cinder and Glance.
    • New is VMware Integrated OpenStack (VIO) is the combination of OpenStack and vCloud Suite. A special OpenStack Virtual Appliance will be available for download. The OpenStack appliance will connect and prepare you vSphere environment to be used by OpenStack.

Hybrid Cloud
VMware vCloud Air (already announced last week) is the new name for vCloud Hybrid Service (vCHS). A pricing calculator has also already been released, and VCDX Chris Colotti shares some information about this new tool:New vCloud Air Pricing Calculator.
Based on top of the IAAS service, vCloud Air will also deliver DR services, desktop services and platform services. vCloud Air also includes SaaS-, object based storage-, automation- and databaseservices and is becoming much more similar (or at least complete in the offer) like AWS or Azure:
  • DevOps as a service
  • Database as a service with support for MS SQL Server and MySQL as a first step. Other database platforms will follow.
  • Object based storage VMware vCloud Air Object Storage (based on EMC ViPR technology) – Designed to offer extremely scalable, cost effective, and durable storage for unstructured data, VMware vCloud Air Object Storage will enable customers to easily scale to petabytes and only pay for what they use. VMware vCloud Air Object Storage will include lifecycle management, versioning and large object features that will simplify and reduce management overhead and for highly available hybrid application deployments.
  • Mobility services offered by vCloud Air are:
    • Enterprise Mobility Management – Offering an industry-leading platform for mobile device, application, email, browser and content management, AirWatch® by VMware solutions will be available this year on vCloud Air. AirWatch provides a simplified, efficient way to manage an organization’s mobile footprint across employee-owned, corporate-owned and shared devices from a centralized console.
    • Mobile Backend as a Service (mBaaS) – Organizations will be able to scale their ability to build mobile applications and integrate them with corporate backend systems and third-party cloud services through mBaaS solutions on vCloud Air, offered by leading enterprise BaaS provider Kinvey, enterprise mobile platform, built.io, and Node.js community leader StrongLoop.
    • Mobile Application Development Platform (MADP) – Developers will be able to create mobile applications on vCloud Air using mobile web application development tools from Sencha’s high-performance, HTML5 mobile application platform Sencha Touch Bundle and cross-platform application development capabilities, including Appcelerator Titanium or the extensible, integrated Appcelerator Platform.
    • Pivotal CF Mobile Services – Enterprise Platform as a Service leader Pivotal® is extending Pivotal CF on vCloud Air with mobile backend capabilities such as Push Notifications, API Gateway and Data Sync, all at enterprise standards of compliance and security.
    • Rapid Application Delivery – Customers can create, deploy, manage and change both mobile and web applications on vCloud Air with the high productivity application platform from OutSystems.
Automation and Management
This area has a lot of interesting new announce (and we can also consider EVO:RAIL another news for this area).
The interesting part is the cloud vision of VMware that is still based on a private part (VMware vSphere and VMware vCloud Suite and on a public part (now renamed in vCloud Air). In this vision vCAC is a powerful automation tool to be used in the “private” part and there is a new tool that could be used from the public part.

  • vCloud Automation Center 6.1 is announced
  • VMware vRealize Suite, a new suite for cloud management and is a complete stack for managing a SDDC and public cloud infrastructure (IaaS). Seems something interesting for cloud manament and also similar to Platform9 (maybe was the same initial team that has start both projects).
    Actually the idea seems similar to the VMware Go tool from some years ago (I think to remember that there was a version to be used as a SaaS, but this one seems more promising and can manage also other clouds.
Network Virtualization
As written VMware has announce both NSX 6.1 for vSphere and also NSX Multi Hypervisor 4.2.
  • VMware NSX 6.1 for vSphere:
    • NSX integrates with vSphere 5.5 and newer;
    • Allows integration with external DHCP servers in the physical world;
    • Several different DHCP server can be configured;
    • Two stage ECMP support;
    • L2 VPN (including VLAN trunking) from two different NSX edges between two different (stretched) datacenter. This feature is comparable with the Cisco OTV feature;
    • Load balancing improvements: UDP & FTP load balancing is supported;
    • Seamless integration with F5 firewalls;
    • Enhancements to the NSX distributed firewall include: reject action, enhancements to troubleshoot and monitoring;
  • VMware NSX Multi Hypervisor 4.2:
    • Includes HA/hitless upgrade;
    • DHCP relay feauture (same as in NSX 6.1 for vSphere);
    • OVS performance enhancements;
    • GA in Q3 2014.
  • Enhanced integration with VMware vCloud Automation Center 6.1. Pre-created shared Distributed Logical Routers, dynamic creation of security groups per application with default isolation policy, create security groups per tenant, assign security tags per vApp.
End User Computing
VMware Horizon 6 was announced several months ago, but on day 2 of VMworld 2014 VMware made some important announcements in the  End User Computing space:
  • VMware, NVIDIA and Google Unveil Future of Graphics-Rich Applications Delivered on Enterprise Cloud Desktops – More information is available here.
  • VMware and SAP Collaborate to Deliver Mobile Security and Simplified User Experience for Mobile Applications – More information is available here.
  • VMware Unifies Mobile, Desktop and Content Management With VMware Workspace Suite – More information here.
  • VMware announces New Horizon DaaS Services and Expansion to Europe – More information in this article.
  • VMware Delivers the new Workspace Suite – More in this blog article.


Sunday, July 13, 2014

A command to monitor snapshots merging via vcli

Have you been annoyed like me looking at snapshots getting merged and the progress in task list pane shows 99% but not doing anything?

I have been... I even raised a ticket with VMware to let me know if its actually getting merged.

One of the GSS support engineer helped me in getting some visibility of the snapshot merging.

SSH to one of the host.

Browse to the relevant datastore and into the virtual machine folder where the vm resides

cd /vmfs/volumes/<vm folder>

watch -d 'ls -luth | grep -E "flat|sparse|delta"'

That will help keep monitoring the delta disks getting merged to the flat files. 

vim-cmd vimsvc/task_list

This command outputs there is a removeallsnapshot task running.

# vim-cmd vimsvc/task_info haTask-x-vim.VirtualMachine.removeAllSnapshots-xxxxxxxx

With this output make sure your actual task is in running state
(vim.TaskInfo) {
dynamicType = <unset>,
key = "haTask-x-vim.VirtualMachine.removeAllSnapshots-xxxxxxxxx",
task = 'vim.Task:haTask-x-vim.VirtualMachine.removeAllSnapshots-xxxxxxxx',
description = (vmodl.LocalizableMessage) null,
name = "vim.VirtualMachine.removeAllSnapshots",
descriptionId = "VirtualMachine.removeAllSnapshots", à <running process>
entity = 'vim.VirtualMachine:9', <---VIM ID>
entityName = "<vm name>",   à <Virtual Machine name>
state = "running",    à  <make sure the status is running & is not in a error state>
cancelled = false,
cancelable = false,
error = (vmodl.MethodFault) null,
result = <unset>,
progress = 99,  à  <progress of task>
reason = (vim.TaskReasonUser) {
dynamicType = <unset>,
userName = "root",
},
queueTime = "2013-10-02T07:22:02.224526Z",
startTime = "2013-10-02T07:22:02.225526Z",
completeTime = <unset>,
eventChainId = 304060994,
changeTag = <unset>,
parentTaskKey = <unset>,
rootTaskKey = <unset>,
}


In my case the progress was on 99% for a long time