Skip to content
This repository has been archived by the owner on Jul 1, 2021. It is now read-only.

Can't find a VM in a folder #117

Open
swalberg opened this issue Jul 28, 2017 · 19 comments
Open

Can't find a VM in a folder #117

swalberg opened this issue Jul 28, 2017 · 19 comments

Comments

@swalberg
Copy link

swalberg commented Jul 28, 2017

I'm trying to use the index to find a VM by its name in vsphere (as opposed to anything reported by the agent) rather than the current method of recursively finding all VMs and comparing the name.

dc.vmFolder.find('foo') seems to work as long as the object is in the root and not a folder. However the other find methods don't seem to care.

(byebug) dc.vmFolder.findByIp('x.x.x.x').name
"myvm"
(byebug) dc.vmFolder.find('myvm')
nil

This object is in a Discovered virtual machine folder.

This is similar to #46. Doesn't seem to matter if I call dc.find_vm 'foo' or dc.vmFolder.find('foo').

@jrgarcia jrgarcia self-assigned this Jul 28, 2017
@jrgarcia
Copy link
Contributor

jrgarcia commented Aug 1, 2017

It looks like find makes a call to FindChild method on the SearchIndex managed object in the SOAP API. The FindChild method states that it

Finds a particular child based on a managed entity name. This only searches the immediate children of a managed entity.

If you're expecting find to be recursive (which honestly, I would have expected too), it's not. I'm not sure about find_vm, I'll have to look into that. At any rate, if these are intentionally not recursive or not easily changed to be, I'll look at adding methods that do support recursion because that's definitely something that's needed. Thanks for bring this to my attention!

@swalberg
Copy link
Author

swalberg commented Aug 1, 2017

Hm, yea that is odd. Are there other APIs I could be missing? Such as to resolve a name to a UUID then do the recursive search on UUID?

@sdorsett
Copy link

sdorsett commented Aug 1, 2017

this has been an issue for a while. I've seen other users of rbvmomi use helper methods like this:
https://github.com/MarkArbogast/vsphere-helper/blob/master/lib/vsphere_helper/helpers.rb#L51-#L68
...which would allow you do this:

credentials = { :host => "192.168.1.10", :user => "root", :password => "vmware", :insecure => true }
vim = RbVmomi::VIM.connect(credentials)
datacenter = vim.rootFolder.childEntity.first

recursive_find_vm(datacenter.vmFolder, "my_vm_name")

@jrgarcia
Copy link
Contributor

jrgarcia commented Aug 1, 2017

@swalberg I will definitely be adding this functionality, but it probably won't be this week. If you need this functionality really soon, I'd suggest doing what @sdorsett mentioned ☝️ for now. At least until I can add this functionality. It's a bummer that this is the case, but it's the best I can offer at the moment.

@sdorsett
Copy link

sdorsett commented Aug 1, 2017

the other way I've seen this done is to use a view to perform the search since it has a recursive flag:

[32] pry(main)> search_name = 'centos-7.3';
[33] pry(main)> vim.serviceContent.viewManager.CreateContainerView({ container: vim.rootFolder, type: ['VirtualMachine'], recursive: true }).view.select {|vm| vm.name == search_name }

=> [VirtualMachine("vm-870")]
[34] pry(main)>

@swalberg
Copy link
Author

swalberg commented Aug 1, 2017

@sdorsett yea we're doing something close to that right now and the problem is that it's really slow. Can take a minute or more in one of our DCs vs seconds when we use the index.

I will look at that search that @sdorsett just mentioned...

@jrgarcia
Copy link
Contributor

jrgarcia commented Aug 1, 2017

@swalberg Thanks for adding that that is slow. I'll make sure to consider that when evaluating possible solutions to this.

@swalberg
Copy link
Author

swalberg commented Aug 1, 2017

Just did a quick test of the ContainerView and it's not any faster than what we have now. I think the problem is that the view returns all the objects for Ruby to search through and the find* do the filtering on the server.

@sdorsett
Copy link

sdorsett commented Aug 1, 2017

I talked to a coworker that has done some tweaking of our rbvmomi based auditing scripts and he recommended maybe trying something like this to filter what is being returned with the vm objects:

require 'rbvmomi';
credentials = { :host => "192.168.1.10", :user => "[email protected]", :password => "vmware", :insecure => true };
@vim = RbVmomi::VIM.connect(credentials);
pc = @vim.serviceInstance.content.propertyCollector;
viewmgr = @vim.serviceInstance.content.viewManager;
rootFolder = @vim.serviceInstance.content.rootFolder;
vmview = viewmgr.CreateContainerView({:container => rootFolder, :type => ['VirtualMachine'], :recursive => true});  
filterSpec = RbVmomi::VIM.PropertyFilterSpec(
            :objectSet => [
                :obj => vmview,
                :skip => true,
                :selectSet => [
                    RbVmomi::VIM.TraversalSpec(
                        :name => "traverseEntities",
                        :type => "ContainerView",
                        :path => "view",
                        :skip => false 
                    )]
            ], 
            :propSet => [
                { :type => 'VirtualMachine', :pathSet => ['name']}
            ]
        );
result = pc.RetrieveProperties(:specSet => [filterSpec])

@sdorsett
Copy link

sdorsett commented Aug 1, 2017

if you look at the returned vm object it will only contain a 'name' property:

[11] pry(main)> result[0]
=> ObjectContent( missingSet: [], obj: VirtualMachine("vm-529"), propSet: [DynamicProperty( name: "name", val: "template-centos-68" )] )

...but you can still call any typical methods through .obj of this object:

[15] pry(main)> result[0].obj
=> VirtualMachine("vm-529")
[16] pry(main)> ls result[0].obj
RbVmomi::BasicTypes::ManagedObject#methods: ==  []  _call  _connection  _get_property  _ref  _set_property  collect  collect!  eql?  hash  pretty_print  to_s  wait_until
RbVmomi::VIM::ExtensibleManagedObject#methods: availableField  availableField=  setCustomValue  setCustomValue!  value  value=
RbVmomi::VIM::ManagedEntity#methods:
  Destroy_Task   Reload!       alarmActionsEnabled   configIssue=   customValue         declaredAlarmState=  effectiveRole   name=           parent   permission   recentTask   tag=
  Destroy_Task!  Rename_Task   alarmActionsEnabled=  configStatus   customValue=        disabledMethod       effectiveRole=  overallStatus   parent=  permission=  recentTask=  triggeredAlarmState
  Reload         Rename_Task!  configIssue           configStatus=  declaredAlarmState  disabledMethod=      name            overallStatus=  path     pretty_path  tag          triggeredAlarmState=
RbVmomi::VIM::VirtualMachine#methods:
  AcquireMksTicket           EnableSecondaryVM_Task                        QueryUnownedFiles              StopRecording_Task                 guestHeartbeatStatus
  AcquireMksTicket!          EnableSecondaryVM_Task!                       QueryUnownedFiles!             StopRecording_Task!                guestHeartbeatStatus=
  AcquireTicket              EstimateStorageForConsolidateSnapshots_Task   RebootGuest                    StopReplaying_Task                 guest_ip
  AcquireTicket!             EstimateStorageForConsolidateSnapshots_Task!  RebootGuest!                   StopReplaying_Task!                layout
  AnswerVM                   ExportVm                                      ReconfigVM_Task                SuspendVM_Task                     layout=
  AnswerVM!                  ExportVm!                                     ReconfigVM_Task!               SuspendVM_Task!                    layoutEx
  AttachDisk_Task            ExtractOvfEnvironment                         RefreshStorageInfo             TerminateFaultTolerantVM_Task      layoutEx=
  AttachDisk_Task!           ExtractOvfEnvironment!                        RefreshStorageInfo!            TerminateFaultTolerantVM_Task!     macs
  CheckCustomizationSpec     MakePrimaryVM_Task                            RelocateVM_Task                TerminateVM                        network
  CheckCustomizationSpec!    MakePrimaryVM_Task!                           RelocateVM_Task!               TerminateVM!                       network=
  CloneVM_Task               MarkAsTemplate                                RemoveAllSnapshots_Task        TurnOffFaultToleranceForVM_Task    parentVApp
  CloneVM_Task!              MarkAsTemplate!                               RemoveAllSnapshots_Task!       TurnOffFaultToleranceForVM_Task!   parentVApp=
  ConsolidateVMDisks_Task    MarkAsVirtualMachine                          ResetGuestInformation          UnmountToolsInstaller              reloadVirtualMachineFromPath_Task
  ConsolidateVMDisks_Task!   MarkAsVirtualMachine!                         ResetGuestInformation!         UnmountToolsInstaller!             reloadVirtualMachineFromPath_Task!
  CreateScreenshot_Task      MigrateVM_Task                                ResetVM_Task                   UnregisterVM                       resourceConfig
  CreateScreenshot_Task!     MigrateVM_Task!                               ResetVM_Task!                  UnregisterVM!                      resourceConfig=
  CreateSecondaryVMEx_Task   MountToolsInstaller                           RevertToCurrentSnapshot_Task   UpgradeTools_Task                  resourcePool
  CreateSecondaryVMEx_Task!  MountToolsInstaller!                          RevertToCurrentSnapshot_Task!  UpgradeTools_Task!                 resourcePool=
  CreateSecondaryVM_Task     PowerOffVM_Task                               SendNMI                        UpgradeVM_Task                     rootSnapshot
  CreateSecondaryVM_Task!    PowerOffVM_Task!                              SendNMI!                       UpgradeVM_Task!                    rootSnapshot=
  CreateSnapshotEx_Task      PowerOnVM_Task                                SetDisplayTopology             add_delta_disk_layer_on_all_disks  runtime
  CreateSnapshotEx_Task!     PowerOnVM_Task!                               SetDisplayTopology!            capability                         runtime=
  CreateSnapshot_Task        PromoteDisks_Task                             SetScreenResolution            capability=                        snapshot
  CreateSnapshot_Task!       PromoteDisks_Task!                            SetScreenResolution!           config                             snapshot=
  CustomizeVM_Task           PutUsbScanCodes                               ShutdownGuest                  config=                            storage
  CustomizeVM_Task!          PutUsbScanCodes!                              ShutdownGuest!                 datastore                          storage=
  DefragmentAllDisks         QueryChangedDiskAreas                         StandbyGuest                   datastore=                         summary
  DefragmentAllDisks!        QueryChangedDiskAreas!                        StandbyGuest!                  disks                              summary=
  DetachDisk_Task            QueryFaultToleranceCompatibility              StartRecording_Task            environmentBrowser                 update_spec_add_delta_disk_layer_on_all_disks
  DetachDisk_Task!           QueryFaultToleranceCompatibility!             StartRecording_Task!           environmentBrowser=
  DisableSecondaryVM_Task    QueryFaultToleranceCompatibilityEx            StartReplaying_Task            guest
  DisableSecondaryVM_Task!   QueryFaultToleranceCompatibilityEx!           StartReplaying_Task!           guest=
instance variables: @connection  @ref  @soap
[17] pry(main)>

@swalberg
Copy link
Author

Sorry I dropped the ball on this one... Playing around with it now and filtering the properties doesn't seem to help at all.

Some co-workers who use powershell say that get-vm -name myvm is really fast, I wonder how they do it?

@jrgarcia
Copy link
Contributor

Thanks for the update. I'll see if I can find someone on the PowerCLI team that may have a few pointers on possible issues.

@solohan83
Copy link

Hi,

Do we know if we plan to have a fix for this?

I have tried to use the helper, but it is not returning the VM I am looking for.

@swalberg
Copy link
Author

So far I seem to have something based on @sdorsett's work.

    pc = @vim.serviceInstance.content.propertyCollector
    viewmgr = @vim.serviceInstance.content.viewManager
    rootFolder = @vim.serviceInstance.content.rootFolder
    vmview = viewmgr.CreateContainerView(container: rootFolder,
                                         type: ['VirtualMachine'],
                                         recursive: true)

    filterSpec = RbVmomi::VIM.PropertyFilterSpec(
      objectSet: [
        obj: vmview,
        skip: true,
        selectSet: [
          RbVmomi::VIM.TraversalSpec(
            name: "traverseEntities",
            type: "ContainerView",
            path: "view",
            skip: false 
          )]
      ],
      propSet: [
        { type: 'VirtualMachine', pathSet: ['name']}
      ]
    )
    result = pc.RetrieveProperties(specSet: [filterSpec])
    vm = result.find { |r| r.propSet.find { |p| p.name == 'name'}.val == vmname }
    vm ? vm.obj : nil
  end

The key is not to look at result.obj until the very end -- it causes a load from the server. You want to look at the propSet for the name.

swalberg added a commit to swalberg/knife-vsphere that referenced this issue Mar 15, 2018
This PR makes use of the
[ContainerView](https://www.vmware.com/support/developer/vc-sdk/visdk41pubs/ApiReference/vim.view.ViewManager.html)
to retrieve a cut down list of servers from which the desired object can
be found, after which the object itself can be downloaded. Previously,
we'd download each VM (about 3.5K and several round trips to the server)
to find the VM we want.

From
```
sean~/play/knife-vsphere (find_speed %)$ time knife vsphere vm disk list myserver
  0          Hard disk 1      nosnapshots_ds4 64.00 GiB

real    1m56.153s
user    0m9.536s
sys    0m1.713s
```
to
```
sean~/play/knife-vsphere (find_speed *%)$ time be knife vsphere vm disk list myserver
/Users/sean/.gem/ruby/2.4.1/gems/builder-3.2.2/lib/builder/xchar.rb:111: warning: constant ::Fixnum is deprecated
  0          Hard disk 1      nosnapshots_ds4 64.00 GiB

real    0m2.929s
user    0m1.038s
sys    0m0.236s
```

This'll have to be retrofitted to the other commands, which we can do
over time.

See vmware-archive/rbvmomi#117 for some more context
swalberg added a commit to swalberg/knife-vsphere that referenced this issue Mar 15, 2018
This PR makes use of the
[ContainerView](https://www.vmware.com/support/developer/vc-sdk/visdk41pubs/ApiReference/vim.view.ViewManager.html)
to retrieve a cut down list of servers from which the desired object can
be found, after which the object itself can be downloaded. Previously,
we'd download each VM (about 3.5K and several round trips to the server)
to find the VM we want.

From
```
sean~/play/knife-vsphere (find_speed %)$ time knife vsphere vm disk list myserver
  0          Hard disk 1      nosnapshots_ds4 64.00 GiB

real    1m56.153s
user    0m9.536s
sys    0m1.713s
```
to
```
sean~/play/knife-vsphere (find_speed *%)$ time be knife vsphere vm disk list myserver
/Users/sean/.gem/ruby/2.4.1/gems/builder-3.2.2/lib/builder/xchar.rb:111: warning: constant ::Fixnum is deprecated
  0          Hard disk 1      nosnapshots_ds4 64.00 GiB

real    0m2.929s
user    0m1.038s
sys    0m0.236s
```

This'll have to be retrofitted to the other commands, which we can do
over time.

See vmware-archive/rbvmomi#117 for some more context
swalberg added a commit to swalberg/knife-vsphere that referenced this issue Mar 19, 2018
This PR makes use of the
[ContainerView](https://www.vmware.com/support/developer/vc-sdk/visdk41pubs/ApiReference/vim.view.ViewManager.html)
to retrieve a cut down list of servers from which the desired object can
be found, after which the object itself can be downloaded. Previously,
we'd download each VM (about 3.5K and several round trips to the server)
to find the VM we want.

From
```
sean~/play/knife-vsphere (find_speed %)$ time knife vsphere vm disk list myserver
  0          Hard disk 1      nosnapshots_ds4 64.00 GiB

real    1m56.153s
user    0m9.536s
sys    0m1.713s
```
to
```
sean~/play/knife-vsphere (find_speed *%)$ time be knife vsphere vm disk list myserver
/Users/sean/.gem/ruby/2.4.1/gems/builder-3.2.2/lib/builder/xchar.rb:111: warning: constant ::Fixnum is deprecated
  0          Hard disk 1      nosnapshots_ds4 64.00 GiB

real    0m2.929s
user    0m1.038s
sys    0m0.236s
```

This'll have to be retrofitted to the other commands, which we can do
over time.

See vmware-archive/rbvmomi#117 for some more context
swalberg added a commit to chef/knife-vsphere that referenced this issue Mar 19, 2018
* Improve the speed of finding an individual VM

This PR makes use of the
[ContainerView](https://www.vmware.com/support/developer/vc-sdk/visdk41pubs/ApiReference/vim.view.ViewManager.html)
to retrieve a cut down list of servers from which the desired object can
be found, after which the object itself can be downloaded. Previously,
we'd download each VM (about 3.5K and several round trips to the server)
to find the VM we want.

From
```
sean~/play/knife-vsphere (find_speed %)$ time knife vsphere vm disk list myserver
  0          Hard disk 1      nosnapshots_ds4 64.00 GiB

real    1m56.153s
user    0m9.536s
sys    0m1.713s
```
to
```
sean~/play/knife-vsphere (find_speed *%)$ time be knife vsphere vm disk list myserver
/Users/sean/.gem/ruby/2.4.1/gems/builder-3.2.2/lib/builder/xchar.rb:111: warning: constant ::Fixnum is deprecated
  0          Hard disk 1      nosnapshots_ds4 64.00 GiB

real    0m2.929s
user    0m1.038s
sys    0m0.236s
```

This'll have to be retrofitted to the other commands, which we can do
over time.

See vmware-archive/rbvmomi#117 for some more context

* Use the new finding on property set

* Add this to vm state, too

* Add same thing to vm config

* Add vm lookup to snapshot
@nimDevOps
Copy link

Just want to share my result with latest rbvmomi 1.12.0, the issue is still there, vm template in folder cannot be found. I use vcenter kitchen driver and the only way to work is to place vm template in the root folder which is not allowed to have in most production vmware vcenter farms.
If there are any plans to fix this issue?

@jrgarcia
Copy link
Contributor

I'll look into this again and see if I can figure out what's going on.

@swalberg
Copy link
Author

@nimDevOps if you downgrade to the older chef-provisioning-vsphere, it supports templates and VMs in subfolders for the kitchen driver.

@nimDevOps
Copy link

@jrgarcia Thanks! I will wait for your suggestions/findings.

@nimDevOps
Copy link

Do you have any updates on this ?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants