ESXi 5 Mac OS X Unlocker

[MSoK]

Just a quick update, VMware have released a patch for ESXi 5.5.0 which takes the build to 1474528, the update is working fine with Unlocker version 1.2.0 or the ESXi only 1.3.0 and OS X Mavericks (10.9.1) including the latest Pre-release: OS X Update Seed 10.9.2 (13C59).

Edited February 13, 2014 by MSoK

[wally007]

Does anyone know how to enter a single user mode ? Usual Cmd+S or S dont work when i hold them down in Console view when machine is powering up.

 

How do you guys repair boot drive permissions ? 

[Clipper87]

I can't get unlocker 1.20 to work with workstation 10.0.1 build-1379776 ?

[Donk]

I can't get unlocker 1.20 to work with workstation 10.0.1 build-1379776 ?

 

Details!

 

Seriously folks there is no way I can help you if this is the level of information posted. 

[Vigger]

Hello

 

I have been trying to install the unlocker on my server and have so far been unsuccesfull so i hope you will be able to help me

 

Info:

HP DL380 G5 with 2x X5365

ESXi 5.5.0 build Build 1331820 (HP customized image)

ESXi is installed onto a flash drive connected to the internal usb connector

I'm trying to install OS X Maverics 10.9 from an usb drive i already have created to reinstall my MBP (so it is tested working).

 

First i made an entire clone of the ESXi usb drive to be able to restore in case of error.

 

I then installed the "official" v1.2.0 from one of my datastores as required (i have the logs if needed)

and rebooted and then one of my VMs failed with the error "Could not start VMX: msg.vmk.status.VMK_ACCESS_DENIED"

But i tried to boot osx anyway and it booted past the "beach ball mode" so that part worked.

I found the post http://www.insanelymac.com/forum/topic/267296-esxi-5-mac-os-x-unlocker/page-31?do=findComment&comment=1958284 with the same error

so i restored my flash drive with the backup so i could be 100% sure i was at the state before the first unlocker install

and installed the posted version 1.2.1.

 

After reboot i again had a VM that wouldn't start up but it wasn't the same vm.

and vsphere was really slow and eventually 3 error messages showed and vsphere disconnected (i grabbed a screen shot, but how do i upload it?)

 

So i restored again and don't know what to do now?

 

Thanks for your time and effort.

 

-Victoria

 

EDIT: The reason why i don't write the error message is because the last 2 was about the connection being lost, the error about the VM that failed to start was overlaid by the error boxes for the connection errors, and vsphere was frozen so i couldn't move the boxes away from the VM error message box.

[Pooch]

As far as I know the unlocker wont work on a USB install due to the way the filesystem is locked. The only way I've gotten around it was to install on a hard drive, apply the unlock and then clone the partitions to a USB stick and then set the logs to use an existing datastore. It's a lot of extra work unless you really don't want to use a physical drive.

[Vigger]

As far as I know the unlocker wont work on a USB install due to the way the filesystem is locked. The only way I've gotten around it was to install on a hard drive, apply the unlock and then clone the partitions to a USB stick and then set the logs to use an existing datastore. It's a lot of extra work unless you really don't want to use a physical drive.

 

Okay, i saw some comments about that someplace here, but people reported the later versions worked so i just assumed it was all good as there were no errors in the output from the install script.

If neither Donk nor MSoK or other with the knowledge have replied when i get home i'll give it a try.

FYI: WinHex does a really good job on the cloning & backup part (it's very easy), i'll post a guide for all USB users out there if this works

 

Thanks Pooch

[Donk]

Okay, i saw some comments about that someplace here, but people reported the later versions worked so i just assumed it was all good as there were no errors in the output from the install script.

If neither Donk nor MSoK or other with the knowledge have replied when i get home i'll give it a try.

FYI: WinHex does a really good job on the cloning & backup part (it's very easy), i'll post a guide for all USB users out there if this works

 

Thanks Pooch

 

There is an issue with the unlocker installer on 5.5.0 due to a memory constraint. I have a new method for running it, which is till being tested. When I am happy with it I will release it. However, all USB users will need to to have a scratch partition available to make full use of it.

[MSoK]

Okay, i saw some comments about that someplace here, but people reported the later versions worked so i just assumed it was all good as there were no errors in the output from the install script.

If neither Donk nor MSoK or other with the knowledge have replied when i get home i'll give it a try.

FYI: WinHex does a really good job on the cloning & backup part (it's very easy), i'll post a guide for all USB users out there if this works

 

Thanks Pooch

Vigger,

 

Until Donk releases the new ESXi Unlocker, you might be able to use the Vanilla ESXi 5.5.0 rather than the HP Build as it does not have the additional HP drivers and is therefore smaller and may leave enough space for the unlocker files, but do use 1.2.1 as it compresses the files as part of the install.

[Vigger]

There is an issue with the unlocker installer on 5.5.0 due to a memory constraint. I have a new method for running it, which is till being tested. When I am happy with it I will release it. However, all USB users will need to to have a scratch partition available to make full use of it.

 

I'm not quite sure if this means it is a install problem or a general problem ie whether Pooch's move-partitions-to-hdd-install-unlocker-move-back-to-usb solution will work?

I would be happy to participate in the beta testing as long as it doesn't have potential to damage the installed VMs if interested?

 

Vigger,

 

Until Donk releases the new ESXi Unlocker, you might be able to use the Vanilla ESXi 5.5.0 rather than the HP Build as it does not have the additional HP drivers and is therefore smaller and may leave enough space for the unlocker files, but do use 1.2.1 as it compresses the files as part of the install.

 

What is the problem exactly? is it a memory or partition layout problem?

Do you know of a way of freeing the necessary space on my current ESXi install? can i simply remove some of the drivers that i don't use or is there more to it?

[Vigger]

As far as I know the unlocker wont work on a USB install due to the way the filesystem is locked. The only way I've gotten around it was to install on a hard drive, apply the unlock and then clone the partitions to a USB stick and then set the logs to use an existing datastore. It's a lot of extra work unless you really don't want to use a physical drive.

 

I don't know if it is the new device type (flash->sata hdd), or the usb disk that sits in an other port (before internal, now in the front), but it doesn't work after i cloned to a hard drive,

 

On boot ESXi writtes following:

 

VMware Hypervisor Recovery

--------------------------------------

BANK5: Invalid configuration,

BANK6: not a VMware boot bank

No hypervisor found.

 

Did you have this problem?

[Pooch]

Yes I did and I swear I did a writeup on this forum about my cloning experience but I can't seem to find it now. As I recall the issue was the GPT not being copied over to the stick properly. I used clonezilla to do it and had to try a few different options before I got it right. I'm pretty sure I ended up having to use gparted to resize the partitions prior to cloning so they'd all fit on the stick. I'll see if I can find my notes on it.

Ok, it's coming back to me now... I had to resize the vmfs partition on my harddrive first using partedutil so it fit onto the usb drive:

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1036609

 

Then I did a full disk clone using clonezilla so that the GPT was properly copied over to the USB stick (it didn't work when I just selectively grabbed partitions) Once I got everything over to the stick I deleted the vmfs and scratch partitions  and repointed the logs to another datastore so the USB drive wasn't constantly being written to.

[nullfork]

Can someone tell me if the unlocker / wrapper scripts are ALWAYS necessary for OSX on ESXi non-Apple?

 

Example; I have installed OSX ML on an AMD based ESXi 5.1 (via the InstallESD/BaseSystem method and amd kernel only) and it works fine. I just wondered why my setup seems to work without the need for patching ESXi?

 

 

[MSoK]

Can someone tell me if the unlocker / wrapper scripts are ALWAYS necessary for OSX on ESXi non-Apple?

 

Example; I have installed OSX ML on an AMD based ESXi 5.1 (via the InstallESD/BaseSystem method and amd kernel only) and it works fine. I just wondered why my setup seems to work without the need for patching ESXi?

nullfork,

 

Since you are not using the vanilla kernel you do not need the unlocker, To use a vanilla ISO or DMG of OS X you need an Intel VT-x enabled processor and the unlocker for non apple hardware, as to meet Apple's EULA VMware checks the hardware and disables OS X if it does not find Apple hardware, the unlocker removes the check.

[Donk]

nullfork,

 

Since you are not using the vanilla kernel you do not need the unlocker, To use a vanilla ISO or DMG of OS X you need an Intel VT-x enabled processor and the unlocker for non apple hardware, as to meet Apple's EULA VMware checks the hardware and disables OS X if it does not find Apple hardware, the unlocker removes the check.

 

To confirm this answer from MSoK; the AMD kernels have got an extra routine that decrypts the protected binaries directly in the kernel. This is different from a vanilla kernel which require the SMC chip keys to decrypt the protected binaries. One of the things the unlocker does is add the relevant details to the vmware-vmx files to return the correct haiku.

[nullfork]

nullfork,

 

Since you are not using the vanilla kernel you do not need the unlocker, To use a vanilla ISO or DMG of OS X you need an Intel VT-x enabled processor and the unlocker for non apple hardware, as to meet Apple's EULA VMware checks the hardware and disables OS X if it does not find Apple hardware, the unlocker removes the check.

 

 

To confirm this answer from MSoK; the AMD kernels have got an extra routine that decrypts the protected binaries directly in the kernel. This is different from a vanilla kernel which require the SMC chip keys to decrypt the protected binaries. One of the things the unlocker does is add the relevant details to the vmware-vmx files to return the correct haiku.

 

 

Thanks for the confirmation guys - all makes sense

[Rubyducky]

Not sure if this has been mentioned...not even sure if it's worth mentioning!

 

Since the unlocker isn't compatible with vCenter managed hosts, there's a very kludgey workaround: nested virtualization!

 

If your hardware is compatible, you can bring up a Windows or Linux VM in ESXi, install Workstation or Player, apply the unlocker to Workstation or Player, and install Mac OS X in Workstation or Player.

 

Here's a guide to see if your hardware is compatible and to enable nested virtualization support: http://www.virtuallyghetto.com/2012/08/how-to-enable-nested-esxi-other.html

 

I was able to install Mavericks in Workstation in a Windows Server 2012 R2 VM, and it runs with no issues that I can tell. It's not even slow! Definitely not an ideal solution but it seems to work

[MSoK]

Not sure if this has been mentioned...not even sure if it's worth mentioning!

 

Since the unlocker isn't compatible with vCenter managed hosts, there's a very kludgey workaround: nested virtualization!

 

If your hardware is compatible, you can bring up a Windows or Linux VM in ESXi, install Workstation or Player, apply the unlocker to Workstation or Player, and install Mac OS X in Workstation or Player.

 

Here's a guide to see if your hardware is compatible and to enable nested virtualization support: http://www.virtuallyghetto.com/2012/08/how-to-enable-nested-esxi-other.html

 

I was able to install Mavericks in Workstation in a Windows Server 2012 R2 VM, and it runs with no issues that I can tell. It's not even slow! Definitely not an ideal solution but it seems to work

Rubyducky,

 

VMware, Donk and I use Nested Virtualisation extensively, have a look at the following thread:

 

http://www.insanelymac.com/forum/topic/292125-how-to-install-and-run-os-x-in-a-nested-instance-of-vmware-esxi-55-within-workstation-10/?p=1948132

[MSoK]

VMware have released ESXi 5.5.0 Update 01 (Build 1623387) which using either the multi product Unlocker (1.2.0) or the ESXi specific Unlocker (1.3.0) works fine with OS X 10.6.8 through 10.9.3. Please note if you are using Unlocker 1.2.0 it will need to be re-installed following the patch update to ESXi 5.5.0 U1.

 

Donk, the update to 5.5.0 U1 (Build 1623387) did not need Unlocker 1.3.0 re-installing, as the patch did not update the local.sh script file.

 

Just out of interest all the above testing was carried out using a nested instance of ESXi running in Workstation 10.0.1.

[Tedstevens]

so this works great in stock esxi 5.5 until you reboot the host.  Then you loose all vsphere management connectivity.  SSH still works, but will not respond to the reboot command, restarting hostd and vpxa fix it sometimes, but this is definitely not ready for even the casual home user at this point.  Unless the mavericks VM is the only VM you are running i suppose...  I used 1.21 and it was rock stable for 3 days until i rebooted my host.  I dont know if there is any reason for me to try 1.30 as all that adds is the persistent scratch partition for usb installs from my understanding.  My install was to a raid1 array on the onboard sas6/ir of my dell t7500.  Surprised there isnt more complaints on this.  I originally patched my dell esxi 5.5 install but later read that it is only for the "vanilla" esxi 5.5 install.  I reloaded with the stock 5.5 image and still was met with the same problems after rebooting the host.

[MSoK]

Ted,

I am sorry you have not had a good experience running OS X under ESXi, the reason for no complaints is because with a few caveats all works well.

 

In terms of caveats, must be a standalone ESXi host, does not support vCenter Server or vCSA for management, should not be used in a production environment.

 

It does however work with custom versions of ESXi, i.e. HP or Dell, however the additional drivers, etc. the manufacturers add leave little space for the Unlocker to make the required changes, hence the introduction of the Unlocker version 1.3.0 and the use of the scratch volume created on a persistent datastore, rather than within the confines of the base OS. The Unlocker v1.3.0 also survives patch updates, all previous versions required re-applying followed by a re-boot.

[Donk]

One teensy-weensy point I haven't formally released 1.3.0 yet!  I will look into releasing it next week.

[amareto]

Hi, I used the 1.2.0 on ESXi 5.5 and installed perfectly Mac OS. Then I upgraded ESXi to 5.5u1 and my VM does not start.

Can I try the 1.3.0? Or should I run 1.2.0 again?

 

Thanks

[Tedstevens]

Ted,

I am sorry you have not had a good experience running OS X under ESXi, the reason for no complaints is because with a few caveats all works well.

 

In terms of caveats, must be a standalone ESXi host, does not support vCenter Server or vCSA for management, should not be used in a production environment.

 

It does however work with custom versions of ESXi, i.e. HP or Dell, however the additional drivers, etc. the manufacturers add leave little space for the Unlocker to make the required changes, hence the introduction of the Unlocker version 1.3.0 and the use of the scratch volume created on a persistent datastore, rather than within the confines of the base OS. The Unlocker v1.3.0 also survives patch updates, all previous versions required re-applying followed by a re-boot.

So i had some more time to troubleshoot this from the beginning and it was working surviving reboots.  It wasn't until i passed through my ATI video card and usb 2.0 hub to the mavericks VM that vsphere managment started crapping out .  So i guess its only buggy if i am using pci passthrough.  Anyways, thanks for the hard work on this donk!  I will be glad to beta test any new releases.  Not sure why the pci passthrough thing messes it up, but having 3 VMs each with their own video card passed through was badass enough.  Now one of them is a mac!

[amareto]

I'm using PCIe Passthrough too. In my case I want to use a Mac OS Server as NAS/Plex Media Server and I want a Sata card directly to the MAC VM.

When I pass the SATA PCIe card, the VM takes like 6min to start.

 

Going to try with a non mac VM to test.

 

Donk thanks for the work on this.