Mavericks kernel testing on AMD (formerly Mountain Lion kernel testing on AMD)

[spakk]

Same here. What bootloader should we be using for this testing? I'm currently using Chameleon 3255.

 

Good work on the compile, theconnactic.

 

I've had the best experience with 10.7.4 with cparm. I will try out various bootloader

[zchef2k]

I've had the best experience with 10.7.4 with cparm. I will try out various bootloader

 

I've noticed that I get nowhere with mach_kernel as a symlink. I'm all good now having abandoned symlinking the kernel and switched bootloaders to chameleon svn r2069.

 

For giggles I moved corecrypto.kext out of the way and now it hangs early in the process with "SAFE BOOT DETECTED - only valid OSBundleRequired kexts will be loaded."

 

Judging by the name of the kext, I can guess what it does. It seems, though, there's a lot dependent on it. I don't guess appeasing it can be avoided.

[theconnactic]

Could you post a screenshot, please?

 

Lordadmiraldrake, first off you are using Mountain Lion? Not Lion?

 

Second, you could always see if you can find an Apple kext in S/L/E with a similar name and then remove it...

 

theconnactic, good find! And that emulator you mentioned earlier...the Leopard kernels indeed had an SSE3 emulator, and a good one too. But you either forgot the extra S in your post or didn't remember that we want an SSSE3 emu.

 

No, Pooky, i didn't forget it. But if we're to write a ssse3 emulator, we'll be better using a successful emulator as a draft, than writing it entirely from scratch. The problem is that i have no sources for the best sse3 emulator, only binaries, which won't help much...

 

I've noticed that I get nowhere with mach_kernel as a symlink. I'm all good now having abandoned symlinking the kernel and switched bootloaders to chameleon svn r2069.

 

For giggles I moved corecrypto.kext out of the way and now it hangs early in the process with "SAFE BOOT DETECTED - only valid OSBundleRequired kexts will be loaded."

 

Judging by the name of the kext, I can guess what it does. It seems, though, there's a lot dependent on it. I don't guess appeasing it can be avoided.

 

What if we do a rollback? Say, use the Lion or even the 64-bit Snow Leopard com.apple.kec.corecrypto? I will try it tonight.

[Lordadmiral Drake]

Lordadmiraldrake, first off you are using Mountain Lion? Not Lion?

 

Second, you could always see if you can find an Apple kext in S/L/E with a similar name and then remove it...

 

theconnactic, good find! And that emulator you mentioned earlier...the Leopard kernels indeed had an SSE3 emulator, and a good one too. But you either forgot the extra S in your post or didn't remember that we want an SSSE3 emu.

 

I have an Intel Hackintosh that is currently running Lion 10.7.4 (trying to upgrade to ML atm) and an AMD Athlon powered HP Laptop that has no OS X on it right now. I want to install ML on the laptop HDD using a USB 3.0 to SATA II adapter.

[PookyMacMan]

No, Pooky, i didn't forget it. But if we're to write a ssse3 emulator, we'll be better using a successful emulator as a draft, than writing it entirely from scratch. The problem is that i have no sources for the best sse3 emulator, only binaries, which won't help much...

 

What if we do a rollback? Say, use the Lion or even the 64-bit Snow Leopard com.apple.kec.corecrypto? I will try it tonight.

Ah OK lol, I see where you're going there. Isn't there a source for the Leo Voodoo kernels? They had an SSE3 emu...I'll see if I can find something on that.

 

Did Snow Leo even have corecrypto? I think only Lion might have that...unless I am completely out of date LOL.

[zchef2k]

Could you post a screenshot, please?

 

This is with corecrypto.kext moved outside /S/L/E and -x -v (or just -v for that matter) as flags.

 

I'm doing this all in ESXi 5.1 on a Supermicro H8DME-2 with dual hex-core Istanbul Opterons.

 

[theconnactic]

Ah OK lol, I see where you're going there. Isn't there a source for the Leo Voodoo kernels? They had an SSE3 emu...I'll see if I can find something on that.

 

If you can, it will help a lot indeed: meklort sent me maxxus' one, but the voodoo emulator performs far far better. Meklort sent me a diff that mentions it, and a binary file of it, which i already had (the binaries are included with the RAWx86's patched kernel, on which i based my own patches). If i can take a look at the C (or even assembly) codes of the voodoo sse3 emulator, our chances improve a lot.

 

Did Snow Leo even have corecrypto? I think only Lion might have that...unless I am completely out of date LOL.

 

I don't know either. If not, we roll back to the Lion one, or remove it altogether, renouncing to the modern cryptography it supposedly provides.

 

This is with corecrypto.kext moved outside /S/L/E and -x -v (or just -v for that matter) as flags.

 

The interesting about it is that it doesn't panic: it hangs instead. So, i guess the CPU id issues are gone for good. Maybe this also indicate that the all-64-bit kexts of Mountain Lion fails to load because of the missing ssse3 instructions. We need further testing, specially with a Bulldozer machine, to confirm this.

 

I'm doing this all in ESXi 5.1 on a Supermicro H8DME-2 with dual hex-core Istanbul Opterons.

 

A little off topic here, but what a heck of a machine! It will be really cool to see it singing the Mountain Lion roaring tune. On the other side, server parts are usually harder to make them to work and usually have less support, even on the Intel side. Did you ever try OSX on it?

[zchef2k]

A little off topic here, but what a heck of a machine! It will be really cool to see it singing the Mountain Lion roaring tune. On the other side, server parts are usually harder to make them to work and usually have less support, even on the Intel side. Did you ever try OSX on it?

 

Not on this rig. I've done an SL build on Opterons before and Lion builds on Xeons.

 

I like it complicated

[Lordadmiral Drake]

We need further testing, specially with a Bulldozer machine, to confirm this.

 

What about AMD E-Series APUs, do they have SSSE3?

[SS01]

No, Pooky, i didn't forget it. But if we're to write a ssse3 emulator, we'll be better using a successful emulator as a draft, than writing it entirely from scratch. The problem is that i have no sources for the best sse3 emulator, only binaries, which won't help much...

I'm guessing you know this already, but Nawcom's Snow Leopard legacy kenel does sse3 emulation, and from what I hear it does a darn good job. I'm sure he'd give you the sources if you asked..

 

Also, you have to remember - back in the 10.5 days, OSx86 was in full swing. With 10.4 we started getting the idea that maybe, just maybe, Apple's trusted computing DRM scheme could be cracked or worked around, and Mac OS X could be run on standard x86 hardware. By 10.5, that idea took off, and pretty much all Mac users smart enough to know what the x86 and PPC differences were started looking in to it (hence distros appeared - the general majority of people wanted a Windows-like, insert-disc-and-click install). Apple halted, or at least slowed down, this growth via various DMCA crackdowns and SMC enchancements, which is why the community is now back to the small group of hackers it was in the 10.4 days. All this goes to say, however, that OSx86's best days were (arguably) the 10.5 ones - so in other words, I'm sure searching on GitHub or Google Code would yield plenty of (potentially valuable) results.

[zchef2k]

On this corecrypto issue. If you look at the Info.plist, the first key is AppleKernelExternalComponent. (Probably not fascinating but it's been a couple of years since I went fiddling around with this stuff). Looking at OSKextLibPrivate.h in the xnu kernel source I found this. Could cpu id(entification) still be at play here?

 

/*!
* @define   kAppleKernelExternalComponentKey
* @abstract A boolean value indicating whether the kext is vending kernel
*		   KPI, and needs special loading behavior.
*/
#define kAppleKernelExternalComponentKey "AppleKernelExternalComponent"
// properties found in the registry root
#define kOSKernelCPUTypeKey			 "OSKernelCPUType"
#define kOSKernelCPUSubtypeKey		  "OSKernelCPUSubtype"
#define kOSStartupMkextCRC			  "OSStartupMkextCRC"			 /* value is 32-bit OSData */
#define kOSPrelinkKextCountKey		  "OSPrelinkKextCount"		    /* value is 32-bit OSNumber */
#define kOSPrelinkPersonalityCountKey   "OSPrelinkPersonalityCount"	 /* value is 32-bit OSNumber */

[ZackehSoul]

CoreCrypto is the module which controls the cryptography of info & comms (in really vague terms, privacy) which is why it's necessary for ML. If there's any issues with corecrypto the machine should usually just flip off and write an error log to system.log (have you checked that?). The reason it then hangs once it's removed is because the system is programmed to shut down the system if the integrity of the module is compromised, or continue with the startup if validated. My guess would be that because it doesn't exist, the system can't figure it out either way and so just hangs because it's unable to confirm either action.

 

Your KP resulting in a backtrace to com.apple.kext.corecrypto is merely due to the fact that the system startup found an issue in it (which doesn't necessarily mean there's something wrong with that specifically, of course) and so flipped a KP.

 

If the issue is actually purely down to corecrypto, it could probably be patched by modifying FIPS_POST in the corecrypto.dylib to throw true even if it returns false. Not sure of the effect this would have on ML functionality though.

 

Can't really be much more specific as I have no AMD machine on me to have a look at it myself, but I figured I'd try throw a couple of ideas out there.

[SS01]

CoreCrypto is the module which controls the cryptography of info & comms (in really vague terms, privacy) which is why it's necessary for ML. If there's any issues with corecrypto the machine should usually just flip off and write an error log to system.log (have you checked that?). The reason it then hangs once it's removed is because the system is programmed to shut down the system if the integrity of the module is compromised, or continue with the startup if validated. My guess would be that because it doesn't exist, the system can't figure it out either way and so just hangs because it's unable to confirm either action.

 

Your KP resulting in a backtrace to com.apple.kext.corecrypto is merely due to the fact that the system startup found an issue in it (which doesn't necessarily mean there's something wrong with that specifically, of course) and so flipped a KP.

 

If the issue is actually purely down to corecrypto, it could probably be patched by modifying FIPS_POST in the corecrypto.dylib to throw true even if it returns false. Not sure of the effect this would have on ML functionality though.

 

Can't really be much more specific as I have no AMD machine on me to have a look at it myself, but I figured I'd try throw a couple of ideas out there.

 

Might be simpler to just whip up a kext to do it tbh.

 

Or --possible noob alert-- just disable corecrypto.

[ZackehSoul]

Or --possible noob alert-- just disable corecrypto.

Disabling corecrypto is what zchef2k did, and it hangs because the system needs to detect it. It's probably easier to have it detected and work around it rather than rewrite the stuff that interacts with it

[SS01]

Disabling corecrypto is what zchef2k did, and it hangs because the system needs to detect it. It's probably easier to have it detected and work around it rather than rewrite the stuff that interacts with it

I see - sorry about that. I'd still use a kext just to be safe - much easier to boot with -x and delete a kext than having to recompile the kernel.

 

Just thought of a possible last resort if we end up in the same 32-bit only boat as Lion:

1. Patch the netkas 32-bit XNU to work with AMD.

2. Switch all ML kexts to their 32-bit Snow Leopard equivalents, or in the case of ML-only kexts rewrite them completely.

3. Install to AMD rig (either via swapping HDs with an Intel rig, or through a ModUSB-like program) and party hard.

 

Obviously very last resort, but just saying it's there.

[zchef2k]

I can test pretty quickly. On my ESXi host I have an Oracle Linux VM to which I can attach the 10.8 VM disk and make modifications (journaling off, of course). I scripted the swap so it takes less than a minute. For me to have a chance I will need emulation as my opterons don't have the SSSE3 instruction set.

 

Back to corecrypto- the first two keys in that kext's Info.plist take booleans and I've toggled them every which way to the same result I reported earlier. Everything else struck me as a time sink.

 

[instant idiot]

Any other things I can do to help?

[zchef2k]

Any other things I can do to help?

 

Welcome to the party.

 

Everyone on this thread is on IRC, right?

[instant idiot]

Uh, no. What channel?

[zchef2k]

chat.osx86.hu - #mountainlion , #hackint0sh , or both.

[instant idiot]

Connecting......

[theconnactic]

What about AMD E-Series APUs, do they have SSSE3?

 

Hi, LordAdmiralDrake.

 

Is this a Bulldozer CPU? If it's the case, the answer is yes, they have.

 

I'm guessing you know this already, but Nawcom's Snow Leopard legacy kenel does sse3 emulation, and from what I hear it does a darn good job. I'm sure he'd give you the sources if you asked..

 

Also, you have to remember - back in the 10.5 days, OSx86 was in full swing. With 10.4 we started getting the idea that maybe, just maybe, Apple's trusted computing DRM scheme could be cracked or worked around, and Mac OS X could be run on standard x86 hardware. By 10.5, that idea took off, and pretty much all Mac users smart enough to know what the x86 and PPC differences were started looking in to it (hence distros appeared - the general majority of people wanted a Windows-like, insert-disc-and-click install). Apple halted, or at least slowed down, this growth via various DMCA crackdowns and SMC enchancements, which is why the community is now back to the small group of hackers it was in the 10.4 days. All this goes to say, however, that OSx86's best days were (arguably) the 10.5 ones - so in other words, I'm sure searching on GitHub or Google Code would yield plenty of (potentially valuable) results.

 

Hi, SS01!

 

Thank you for the suggestion. In fact, i asked him. He didn't answer yet. Meklort shared the maxxus' sse3 emulator (an older emulator, not so good as the voodoo/mercurysquad one, but it's something to start with). He linked a diff of a voodoo build of the Leopard kernel that mentions a sse3emu.c: unfortunately, the entire code was not there. He gave me also a copy of an old 64-bit sse3-supportive Bcopy.s. As you can see, more material from one person than i managed to gather in hours of Google research.

[instant idiot]

The Connactic, are you on any IRC channels?

[theconnactic]

I see - sorry about that. I'd still use a kext just to be safe - much easier to boot with -x and delete a kext than having to recompile the kernel.

 

Just thought of a possible last resort if we end up in the same 32-bit only boat as Lion:

1. Patch the netkas 32-bit XNU to work with AMD.

2. Switch all ML kexts to their 32-bit Snow Leopard equivalents, or in the case of ML-only kexts rewrite them completely.

3. Install to AMD rig (either via swapping HDs with an Intel rig, or through a ModUSB-like program) and party hard.

 

Obviously very last resort, but just saying it's there.

 

As for the arch i386 kernel, i don't think it's a good idea: the kernel extensions aren't open source, it isn't easy, not to say not possible at all, to make 32-bit version of all of them, or any of them, better saying.

 

On the latest iterations of OSX, we had the "easy way" (quoting David Elliot): stick to -legacy mode (pure 32-bit mode) or the way most of the patched kernels were: arch i386 kernels with -force64 (64-bit user land) boot flag. We can avoid it anymore: Mountain Lion is designed to be purely 64-bit, so both the easy way (that saved many non-ssse3 AMD CPU users with Lion, despite the severe limitations) and the 64-bit user land (which prevailed until Snow Leopard) won't help anymore: we must face the challenge of making a full 64-bit patched kernel, and maybe even a ssse3 emulator.

 

Disabling corecrypto is what zchef2k did, and it hangs because the system needs to detect it. It's probably easier to have it detected and work around it rather than rewrite the stuff that interacts with it

 

Hi, Zach.

 

IMO, you are completely right. I intend, as a first try, to do a rollback to the Lion version of com.apple.kec.corecrypto, if it exists.

 

On this corecrypto issue. If you look at the Info.plist, the first key is AppleKernelExternalComponent. (Probably not fascinating but it's been a couple of years since I went fiddling around with this stuff). Looking at OSKextLibPrivate.h in the xnu kernel source I found this. Could cpu id(entification) still be at play here?

 

/*!
* @define kAppleKernelExternalComponentKey
* @abstract A boolean value indicating whether the kext is vending kernel
*		 KPI, and needs special loading behavior.
*/
#define kAppleKernelExternalComponentKey "AppleKernelExternalComponent"
// properties found in the registry root
#define kOSKernelCPUTypeKey			 "OSKernelCPUType"
#define kOSKernelCPUSubtypeKey		 "OSKernelCPUSubtype"
#define kOSStartupMkextCRC			 "OSStartupMkextCRC"			 /* value is 32-bit OSData */
#define kOSPrelinkKextCountKey		 "OSPrelinkKextCount"		 /* value is 32-bit OSNumber */
#define kOSPrelinkPersonalityCountKey "OSPrelinkPersonalityCount"	 /* value is 32-bit OSNumber */

 

Hi, zchef2k!

 

I don't know. What's the relationship did you perceive between this piece of code and a possible remaining cpuid.c issue?

 

What do i intend to do now:

 

1) Remove core crypto altogether, to repeat the hang zchef2k experienced. Then, do the rollback for the Lion version, if there is one;

 

2) Patch the bcopy.s (only the bcopy) of the Mountain Lion XNU using this diff file: xnu-1228.7.58_voodoo_release1.0.diff.zip

 

3) Combine the to options above.

 

I'll do it later tonight or, more probably, tomorrow morning, because there's an important download in progress in my Lion hackintosh right now, and it's a really large file, so it will probably take hours to finish. That's why i uploaded the diff: if someone doesn't want to wait to tomorrow, can do it right now (don't forget to keep a backup of the original bcopy.s). If somebody actually do it, please post a screenshot here.

 

The Connactic, are you on any IRC channels?

 

Yes, i am, thanks to PookyMacMan, who kindly pointed me to the #lion and #MountainLion channels of the ircosx86.hu so i could get advice from meklort and nawcom about this little work of ours.

[Lordadmiral Drake]

Hi, LordAdmiralDrake.

 

Is this a Bulldozer CPU? If it's the case, the answer is yes, they have.

 

Just looked it up on WIkipedia. The E-Series APUs with Zacate core support SSSE3 and SSE4a