Mavericks kernel testing on AMD (formerly Mountain Lion kernel testing on AMD)

[PookyMacMan]

Aha! So it is only bcopy! I remember reading that bcopy was the main kernel issue back in the Leopard days and the Pentium/vanilla kernel discussion. But, if it is truly the only culprit, it makes our job that much easier! So, here's what I believe needs to be done:

 

1. Remove all cpuid checks and CPU instruction checks! Particularly with the former, that is mandatory to get the kernel to boot without a blank, black screen.

2. Investigate bcopy.s and translate all SSSE3 instructions to their SSE3 equivalents (harder than the first ). Once that's done, most likely (I believe) all the kernel issues will be taken care of; further issues would probably be fixed by binary patching, like in the olden days.

 

(BTW, anyone with an AMD system and the Lion kernel care to try the binary patching? Look up the binary patches used, apply them to the files necessary, and then see if any issues are fixed? A user earlier mentioned I think the most crucial one, the dyld patch.)

[theconnactic]

Aha! So it is only bcopy! I remember reading that bcopy was the main kernel issue back in the Leopard days and the Pentium/vanilla kernel discussion. But, if it is truly the only culprit, it makes our job that much easier! So, here's what I believe needs to be done:

 

Well, this is a big "if" anyway, but it's indeed a feasible prospect.

 

1. Remove all cpuid checks and CPU instruction checks! Particularly with the former, that is mandatory to get the kernel to boot without a blank, black screen.

 

Not to say that, if there's no other obstacle other than these checks, our work will be done. This would be the best of all scenarios, because it's something quite simple to be done, but it's not likely to happen to be that easy for us. Cross fingers...

 

2. Investigate bcopy.s and translate all SSSE3 instructions to their SSE3 equivalents (harder than the first ). Once that's done, most likely (I believe) all the kernel issues will be taken care of; further issues would probably be fixed by binary patching, like in the olden days.

 

Yeah, here lies the trouble. Not only because of the difficulty degree intrinsic to the task, but because even figure the ssse3 instructions out to start it is also very hard, at least to me. It's why i asked for a more skilled hand to help me: when i examine the bcopy.s file of the Mountain Lion XNU, all i see is ordinary general purpose instructions, like "mov", "add", etc. Okay, but maybe they don't exist at all, sending us to the best of all words, or maybe they're in another file or diffused in various files (the worst possible scenario). The concrete info i have is from Dave Elliot's page, that the bcopy.s routine in the Lion XNU requires ssse3. But even in that XNU, i cannot find anything but the same general purpose instructions, which mean that they are either not there, not visible or (the most probable case) i couldn't read one even looking at it directly because of my insufficient programming skills (i was expecting to find instructions like, say, PHSUBSW, but it obviously doesn't work like that).

 

(BTW, anyone with an AMD system and the Lion kernel care to try the binary patching? Look up the binary patches used, apply them to the files necessary, and then see if any issues are fixed? A user earlier mentioned I think the most crucial one, the dyld patch.)

 

Well, if you or somebody else know which binary patches exactly, link them to me here in this topic, tell me which are the necessary files i have to set as targets, and teach me how to apply a binary patch (i think FileMerge won't do this job, right?), i'll gladly do it.

[bcobco]

if bcopy is the thing, maybe looking at this both gives some ideas...

 

bcopy.S from FreeBSD project

http://www.freebsd.o...te=1.13.2.1.4.1

 

bcopy.s from Apple

http://www.opensourc...mk/i386/bcopy.s

 

 

 

asm.h maybe has some things to look at

Edited September 24, 2012 by bcobco

[theconnactic]

if bcopy.s is the thing, maybe looking at this both gives some ideas...

 

bcopy.S from FreeBSD project

http://www.freebsd.o...te=1.13.2.1.4.1

 

bcopy.s from Apple

http://www.opensourc...mk/i386/bcopy.s

 

Thank you, bcobco. By the way, i forgot to tell you how useful was that link to Intel's manuals.

 

The bcopy.s from Apple is precisely the one i've been studying, trying to find where are the dreaded forbidden instructions and how they look like. But i'll take a look at the BSD one anyway (which is the base for Apple's).

[bcobco]

Thank you, bcobco. By the way, i forgot to tell you how useful was that link to Intel's manuals.

i have updated that post. now is more legible because of its relevance in this task (SupplementalSSE3, SSE3, ...), and points to the correct manuals page. (before, it pointed only to volumen 2 instructions m to z... now points to the full manuals).

[theconnactic]

Oh, i know, and i already downloaded the combo manual, with 3000+ pages

 

Thank you anyway!

[bcobco]

quote from intel developers manual (the pdf with 3020 pages)

 

12.7.2 Checking for SSSE3 Support

Before an application attempts to use the SSSE3 extensions, the application should follow the steps illustrated in Section 11.6.2, “Checking for SSE/SSE2 Support.” Next, use the additional step provided below:

• Check that the processor supports SSSE3 (if CPUID.01H:ECX.SSSE3[bit 9] = 1).

 

if CPUID.01H:ECX.SSSE3[bit 9] = 1
then mach_kernel
else legacy_kernel

[theconnactic]

quote from intel developers manual (the pdf with 3020 pages)

 

 

if CPUID.01H:ECX.SSSE3[bit 9] = 1
then mach_kernel
else legacy_kernel

 

Hi, @bcobco!

 

I'm not sure if i understand your second quote. From where is this piece of code?

 

@PookyMacMan, here's the bcopy.s routine from the x86_64 section (/http://opensource.apple.com/source/xnu/xnu-1699.22.73/osfmk/x86_64/bcopy.s):

 

 

#include

 

/* void *memcpy((void *) to, (const void *) from, (size_t) bcount) */

/* rdi, rsi, rdx */

/*

* Note: memcpy does not support overlapping copies

*/

ENTRY(memcpy)

movq %rdx,%rcx

shrq $3,%rcx /* copy by 64-bit words */

cld /* copy forwards */

rep

movsq

movq %rdx,%rcx

andq $7,%rcx /* any bytes left? */

rep

movsb

ret

 

/* void bcopy((const char *) from, (char *) to, (unsigned int) count) */

/* rdi, rsi, rdx */

 

ENTRY(bcopy_no_overwrite)

xchgq %rsi,%rdi

jmp EXT(memcpy)

 

/*

* bcopy(src, dst, cnt)

* rdi, rsi, rdx

* ws@tools.de (Wolfgang Solfrank, TooLs GmbH) +49-228-985800

*/

ENTRY(bcopy)

xchgq %rsi,%rdi

movq %rdx,%rcx

 

movq %rdi,%rax

subq %rsi,%rax

cmpq %rcx,%rax /* overlapping && src

jb 1f

 

shrq $3,%rcx /* copy by 64-bit words */

cld /* nope, copy forwards */

rep

movsq

movq %rdx,%rcx

andq $7,%rcx /* any bytes left? */

rep

movsb

ret

 

/* ALIGN_TEXT */

1:

addq %rcx,%rdi /* copy backwards */

addq %rcx,%rsi

decq %rdi

decq %rsi

andq $7,%rcx /* any fractional bytes? */

std

rep

movsb

movq %rdx,%rcx /* copy remainder by 32-bit words */

shrq $3,%rcx

subq $7,%rsi

subq $7,%rdi

rep

movsq

cld

ret

 

 

Did you see any ssse3 instructions somewhere? Yeah, neither do i. Therefore:

 

1) They're not there, we're in deep trouble because they can be anywhere;

 

2) They're neither there nor anywhere, we just need to apply AMD patches, fix the CPU ID checks and we're good to go;

 

3) They're there, but we cannot see because... well, because we know not.

 

[bcobco]

Hi, @bcobco!

 

I'm not sure if i understand your second quote. From where is this piece of code?

 

@PookyMacMan, here's the bcopy.s routine from the x86_64 section (/http://opensource.apple.com/source/xnu/xnu-1699.22.73/osfmk/x86_64/bcopy.s):

 

that piece of code in the second quote is invented. i found

if CPUID.01H:ECX.SSSE3[bit 9] = 1

in Volume 1 Chapter 12.5 (page 266) of intel developers manual. But in that manual there are a lot of more things.

 

ps: please put code inside [ code ] [ /code ] marks to make posts easier to read.

 

 

 

 

is there a way to tell a compiler not to make use of specific set of instructions?

would be easier to tell the compiler "instead of SupplementalSSE3 please use SSE3 ok? i invite you a beer dude"

[PookyMacMan]

@PookyMacMan, here's the bcopy.s routine from the x86_64 section (/http://opensource.apple.com/source/xnu/xnu-1699.22.73/osfmk/x86_64/bcopy.s):

 

Did you see any ssse3 instructions somewhere? Yeah, neither do i. Therefore:

 

1) They're not there, we're in deep trouble because they can be anywhere;

 

2) They're neither there nor anywhere, we just need to apply AMD patches, fix the CPU ID checks and we're good to go;

 

3) They're there, but we cannot see because... well, because we know not.

 

I have a feeling that it's #3. But I hope it's 2.

 

See if you can contact meklort on IRC (server irc.osx86.hu, channel will be either #lion or #mountainlion), if he's not there maybe someone else (conti or nawcom maybe) can help you out.

[theconnactic]

Thank you, Pooky!

 

I'll follow your advice! In fact, i confess i could have resorted to it before, but i was wary of bothering them.

 

 

Update: already there, already being trolled, lol. But i'm failing to join the rooms you told me to, Pooky. I'm getting this annoying message:

 

You need to identify with network services to join the room "#lion" on "irc.osx86.hu".

 

Another update: joined them, finally. Never used IRC before, so i think i'm forgiven, lol!

 

Uptading again: talked with Nawcom. He was busy, neverthless he answered me. It was a good conversation, and he briefly explained some of the issues he found when he tried to add support for AMD CPUs in Lion, and the real dimensions of the work we're trying to do. We didn't talked much about the ssse3 instructions and how to find them in the commpage though. Instead, we focused on the AMD patches themselves. It was very brief, as i said, and i expect we talk more about it. He is really a nice guy. He told me that he'll even try something with Mountain Lion on his Bulldozer machine. It will be great if he succeds.

 

 

Later, if i can, i'll try to talk with meklort (is his IRC nickname "meklort"?), if he goes online. Any news from this conversation, if it really happens, i'll post here. Thank you, Pooky, for the IRC chat tip.

[PookyMacMan]

You're welcome. In my spare time I'll see if I can have a word with one of them too.

[theconnactic]

Thank you again, Pooky!

 

I'm with the Mountain Lion XNU open at my screen at this very moment, lol. Two PDFs at its side: the Intel combo guide that bcobco discovered (http://www.intel.com...-2b-manual.html), and my Assembly-learn-it-yourself companion (http://www.drpaulcarter.com/pcasm/ - you dig it out, remember?). I'll rescan all the files in the commpage (http://www.opensourc...k/i386/commpage) to see if i find something out. If i manage to do it fast, i'll take a close look at the files at the x86_64 folder (http://www.opensourc...8/osfmk/x86_64/). I think the crucial files must be inside either one or the other.

 

Notice that the x86_64 doesn't contain the commpage, which is inside i386 instead. It's odd, since there's no 32 bit xnu-1699.26.8 in the real life...

 

 

EDIT: i'm also doing some useful extra reading, check this one out: http://www.macresear...or_extensions_0 - it's quite outdated (mentioning Tiger!), but gave some me some interesting insights.

[theconnactic]

Well, i talked to meklort. He was nice, too. Lot of patience with the loads of inept questions i just threw at him, lol.

 

I'm not going to explain what he told me exactly, because i need to understand it first. He told me, however, that someday he could well do something with AMD and Mountain Lion. But don't count on it for now, guys. That someday could well take three years, according to him.

 

By the way, i was right: 3) We know not.

 

It'll take a lot of effort, reading and learning for me to do the job. Alone, to be candid, it's probably above my capabilities for now. Long term goal, my friends. But i'm still on the front.

[bcobco]

im impressed. seems that this is a really hard task to fork xnu...

 

can you please tell us some conclusions of the interviews with those kernel hackers?

[theconnactic]

Hi, bcobco!

 

Essentially, they both (meklort and nawcom before him) said quite the same thing: it's an awfully tedious task, to find each and any incompatibility between the XNU and AMD CPUs and deal with it. Tedious and time-consuming. As for the ssse3 problem, maybe the instructions aren't really at the files we thought they were at first (this is one of the possibilities i raised before). Anyway, any crucial task performed using ssse3 must be dealth with, rewriting them so they could be performed using supported instructions. We didn't discuss anything about ssse3 emulation.

 

Anyway, it's very clear that is not an easy task for beginners (like me), since it requires advanced knowledge of C and assembly to perform and this knowledge won't be achieved overnight. The positive thing is that, according to meklort, it's possible to be done: there's no dead end here. He warn, though, about the bad quality of anterior patches, which have been applied without much criteria. He suggested that, if we are to do that despite of all difficulties, we try not to simply "hack it for AMD", but to alter the specific Intel-blocks to make the kernel more CPU-agnostic.

 

We talk about debugging too: as we should know, one of our greatest difficulties is to know what actually went wrong when something goes wrong. He told me about VMware (my first hackintosh experience was with a virtual machine; if i just knew... time for experimenting again), but VMware somehow blurs the whole thing. Other debugging possibilities are either using FireWire (that i confess i didn't understand) or adding the C function kprintf to the code. I'll try all but FireWire when it comes to the time.

 

We should (and i will, for sure) keep trying to do that, but it's not reasonable to expect short term results, unless some really capable person start working on it. Anyway, my road map now is to find these Intel specific pieces and start working on it. I'm reserving at least one entire hour a day to play with the kernel, since this is the maximum i can do regularly. Maybe we go slowly, but remember that, in the end, the tortoise defeated the hare.

[bcobco]

so it seems that would be nice to create an open project, an official fork of darwin/xnu (in github or similar), to make the kernel compatible with all x86 architecture, capable to handle incompatible instructions of userland code.

 

 

a huge challenge!

[theconnactic]

Precisely, bcobco! You speak my thoughts out!

 

Pursuing one of meklort's/nawcom's tips, i found this: http://www.hexblog.com/?p=94

 

Lucky me, i still have a Windows 7 64-bit installation in my desktop. Gotta brush the spider webs from it and put a little grease so it runs, lol. An eternity since i opened it last time.

 

EDIT: just to make myself clear, i'm'not looking forward to run OSX on a virtual machine. Virtualization is useful, but it hinder the OS capabilities, and i'd rather run Windows under OSX on a Parallels virtual machine (in fact, that's precisely what i do ). What is implicated here is the use of VMware as a debugging tool.

[bcobco]

Pursuing one of meklort's/nawcom's tips, i found this: http://www.hexblog.com/?p=94

huh, i took a look at that, but that knowledges escapes from my possibilities...

 

im happy with my objetive: since unfortunately linux is actually not suitable for audio production and windows does not convince me, i struggled hard to get an almost working osx system running with an overclocked custom build computer. its pretty stable, and gives very good audio performance for music production. thats why im very grateful with all the people that make this possible.

 

____

edit

it would be useful to collect all links and things collected in this thread, in the first topic. it would be useful for those who want to join this quest and find this thread,

Edited September 27, 2012 by bcobco

[theconnactic]

Take a look at this header: this is cpuid.h, included at the beginning of the commpage.c file. It seems quite obvious to me that it has a crucial role in making our XNU kernel intel-only. Notice the assignment of the much-dreaded ssse3 instructions to the HBit (9), as well as the definitions of various intel CPUs.

 

Notice that it lacks Intel CPUs that are reportedly not capable to run Mountain Lion (perhaps are those definitions called in some kind of CPU check that prevents the kernel from running on unlisted CPUs, nut at the same time there's a definition for a "VMware CPU" and an "unknown CPU").

 

As a side note, the definition of Syscall and Sysret as external features: they're AMD children, but are universally supported as far as i know.

 

(occasional supplemental comments by me at the right side of lines marked with red underlined bold letters).

 

#ifndef _MACHINE_CPUID_H_

#define _MACHINE_CPUID_H_

 

#include

 

#ifdef __APPLE_API_PRIVATE

 

#define CPUID_VID_INTEL "GenuineIntel"

#define CPUID_VID_AMD "AuthenticAMD" (theconnactic: look how funny! a definition for AMD CPUs! Why?)

 

#define CPUID_VMM_ID_VMWARE "VMwareVMware"

 

#define CPUID_STRING_UNKNOWN "Unknown CPU Typ"

 

#define _Bit(n) (1ULL

#define _HBit(n) (1ULL

 

/*

* The CPUID_FEATURE_XXX values define 64-bit values

* returned in %ecx:%edx to a CPUID request with %eax of 1:

*/

#define CPUID_FEATURE_FPU _Bit(0) /* Floating point unit on-chip */

#define CPUID_FEATURE_VME _Bit(1) /* Virtual Mode Extension */

#define CPUID_FEATURE_DE _Bit(2) /* Debugging Extension */

#define CPUID_FEATURE_PSE _Bit(3) /* Page Size Extension */

#define CPUID_FEATURE_TSC _Bit(4) /* Time Stamp Counter */

#define CPUID_FEATURE_MSR _Bit(5) /* Model Specific Registers */

#define CPUID_FEATURE_PAE _Bit(6) /* Physical Address Extension */

#define CPUID_FEATURE_MCE _Bit(7) /* Machine Check Exception */

#define CPUID_FEATURE_CX8 _Bit(8) /* CMPXCHG8B */

#define CPUID_FEATURE_APIC _Bit(9) /* On-chip APIC */

#define CPUID_FEATURE_SEP _Bit(11) /* Fast System Call */

#define CPUID_FEATURE_MTRR _Bit(12) /* Memory Type Range Register */

#define CPUID_FEATURE_PGE _Bit(13) /* Page Global Enable */

#define CPUID_FEATURE_MCA _Bit(14) /* Machine Check Architecture */

#define CPUID_FEATURE_CMOV _Bit(15) /* Conditional Move Instruction */

#define CPUID_FEATURE_PAT _Bit(16) /* Page Attribute Table */

#define CPUID_FEATURE_PSE36 _Bit(17) /* 36-bit Page Size Extension */

#define CPUID_FEATURE_PSN _Bit(18) /* Processor Serial Number */

#define CPUID_FEATURE_CLFSH _Bit(19) /* CLFLUSH Instruction supported */

#define CPUID_FEATURE_DS _Bit(21) /* Debug Store */

#define CPUID_FEATURE_ACPI _Bit(22) /* Thermal monitor and Clock Ctrl */

#define CPUID_FEATURE_MMX _Bit(23) /* MMX supported */

#define CPUID_FEATURE_FXSR _Bit(24) /* Fast floating pt save/restore */

#define CPUID_FEATURE_SSE _Bit(25) /* Streaming SIMD extensions */

#define CPUID_FEATURE_SSE2 _Bit(26) /* Streaming SIMD extensions 2 */

#define CPUID_FEATURE_SS _Bit(27) /* Self-Snoop */

#define CPUID_FEATURE_HTT _Bit(28) /* Hyper-Threading Technology */

#define CPUID_FEATURE_TM _Bit(29) /* Thermal Monitor (TM1) */

#define CPUID_FEATURE_PBE _Bit(31) /* Pend Break Enable */

 

#define CPUID_FEATURE_SSE3 _HBit(0) /* Streaming SIMD extensions 3 */

#define CPUID_FEATURE_PCLMULQDQ _HBit(1) /* PCLMULQDQ instruction */

#define CPUID_FEATURE_DTES64 _HBit(2) /* 64-bit DS layout */

#define CPUID_FEATURE_MONITOR _HBit(3) /* Monitor/mwait */

#define CPUID_FEATURE_DSCPL _HBit(4) /* Debug Store CPL */

#define CPUID_FEATURE_VMX _HBit(5) /* VMX */

#define CPUID_FEATURE_SMX _HBit(6) /* SMX */

#define CPUID_FEATURE_EST _HBit(7) /* Enhanced SpeedsTep (GV3) */

#define CPUID_FEATURE_TM2 _HBit(8) /* Thermal Monitor 2 */

#define CPUID_FEATURE_SSSE3 _HBit(9) /* Supplemental SSE3 instructions */ (theconnactic: aha! here it is. Are this checked for in any routine? Probably an asm one. bcopy?)

#define CPUID_FEATURE_CID _HBit(10) /* L1 Context ID */

#define CPUID_FEATURE_SEGLIM64 _HBit(11) /* 64-bit segment limit checking */

#define CPUID_FEATURE_CX16 _HBit(13) /* CmpXchg16b instruction */

#define CPUID_FEATURE_xTPR _HBit(14) /* Send Task PRiority msgs */

#define CPUID_FEATURE_PDCM _HBit(15) /* Perf/Debug Capability MSR */

 

#define CPUID_FEATURE_PCID _HBit(17) /* ASID-PCID support */

#define CPUID_FEATURE_DCA _HBit(18) /* Direct Cache Access */

#define CPUID_FEATURE_SSE4_1 _HBit(19) /* Streaming SIMD extensions 4.1 */

#define CPUID_FEATURE_SSE4_2 _HBit(20) /* Streaming SIMD extensions 4.2 */

#define CPUID_FEATURE_xAPIC _HBit(21) /* Extended APIC Mode */

#define CPUID_FEATURE_MOVBE _HBit(22) /* MOVBE instruction */

#define CPUID_FEATURE_POPCNT _HBit(23) /* POPCNT instruction */

#define CPUID_FEATURE_TSCTMR _HBit(24) /* TSC deadline timer */

#define CPUID_FEATURE_AES _HBit(25) /* AES instructions */

#define CPUID_FEATURE_XSAVE _HBit(26) /* XSAVE instructions */

#define CPUID_FEATURE_OSXSAVE _HBit(27) /* XGETBV/XSETBV instructions */

#define CPUID_FEATURE_AVX1_0 _HBit(28) /* AVX 1.0 instructions */

#define CPUID_FEATURE_VMM _HBit(31) /* VMM (Hypervisor) present */

#define CPUID_FEATURE_SEGLIM64 _HBit(11) /* 64-bit segment limit checking */

#define CPUID_FEATURE_PCID _HBit(17) /* ASID-PCID support */

#define CPUID_FEATURE_TSCTMR _HBit(24) /* TSC deadline timer */

#define CPUID_FEATURE_AVX1_0 _HBit(28) /* AVX 1.0 instructions */

#define CPUID_FEATURE_F16C _HBit(29) /* Float16 convert instructions */

#define CPUID_FEATURE_RDRAND _HBit(30) /* RDRAND instruction */

 

/*

* Leaf 7, subleaf 0 additional features.

* Bits returned in %ebx to a CPUID request with {%eax,%ecx} of (0x7,0x0}:

*/

#define CPUID_LEAF7_FEATURE_RDWRFSGS _Bit(0) /* FS/GS base read/write */

#define CPUID_LEAF7_FEATURE_SMEP _Bit(7) /* Supervisor Mode Execute Protect */

#define CPUID_LEAF7_FEATURE_ENFSTRG _Bit(9) /* ENhanced Fast STRinG copy */

 

/*

* The CPUID_EXTFEATURE_XXX values define 64-bit values

* returned in %ecx:%edx to a CPUID request with %eax of 0x80000001:

*/

#define CPUID_EXTFEATURE_SYSCALL _Bit(11) /* SYSCALL/sysret */

#define CPUID_EXTFEATURE_XD _Bit(20) /* eXecute Disable */

 

#define CPUID_EXTFEATURE_1GBPAGE _Bit(26) /* 1GB pages */

#define CPUID_EXTFEATURE_RDTSCP _Bit(27) /* RDTSCP */

#define CPUID_EXTFEATURE_EM64T _Bit(29) /* Extended Mem 64 Technology */

 

#define CPUID_EXTFEATURE_LAHF _HBit(0) /* LAFH/SAHF instructions */

 

/*

* The CPUID_EXTFEATURE_XXX values define 64-bit values

* returned in %ecx:%edx to a CPUID request with %eax of 0x80000007:

*/

#define CPUID_EXTFEATURE_TSCI _Bit(8) /* TSC Invariant */

 

#define CPUID_CACHE_SIZE 16 /* Number of descriptor values */

 

#define CPUID_MWAIT_EXTENSION _Bit(0) /* enumeration of WMAIT extensions */

#define CPUID_MWAIT_BREAK _Bit(1) /* interrupts are break events */

 

#define CPUID_MODEL_YONAH 0x0E (theconnactic: here the CPU models. only C2duo and later ones! Are they called back in a CPU check routine?)

#define CPUID_MODEL_MEROM 0x0F

#define CPUID_MODEL_PENRYN 0x17

#define CPUID_MODEL_NEHALEM 0x1A

#define CPUID_MODEL_FIELDS 0x1E /* Lynnfield, Clarksfield, Jasper */

#define CPUID_MODEL_DALES 0x1F /* Havendale, Auburndale */

#define CPUID_MODEL_NEHALEM_EX 0x2E

#define CPUID_MODEL_DALES_32NM 0x25 /* Clarkdale, Arrandale */

#define CPUID_MODEL_WESTMERE 0x2C /* Gulftown, Westmere-EP, Westmere-WS */

#define CPUID_MODEL_WESTMERE_EX 0x2F

#define CPUID_MODEL_SANDYBRIDGE 0x2A

#define CPUID_MODEL_JAKETOWN 0x2D

#define CPUID_MODEL_IVYBRIDGE 0x3A

 

 

#define CPUID_VMM_FAMILY_UNKNOWN 0x0

#define CPUID_VMM_FAMILY_VMWARE 0x1

 

#ifndef ASSEMBLER

#include

#include

#include

#include

 

 

typedef enum { eax, ebx, ecx, edx } cpuid_register_t;

static inline void

cpuid(uint32_t *data)

{

asm("cpuid"

: "=a" (data[eax]),

"=b" (data[ebx]),

"=c" (data[ecx]),

"=d" (data[edx])

: "a" (data[eax]),

"b" (data[ebx]),

"c" (data[ecx]),

"d" (data[edx]));

}

 

static inline void

do_cpuid(uint32_t selector, uint32_t *data)

{

asm("cpuid"

: "=a" (data[0]),

"=b" (data[1]),

"=c" (data[2]),

"=d" (data[3])

: "a"(selector),

"b" (0),

"c" (0),

"d" (0));

}

 

/*

* Cache ID descriptor structure, used to parse CPUID leaf 2.

* Note: not used in kernel.

*/

typedef enum { Lnone, L1I, L1D, L2U, L3U, LCACHE_MAX } cache_type_t ;

typedef struct {

unsigned char value; /* Descriptor value */

cache_type_t type; /* Cache type */

unsigned int size; /* Cache size */

unsigned int linesize; /* Cache line size */

#ifdef KERNEL

const char *description; /* Cache description */

#endif /* KERNEL */

} cpuid_cache_desc_t;

 

#ifdef KERNEL

#define CACHE_DESC(value,type,size,linesize,text) \

{ value, type, size, linesize, text }

#else

#define CACHE_DESC(value,type,size,linesize,text) \

{ value, type, size, linesize }

#endif /* KERNEL */

 

/* Monitor/mwait Leaf: */

typedef struct {

uint32_t linesize_min;

uint32_t linesize_max;

uint32_t extensions;

uint32_t sub_Cstates;

} cpuid_mwait_leaf_t;

 

/* Thermal and Power Management Leaf: */

typedef struct {

boolean_t sensor;

boolean_t dynamic_acceleration;

boolean_t invariant_APIC_timer;

boolean_t core_power_limits;

boolean_t fine_grain_clock_mod;

boolean_t package_thermal_intr;

uint32_t thresholds;

boolean_t ACNT_MCNT;

boolean_t hardware_feedback;

boolean_t energy_policy;

} cpuid_thermal_leaf_t;

 

 

/* XSAVE Feature Leaf: */

typedef struct {

uint32_t extended_state[4]; /* eax .. edx */

} cpuid_xsave_leaf_t;

 

 

/* Architectural Performance Monitoring Leaf: */

typedef struct {

uint8_t version;

uint8_t number;

uint8_t width;

uint8_t events_number;

uint32_t events;

uint8_t fixed_number;

uint8_t fixed_width;

} cpuid_arch_perf_leaf_t;

 

/* Physical CPU info - this is exported out of the kernel (kexts), so be wary of changes */

typedef struct {

char cpuid_vendor[16];

char cpuid_brand_string[48];

const char *cpuid_model_string;

 

cpu_type_t cpuid_type; /* this is *not* a cpu_type_t in our */

uint8_t cpuid_family;

uint8_t cpuid_model;

uint8_t cpuid_extmodel;

uint8_t cpuid_extfamily;

uint8_t cpuid_stepping;

uint64_t cpuid_features;

uint64_t cpuid_extfeatures;

uint32_t cpuid_signature;

uint8_t cpuid_brand;

uint8_t cpuid_processor_flag;

 

uint32_t cache_size[LCACHE_MAX];

uint32_t cache_linesize;

 

uint8_t cache_info[64]; /* list of cache descriptors */

 

uint32_t cpuid_cores_per_package;

uint32_t cpuid_logical_per_package;

uint32_t cache_sharing[LCACHE_MAX];

uint32_t cache_partitions[LCACHE_MAX];

 

cpu_type_t cpuid_cpu_type; /* */

cpu_subtype_t cpuid_cpu_subtype; /* */

 

/* Per-vendor info */

cpuid_mwait_leaf_t cpuid_mwait_leaf;

#define cpuid_mwait_linesize_max cpuid_mwait_leaf.linesize_max

#define cpuid_mwait_linesize_min cpuid_mwait_leaf.linesize_min

#define cpuid_mwait_extensions cpuid_mwait_leaf.extensions

#define cpuid_mwait_sub_Cstates cpuid_mwait_leaf.sub_Cstates

cpuid_thermal_leaf_t cpuid_thermal_leaf;

cpuid_arch_perf_leaf_t cpuid_arch_perf_leaf;

cpuid_xsave_leaf_t cpuid_xsave_leaf;

 

/* Cache details: */

uint32_t cpuid_cache_linesize;

uint32_t cpuid_cache_L2_associativity;

uint32_t cpuid_cache_size;

 

/* Virtual and physical address aize: */

uint32_t cpuid_address_bits_physical;

uint32_t cpuid_address_bits_virtual;

 

uint32_t cpuid_microcode_version;

 

/* Numbers of tlbs per processor [i|d, small|large, level0|level1] */

uint32_t cpuid_tlb[2][2][2];

#define TLB_INST 0

#define TLB_DATA 1

#define TLB_SMALL 0

#define TLB_LARGE 1

uint32_t cpuid_stlb;

 

uint32_t core_count;

uint32_t thread_count;

 

/* Max leaf ids available from CPUID */

uint32_t cpuid_max_basic;

uint32_t cpuid_max_ext;

 

/* Family-specific info links */

uint32_t cpuid_cpufamily;

cpuid_mwait_leaf_t *cpuid_mwait_leafp;

cpuid_thermal_leaf_t *cpuid_thermal_leafp;

cpuid_arch_perf_leaf_t *cpuid_arch_perf_leafp;

cpuid_xsave_leaf_t *cpuid_xsave_leafp;

uint32_t cpuid_leaf7_features;

} i386_cpu_info_t;

 

#ifdef MACH_KERNEL_PRIVATE

typedef struct {

char cpuid_vmm_vendor[16];

uint32_t cpuid_vmm_family;

uint32_t cpuid_vmm_bus_frequency;

uint32_t cpuid_vmm_tsc_frequency;

} i386_vmm_info_t;

#endif

 

#ifdef __cplusplus

extern "C" {

#endif

 

/*

* External declarations

*/

extern cpu_type_t cpuid_cputype(void);

extern cpu_subtype_t cpuid_cpusubtype(void);

extern void cpuid_cpu_display(const char *);

extern void cpuid_feature_display(const char *);

extern void cpuid_extfeature_display(const char *);

extern char * cpuid_get_feature_names(uint64_t, char *, unsigned);

extern char * cpuid_get_extfeature_names(uint64_t, char *, unsigned);

extern char * cpuid_get_leaf7_feature_names(uint64_t, char *, unsigned);

 

extern uint64_t cpuid_features(void);

extern uint64_t cpuid_extfeatures(void);

extern uint64_t cpuid_leaf7_features(void);

extern uint32_t cpuid_family(void);

extern uint32_t cpuid_cpufamily(void);

 

extern void cpuid_get_info(i386_cpu_info_t *info_p);

extern i386_cpu_info_t *cpuid_info(void);

 

extern void cpuid_set_info(void);

 

#ifdef MACH_KERNEL_PRIVATE

extern boolean_t cpuid_vmm_present(void);

extern i386_vmm_info_t *cpuid_vmm_info(void);

extern uint32_t cpuid_vmm_family(void);

#endif

 

#ifdef __cplusplus

}

#endif

 

#endif /* ASSEMBLER */

 

#endif /* __APPLE_API_PRIVATE */

#endif /* _MACHINE_CPUID_H_ */

 

Edited September 27, 2012 by PookyMacMan
Please use [code] [/code] tags around large portions of text like this, it makes navigating the post much easier. Thanks. Edit 2: Yikes, that didn't work well. Just remember the previous as a future principle.

[PookyMacMan]

Hm...could be something important indeed...

 

However, the next step would probably be to define what #define is (pardon the pun). From your knowledge of C, do you have any idea?

[theconnactic]

Hi, Pooky! Thanks for the answer.

 

Perhaps i can "define the define" (forgiven!): it's a C preprocessor directive that tells the preprocessor to replace with a specific value any ocurrence of a given character string. Its syntax is:

 

#define name_of_the_macro body_of_the_macro

 

where the name of the macro is the character string and its body, the value specified.

 

For exemple, #define CPUID_FEATURE_SSSE3 _HBit(9) replaces the character string CPUID_FEATURE_SSSE3 for the value _HBit every time it appears in the file.

 

Or, at least, it's what i learned. There's people more skilled than i am that could wish to correct what i just wrote.

 

P.S.: I'll follow your advice about the code pastes, Pooky: thank you!

[bcobco]

in cpuid.c

/* this function is Intel-specific */
static void
cpuid_set_cache_info( i386_cpu_info_t * info_p )

/* verify we are running on a supported CPU */
if ((strncmp(CPUID_VID_INTEL, info_p->cpuid_vendor,
 min(strlen(CPUID_STRING_UNKNOWN) + 1,
sizeof(info_p->cpuid_vendor)))) ||
(cpuid_set_cpufamily(info_p) == CPUFAMILY_UNKNOWN))
panic("Unsupported CPU");
info_p->cpuid_cpu_type = CPU_TYPE_X86;
info_p->cpuid_cpu_subtype = CPU_SUBTYPE_X86_ARCH1;

 

in fpu.c

/* Advertise SSE support */
if (cpuid_features() & CPUID_FEATURE_FXSR) {
fp_kind = FP_FXSR;
set_cr4(get_cr4() | CR4_OSFXS);
/* And allow SIMD exceptions if present */
if (cpuid_features() & CPUID_FEATURE_SSE) {
set_cr4(get_cr4() | CR4_OSXMM);
}
fp_register_state_size = sizeof(struct x86_fx_thread_state);
} else
panic("fpu is not FP_FXSR");

/*
* SSE arithmetic exception handling code.
* Basically the same as the x87 exception handler with a different subtype
*/
void
fpSSEexterrflt(void)

there are a lot of things to look at. i think that with proper knowledge, it should be possible to make darwin/xnu completely compatible with generic x86 and x86_64 architectures, not only intel specific hardware.

 

 

 

one legal question: can be forked the xnu source; in github, sourceforge, googlecode, or whatever public repository?

the idea is to create a darwin/xnu68 project. actually it seems that there is no project with this objetive (if there is some official project, lets join it.)

Edited September 27, 2012 by bcobco

[Lordadmiral Drake]

I did a cross-file-search for CPUID_FEATURE_SSSE3 and found it in the following locations:

 

..\osfmk\i386\cpuid.c line 888

..\osfmk\i386\cpuid.h line 97

..\osfmk\i386\machine_routines.c line 444

 

Can't say about the files in assembly code

 

In machine_routines.c

 

 

 

/*
 * Are we supporting MMX/SSE/SSE2/SSE3?
 * As distinct from whether the cpu has these capabilities.
 */
os_supports_sse = !!(get_cr4() & CR4_OSXMM);
if (ml_fpu_avx_enabled())
 cpu_infop->vector_unit = 9;
else if ((cpuid_features() & CPUID_FEATURE_SSE4_2) && os_supports_sse)
 cpu_infop->vector_unit = 8;
else if ((cpuid_features() & CPUID_FEATURE_SSE4_1) && os_supports_sse)
 cpu_infop->vector_unit = 7;
else if ((cpuid_features() & CPUID_FEATURE_SSSE3) && os_supports_sse)
 cpu_infop->vector_unit = 6;
else if ((cpuid_features() & CPUID_FEATURE_SSE3) && os_supports_sse)
 cpu_infop->vector_unit = 5;
else if ((cpuid_features() & CPUID_FEATURE_SSE2) && os_supports_sse)
 cpu_infop->vector_unit = 4;
else if ((cpuid_features() & CPUID_FEATURE_SSE) && os_supports_sse)
 cpu_infop->vector_unit = 3;
else if (cpuid_features() & CPUID_FEATURE_MMX)
 cpu_infop->vector_unit = 2;
else
 cpu_infop->vector_unit = 0;

 

 

 

And vector_unit is used here:

 

commpage.c

 

 

 

static void
commpage_init_cpu_capabilities( void )
{
uint32_t bits;
int cpus;
ml_cpu_info_t cpu_info;
bits = 0;
ml_cpu_get_info(&cpu_info);

switch (cpu_info.vector_unit) {
 case 9:
  bits |= kHasAVX1_0;
  /* fall thru */
 case 8:
  bits |= kHasSSE4_2;
  /* fall thru */
 case 7:
  bits |= kHasSSE4_1;
  /* fall thru */
 case 6:
  bits |= kHasSupplementalSSE3;
  /* fall thru */
 case 5:
  bits |= kHasSSE3;
  /* fall thru */
 case 4:
  bits |= kHasSSE2;
  /* fall thru */
 case 3:
  bits |= kHasSSE;
  /* fall thru */
 case 2:
  bits |= kHasMMX;
 default:
  break;
}
switch (cpu_info.cache_line_size) {
 case 128:
  bits |= kCache128;
  break;
 case 64:
  bits |= kCache64;
  break;
 case 32:
  bits |= kCache32;
  break;
 default:
  break;
}

 

 

kern_mib.c

 

 

 

case HW_VECTORUNIT: {
 int vector = cpu_info.vector_unit == 0? 0 : 1;
 return(SYSCTL_RETURN(req, vector));
}

 

And machine_routines.h

 

 

 

/* Struct for ml_cpu_get_info */
struct ml_cpu_info {
uint32_t vector_unit;
uint32_t cache_line_size;
uint32_t l1_icache_size;
uint32_t l1_dcache_size;
uint32_t l2_settings;
uint32_t l2_cache_size;
uint32_t l3_settings;
uint32_t l3_cache_size;
};

[bcobco]

how did you searched across files? thanks in advance

(i would like to search for "amd", "ssse3", "supplementalsse3", ... in all darwin source code)

 

seems that we -amd users- are in cpu_info.vector_unit: case 5: bits |= kHasSSE3;

 

 

 

http://darwinbuild.macosforge.org/

http://code.google.com/p/puredarwin/

i have found this project, but seems outdated (2009...)

if osx runs over darwin, can osx be run over puredarwin or whatever darwin fork ...?

it should be an hypothetical up to date version, that runs on generic x86 processo

Edited September 27, 2012 by bcobco