Fix for XCode 5.1.1 and 6.0.1 Segmentation Fault in VM on AMD systems

[JVene]

INTRO:

 

This thread is for the discussion of using XCode 5.1.1 and 6.0.1 (with future versions to be appended), on OS X 10.9.4 or 10.8.4 running in VM's with (or not) AMD processors, specifically the Phenom II and others without SSSE3.

 

There are a few problems, foremost is:

 

The clang compiler issues a segmentation fault compiling just about anything under those conditions, and the reason is not entirely clear. It could be the CPU, or it could be the virtualization. I started this thread because I found a way to get Xcode working when an answer didn't appear anywhere else on the Internet. It appears the problem is recognized in the discussions I found on the subject, in support forums for other products being compiled in Xcode. Responders typically recognized the segmentation fault by asking if the work was being done on a Hackintosh, after which all support for the problem was withdrawn.

 

However, other issues have appeared (not as catastrophic), and I hope anyone experiencing this, and experimenting on solutions to this, to add to the discussion:

 

In 10.8.4 VM's under AMD processors, the hardware simulator works, even when representing OpenGL ES 2 code with shaders. You can debug, run...exactly as expected.

In 10.9.4, however, there is currently a display bug which obscures visibility of the simulator. This can be seen, also, in the 3D screen saver called "Flurry" - it looks oddly pixellated.

 

XCode 3.2.6 was the last known stable IDE in the product line, to my knowledge. Google reveals YEARS worth of complaints regarding XCode (from 4 up) just quitting on you. I've yet to see if 6.0.1 addresses this issue, but it is widely known on various OS X versions on genuine Apple hardware. About the only hint of a resolution is throwing RAM at the problem. I've not been convinced that works, or that it's even justifiable as a suggested fix (some people get real sensetive if anyone suggests Apple is at fault here....but they are).

 

The rest of this post focuses on the segmentation fault of XCode 5.1.1 and 6.0.1 during compilation.

 

 

EXECUTIVE SUMMARY:

 

The solution, I found, is to obtain a replacement for clang from LLVM.org. They have binaries built for OS X (Darwin) and other operating systems. The OS X build is about 99% the same as the one provided by Apple, but it doesn't crash in a segmentation fault.

 

That said, it should be noted that this is a different clang. You will not notice a difference, as far as I can tell, once it's installed, but Apple might be able to see that the compiler used to generate an App Store product isn't their own compiler (I have no idea if they can or can't). As such, while all functionality you expect of Xcode should return to normal, it is not advisable to plan on submitting a build from this compiler to Apple's App Store. You can use this installation for debugging and development, experimentation and learning, but plan to make any release build for submission on another machine without a substituted clang.

 

 

THE BASIC PLAN:

 

This discussion is not strictly a step by step list, because your details may differ from mine. Instead, I've written a “walk through” which should help you figure out the specifics of your machine. What you're reading are the notes from an engineer's desk (yes, I am actually an engineer...been at this 30+ years, and I might be old enough to be your father   ).

 

That said, there are basically three steps once you have a new clang downloaded or built.

 

First, determine what two directories Apple's clang is looking for.

Second, substitute Apple's clang for LLVM's clang.

Third, create two links which point LLVM's clang to the directories Apple used.

 

After performing these steps, sample code from Apple and new projects created in Xcode compile and function in OS X and iOS as expected.

 

 

QUICK EXAMPLE WITH DANGEROUS ASSUMPTIONS:

 

If you're hoping for a quick list, proceed with caution under the caveat that this makes some assumptions that require substitution for your particular version and installation of Xcode. Assuming Xcode 5.1.1 (6.0.1 is similar), and then assuming these are the two includes Apple's own clang searches:

 

/Applications/Xcode5.1.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1

/Applications/Xcode5.1.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.1/include

 

..as determined with an example compilation using the -v flag..

 

Then:

 

1) Backup the existing clang executable found in:

 

/Applications/Xcode5.1.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang

 

2) Copy (or use ln if you prefer) the LLVM version of clang into that directory, replacing Apple's original (maybe check it's permission includes X).

 

3) Create two symbolic links INSIDE the following parent path:

 

/Applications/Xcode5.1.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain

 

Such that “usr/include/c++/v1” points to “usr/lib/c++/v1 ”

 

Then, a second link such that “usr/lib/clang/3.4/include” points to “usr/lib/clang/5.1/include”

 

To be clear, the “usr” in the above is UNDER XcodeDefault.xctoolchain

 

 

The objective of these two links is to correct the assumption made by LLVM's clang, which is version 3.4. Naturally it looks in “clang/3.4” for headers, but Apple calls their clang “5.1”, so the headers required are not in “clang/3.4”, they're in “clang/5.1”.

 

Similarly, LLVM assumes other important headers are in “user/include/c++/v1”, but Apple put theirs in “usr/lib/c++/v1”.

 

They also use the screwy “bin../lib”, which means “in bin, no I mean in lib” format for the path.

 

All this assuming Xcode 5.1.1

 

Now, if that's all you need, you're done.

 

On the other hand, if you're in need of more information or options, the following is the rest of the story.

 

 

 

BEFORE STEP ONE:

 

Before you can get to step one, you must obtain LLVM from LLVM.org. To determine which version of LLVM to download, review the compiler's output or at the command line give the --version flag. Apple calls their build “Apple LLVM” with their own version number, but at the end of the line they state it's “Based on LLVM 3.4svn”, or similar. That gives you the version you want, which for Xcode 5.1.1 is LLVM 3.4. Xcode 6.0.1 is LLVM 3.5.

 

 

GET LLVM AND CLANG:

 

I chose to build clang on OS X, but I also tested the pre-built binary. Both worked. However, the clang I built supports targets like MIPS which Apple's version doesn't support, and I had the option of building a debug version of clang. Further, I could modify the version output to match Apple's output if desired. I might even take the time to figure out how to change the default directories to match Apple's choices, but I just wanted to get over this (it had been 8 or 9 days).

 

If you choose the pre-built binary, you can skip the following discussion about building clang and proceed to “Substitute the new clang for the old one”.

 

 

BUILDING CLANG:

 

I used SVN to download the clang source. I tried getting the tar packages, but unpacking them leads to confusion. SVN is slower, but if you follow their instructions, it works.

 

I obtained LLVM versions 3.3, 3.4 and 3.5, creating a directory structure like this:

 

/users/jv/llvm/llvm_33

/users/jv/llvm/llvm_34

/users/jv/llvm/llvm_35

 

The rest of this assumes llvm_34, but is the same for the other two.

 

Follow the instructions for obtaining LLVM from svn for your intended version, using this as the top level directory:

 

/users/...../llvm/llvm_34

 

Obviously, assuming LLVM 3.4 here. This means the llvm_34 directory is the root of your copy of the source code. It takes 3 steps to get LLVM, clang and clang-rt.

 

DO NOT bother with the clang extras! (It's a fourth download step). It's not only unnecessary, it doesn't seem to work or build correctly.

 

First and foremost, to build clang you must have a compatible AND FUNCTIONAL compiler. Xcode 5.0.2 is fine, being based on clang and LLVM 3.3 (building LLVM requires clang 3.1 or higher). Make sure that you've selected this version for command line tools for OS X (either in preferences within Xcode, or with the xcode-select command). Verify with a command line like: clang –version, to be sure it's the LLVM 3.3 based compiler currently selected. Otherwise, you could get the pre-built binary of LLVM 3.3 and use that if you don't want to obtain the XCode 5.0.2.

 

I built several versions, which all follow this same pattern. First, cd to /users/.../llvm_34, then mkdir build_release. Next, cd to build_release. That means the PWD is, on my machine,

 

/users/jv/llvm/llvm_34/build_release

 

To run configure, issue ../llvm/configure. That routes the path upward out of build_release, which happens to be llvm_34, then down into llvm (which SVN created under llvm_34) to get to the configure executable.

 

For the release build I used these two options to “configure”

 

--prefix=/users/jv/llvm/llvm_34/build_release/install

--enable-optimized.

 

SPECIFY A PREFIX (the prefix option above). The default is to install into /usr/bin, which isn't desirable for these purposes. You're not going to assume this LLVM as your OS X system compiler outside of Xcode.

 

Now, the LLVM instructions assume you could be on any linux or unix OS (maybe even Windows on cygwin). They insist on using gmake. However, on OS X, “make” is actually gmake. So, when LLVM says to issue the command:

 

gmake

 

You should issue “make” instead.

 

Build time is around 30 to 45 minutes, give or take processor speed and disk speed.

 

It builds fine in 4 Gbytes of RAM with 4 CPU cores using 4 processes, as in:

 

make -j4

 

Yet, if you want to still be able to use your computer (a little) while it's building, maybe leave a core free with somethinking like

 

make -j3

 

or

 

make -j2

 

When it's finished you can safely run “make install” according to the instructions, ASSUMING YOU USED A --prefix OPTION DURING CONFIGURE. The default is to install in /usr/bin, or something like that which is not desirable.

 

 

When that's finished, you have a counterpart to the pre-built binary from LLVM.org, built according to whatever options you select during the configure command. In this example that means the “clang” executable I need for my solution was created in:

 

/Users/jv/llvm/llvm_34/build_release/install/bin

 

It will be over 30 megabytes, maybe even over 100 megabytes (if built in debug).

 

 

 

 

 

 

 

SUBSTITUTE THE NEW CLANG FOR THE OLD ONE:

 

If you downloaded the pre-built binaries for OS X, browse or search for the clang executable..it's about 30+ megabytes. To verify you found it, either explicitly issue the full path to it or cd to the directory where it exists and type:

 

./clang --version

 

It should declare it's LLVM 3.x, not Apple's Clang based on LLVM.

 

Now the objective is to determine what two directories Apple assumes for the includes for the clang compiler they provided. It could be different in future versions, and is slightly different on my machine than yours because I have 3 versions of Xcode side by side.

 

If you built clang, you're probably still “selected” for Xcode 5.0.2. command line tools. You should now select for the 5.1.1 or the 6.0.1 command line tools. You can use “preferences” in Xcode, or you can use the xcode-select command. In either case, verify you're correctly selected with something like:

 

clang --version

 

To be sure the compiler declares itself to be Apple's clang of the version you expect (the one you're FIXING now, the 5.1.1 in this example discussion).

 

We need an example program that's simple, short and likely doesn't crash Apple's compiler (it's not critical that 5.1.1 or 6.0.1 be able to compile, but it shortens the output we're about to read).

 

Something like this would do:

 

int main( int argc, char *argv[] )

{

return 0;

}

 

Yes, no includes, no code....basically nothing.

 

Create such a trivial code wherever you like (some temporary development directory) and issue:

 

g++ -v main.cpp

 

...assuming you named your example code main.cpp.

 

 

The output will show where Apple's clang searches for it's includes in your particular installation of that version of Xcode.

 

Were looking for the section of the output with this content:

 

 

ignoring nonexistent directory "/usr/include/c++/v1"

ignoring nonexistent directory "/usr/local/include"

#include "..." search starts here:

#include <...> search starts here:

/Applications/Xcode5.1.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1

/Applications/Xcode5.1.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.1/include

/usr/include

/System/Library/Frameworks (framework directory)

/Library/Frameworks (framework directory)

End of search list.

 

 

This stuff tells you where Apple's own clang is looking for headers.

 

The problem is that the new clang looks in different directories (Apple had to be unique here). In the above you'll notice the compiler mentions “ignoring non-existent directory”. You may ingore that as well, because you might not be using these standard or local locations for headers.

 

What we need is the material in:

 

/Applications/Xcode5.1.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1

/Applications/Xcode5.1.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.1/include

 

 

The new clang will look in different locations for these two, and get confused. It will issue copious complaints about missing header files, so the objective will be to create symblic links pointing the new clang to these two locations listed above.

 

Note, however, Apple's slightly screwy syntax, “usr/bin/../lib”, which means “in bin, no I mean lib”. This screwy output is probably the result of concatenating two strings in code, when Apple wanted to “get rid” of the “bin” in this passage, but didn't want to modify the code that provided “bin”...so, they used the “up a directory” notion of “../” to do that, instead.

 

So, “usr/bin../lib” resolves to “usr/lib”.

 

RECORD THESE TWO LOCATIONS. You'll need them in notes to refer to from your installation.

 

 

MOVE THE NEW CLANG INTO POSITION:

 

Now, with a backup of clang in place (you do that automatically, right?) - either copy the new clang over Apple's old one, or create a link. Your clang is inside the “Toolchains/XcodeDefault.xctoolchain” directory of your Xcode.app installation, in usr/bin.

 

After you copy or link the new clang into position, repeat the attempt at compiling the example code with:

 

g++ -v main.cpp

 

This will use the new clang instead of the old one, but it will complain about missing headers. The output will contain the locations it tried to search, as in:

 

ignoring nonexistent directory "/usr/include/c++/v1"

ignoring nonexistent directory "/usr/local/include"

#include "..." search starts here:

#include <...> search starts here:

/Applications/Xcode5.1.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1

/Applications/Xcode5.1.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/3.4/include

 

 

Here notice, just after the screwy bin/../lib or bin/../include notation, that clang is looking in different places than Apple's version was. This is where you must create the symbolic links, so this version of clang finds Apple's material where it expects to find them.

 

For example, change to the /usr/lib/clang directory (from the most recent output exampled above):

 

cd /Applications/Xcode5.1.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang

 

ls

 

At this point you'll see Apple has a 5.1 directory here, but the new clang is expecting a 3.4 here. So:

 

mkdir 3.4

cd 3.4

 

The new clang expects a directory include in here, but Apple put it under 5.1, so:

 

ln -s /Applications/Xcode5.1.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/5.1/include include

 

To be clear, this ln command assumes the PWD is lib/clang/3.4 (under the default toolchain directory of XCode.app).

 

Now, I have to remind myself every time I use ln....the last parameter is the link you're creating.

 

The first parameter is the directory the link will point to.

 

Directories must be “soft linked” or “symbolically linked”....hence the -s flag.

 

Follow the same reasoning for the “C++/V1” directory clang expects to find.

 

If all goes well, Xcode will now act exactly as you expect it should.

[ameris_cyning]

Nice post

[Saibalter]

This thread deserves many many many more likes. I made an account just to come and post this lol.

 

I was searching around forever for a solution to why i couldn't compile anything on my hackintosh thru Xcode.... spent maybe 2 or 3 days searching for an answer and found this thread (through another post that OP posted in another thread).

 

Since Xcode is now on version 6.1, the directories are a bit different but what i've found is  

 

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/6.1.0/include Needs to be linked to from 3.6.0/include.

 

At this time, it appears that  no other directory needs modifying, however I haven't fully tested it yet. Will post back with results.

[Saibalter]

Ran into a roadblock. Trying to compile for swift and getting the same Seg Faults.

 

@JVene, how did you determine the location of the clang executable? I can't figure out where the directory of the swift compiler executable is, or if its possible to replace.

[JVene]

Saibalter, I'm so sorry to have been away a while, didn't see your post until today.

 

I have to admit, to find clang I simply searched for clang. Unfortunately I have no idea what they do for swift. Part of the key might be to compile swift code with verbose output and look at the command line generated to see what compile commands they issue to process swift source, then search for that.