Bootloader Build_Clover.command, another Script to build standard Clover (or customized)

[Micky1979]

Well,.

 

1) setted up a new var with default and common macros enabled:

DEFAULT_MACROS="-D NO_GRUB_DRIVERS_EMBEDDED -D CHECK_FLAGS"

affect only  standards build with no custom arguments (macros).

 

2) Now the script rebuild BaseTools automatically. Better to say that clean it if a new revision comes out, so that ebuild should rebuild it. By default is:

AUTO_REBUILD_BASETOOL="YES"

otherwise can be:

AUTO_REBUILD_BASETOOL="NO"

3) nasm is added only if not found in $PATH or is older than 2.12 (so if you already have a recent version, that will be used). Actually still download, if needed, the 2.12.02.rc1with Zenith patch, please advice!

4) gettext added only if not found in $PATH, but no check on the revision (how can affect Clover even if old??)

5) STLVNB is now STLVNUB   

Build_Clover_v3.2_Beta.command.zip

[Philip Petev]

Nice script indeed, but I think you should write the requirements for using that script in post 1: minimum OSX/Xcode/other stuff version etc.

[cecekpawon]

I didn't remember why we have two files

"$CLOVERROOT"/rEFIt_UEFI/Version.h

"$CLOVERROOT"/Version.h

 

>>> in WIndows always read the 1st

[Micky1979]

Nice script indeed, but I think you should write the requirements for using that script in post 1: minimum OSX/Xcode/other stuff version etc.

Thanks. Initially this Topic was ripped from "Clover general discussion", some posts moved here. I'll follow your advice, btw the script it-self provide you some info selecting option 9:

9) info and limitations about this script

and will show you this:

===============================================================================
INFO
                          <----------------------------------------------------
This script was originally created to be run in newer OSes like El Capitan
+ using Xcode 7.3 +, but should works fine using gcc 4,9 (GCC49)
in older ones. Also gcc 5,3 or 6,1 are good.

Don't be surprised if this does not work in Snow Leopard (because can't).
In Lion can require you to uncomment/comment some line in clover() function.

Using old Xcode like v7.2 and older this script automatically disable
LTO as suggested by devs, and the result will be a binaries increased in size.
Off course that is automatic only for standard compilation, but consider to
switch back to gcc 4,9 (GCC49).

By default the script no longer build the iso image but if you need it,
just edit BUILD_ISO="NO" to BUILD_ISO="YES" at the "Preferences" section.

You can build both 32/64 bit, only 32bit or only 64 bit packages!

Enabling macros to build Clover EFI apply only to boot3/boot7,
ia32 buid can fail defining additional macros. In this case don't panic:
use only 64-bit target.. or rebuild it with supported functionalities!

Warning using the "R" mode of this script to create the src folder
outside the Home folder:
Blanck spaces in the path are not allowed because it will fail!
                          <----------------------------------------------------

Also is not tested in linux (read "TODO" inside the script),..(no time is summer ...  ). But since I have in mind to produce some graphical interface apps for Ubuntu (and any other compatible with GnuStep) maybe a Clover installer can become possible... just wait to re-install Ubuntu and start working on it 

 

EDIT

notice only now a typo ("ia32 buid" , if you guys can see more of it, please tell me)

[Slice]

Probably wouldn't hurt to rebuild BaseTools IF it gets updated as well.

 

Not sure....

I think we should use good working old tools instead of new updated not working tools.

Someone tested and said: OK. we can use tools rev 224000 as well. And we move to this version.

See, Clover always live with this tactic keeping Patches_to_EDK2.. Are use sure EDK is good enough to not apply those ancient patches?

 

 

Hi Guys.

Slice, STLVNUB, the script actually apply a patch to the nasm source made by Zenith432 and I don't know if in newer versions that is solved (easy to see I guess).... that's why install nasm and disasm if not installed inside the src bin path. 

 

Remind me, please, what are the patches by Zenith432. Are they obsoleted by version 2.12?

I am using native nasm-2.12 and it works fine as you may see with my Clover compilations.

[Micky1979]

Remind me, please, what are the patches by Zenith432. Are they obsoleted by version 2.12?

I am using native nasm-2.12 and it works fine as you may see with my Clover compilations.

 

Hi Slice, some history about bugs in relocations for macho-32 and macho-64 objects :

 

http://www.insanelymac.com/forum/topic/306156-clover-bugissue-report-and-patch/page-30?p=2233533&do=findComment&comment=2233533

http://www.insanelymac.com/forum/topic/306156-clover-bugissue-report-and-patch/page-34?p=2235188&do=findComment&comment=2235188

 

Zenith432 pull request to nasm devel:

https://sourceforge.net/p/nasm/mailman/message/35051007/

 

Yeha! .. just downloaded 2.12.02rc9 and infact was accepted:

/* If the relocation is internal add to the current section
	       offset. Otherwise the only value we need is the symbol
	       offset which we already have. The linker takes care
	       of the rest of the address.  */
	    if (!r->ext) {
		/* generate final address by section address and offset */
		nasm_assert(r->snum <= seg_nsects);
		l += sectstab[r->snum]->addr;
		if (r->pcrel)
		    l -= s->addr;
	    } else if (r->pcrel && r->type == GENERIC_RELOC_VANILLA) {
		l -= s->addr;
	    }

	    /* write new offset back */
	    p = blk.buf;
	    WRITEDLONG(p, l);
	    saa_fwrite(s->data, r->addr, blk.buf, len);

So today all nasm with that patch applied are ok, but I guess 2.12 is still buggie. And Yes nasm 2.12.02 is ok and is the same you link me.

The bad is that I have now to check the third component of the array but this can be also a string "rc9" instead of a number.

 

good ones are:

major version >2

or

major version >=2, minor version >= 12 and the revision should not be a string and anyway >= 2

 

The script actually check  nasm this way (ATM only the "major" and "minor" versions, must be changed..):

 

1)check if is present in $PATH, but since you can have more than one (an array), the working one is the one at index 0 (this will be used).

 

2)If you have one that is good (but will not be used because is not the first), the script copy the good version in ~/src/opt/local/bin (because we ensured also in ebuild.sh that this is the first place in $PATH, so working nasm will be there). And no download are needed.

 

3)Otherwise install it if not present or bad.

 

example of "which -a nasm" after adding multiple paths:

===============================================================================
nasm check:
                          <----------------------------------------------------
found nasm v2.12.02rc1 at /Users/Micky1979/src/opt/local/bin
found nasm v0.98.40 at /Applications/Xcode.app/Contents/Developer/usr/bin
found nasm v0.98.40 at /usr/bin 

off course the script check the version of each ones

 

 

BTW why not build it, configured/compiled on the running machine... than?

 

Not sure....

I think we should use good working old tools instead of new updated not working tools.

Someone tested and said: OK. we can use tools rev 224000 as well. And we move to this version.

See, Clover always live with this tactic keeping Patches_to_EDK2.. Are use sure EDK is good enough to not apply those ancient patches?

Default to no update? Consider that I can force rebuild BaseTools also if too old or "less-than". mmmh This need a rule (you are the Chief  ) because a newer user will always get the latest revision, I mean you should create a txt file in the source with that revision inside to be aquired?

[Fljagd]

I have a compilation error

[Slice]

Guys,

simple users should not compile Clover. There is ready-to-use binaries available for download.

I need build script as fastest as possible to compile Clover after changing one line in sources, in one file and recompile only this file.

I don't want to update EDK every time.

I don't want to recompile BaseTools every time nor every week not every month.

I don't want to recompile gcc if not found, I am using xcode5 toolset.

I don't want to recompile gettext and nasm if not found in my home folder, they are installed in the computer for other reasons.

Moreover I don't need to recompile boot file. Old one works fine.

ebuild.sh is good enough for me, except the last note. May be we can make a flag to not recompile CloverEFI.

 

Custom clover compilation should exists for developers who are able to understand what is wrong and why there are compilations errors.

[Riley Freeman]

I got that error yesterday with 3719. If you scroll up a bit, it's looking for nasm in /opt rather than opt inside the src folder.

[Fljagd]

 

 

Custom clover compilation should exists for developers who are able to understand what is wrong and why there are compilations errors.

message received

I got that error yesterday with 3719. If you scroll up a bit, it's looking for nasm in /opt rather than opt inside the src folder.

Thanks

[Micky1979]

Guys,

simple users should not compile Clover. There is ready-to-use binaries available for download.

I need build script as fastest as possible to compile Clover after changing one line in sources, in one file and recompile only this file.

I don't want to update EDK every time.

I don't want to recompile BaseTools every time nor every week not every month.

I don't want to recompile gcc if not found, I am using xcode5 toolset.

I don't want to recompile gettext and nasm if not found in my home folder, they are installed in the computer for other reasons.

Moreover I don't need to recompile boot file. Old one works fine.

ebuild.sh is good enough for me, except the last note. May be we can make a flag to not recompile CloverEFI.

 

Custom clover compilation should exists for developers who are able to understand what is wrong and why there are compilations errors.

Thanks, more clear. But we actually don't know some of your source(s), I can see some time commits "sync with edk rxxxx", so why not add to the source a config file (a txt its enough) where we can see stuff used by you? That will conform compilations also for other devs. What I mean is that:

#!/bin/bash

SliceConfig="$HOME/src/edk2/Clover/official.txt"
cd $HOME/src/edk2
echo "edk2 $(svnversion -n | tr -d [:alpha:])" > $SliceConfig

cd $HOME/src/edk2/BaseTools
echo "BaseTools $(svnversion -n | tr -d [:alpha:])" >> $SliceConfig

echo "nasm 2.12.02" >> $SliceConfig

and everyone will see something like this:

edk2 22400
BaseTools 22400
nasm 2.12.02

and all the compilations around the world will know how to do to conform to you! Than also a simple wrapper for ebuild.sh will know if update or leave all as is!

nasm will not be a problem if equal to your, otherwise to conform to official Clover must be inside the src folder as official tool to build it.

gettext... mmmh .. who cares? does nothing to build binaries.

 

Why not?

I got that error yesterday with 3719. If you scroll up a bit, it's looking for nasm in /opt rather than opt inside the src folder.

 

Yep, is defined by tools-def, but that is not really a problem

message received

I think Slice does not want to fight towards problems related to compilation errors, since conditions are not the same ones that he uses.

[Fljagd]

I think Slice does not want to fight towards problems related to compilation errors, since conditions are not the same ones that he uses.

His work is substantial, I can understand

His patient can sometimes be sorely ordeal

[Slice]

Thanks, more clear. But we actually don't know some of your source(s), I can see some time commits "sync with edk rxxxx", so why not add to the source a config file (a txt its enough) where we can see stuff used by you? That will conform compilations also for other devs. What I mean is that:

#!/bin/bash

SliceConfig="$HOME/src/edk2/Clover/official.txt"
cd $HOME/src/edk2
echo "edk2 $(svnversion -n | tr -d [:alpha:])" > $SliceConfig

cd $HOME/src/edk2/BaseTools
echo "BaseTools $(svnversion -n | tr -d [:alpha:])" >> $SliceConfig

echo "nasm 2.12.02" >> $SliceConfig

and everyone will see something like this:

edk2 22400
BaseTools 22400
nasm 2.12.02

and all the compilations around the world will know how to do to conform to you! Than also a simple wrapper for ebuild.sh will know if update or leave all as is!

nasm will not be a problem if equal to your, otherwise to conform to official Clover must be inside the src folder as official tool to build it.

gettext... mmmh .. who cares? does nothing to build binaries.

 

 

But I have no such information because I didn't keep such file.

We can see a log in sf.net https://sourceforge.net/p/cloverefiboot/code/3720/log/?path=

[r3714] by  slice2009
sync with edk2-22435

And so on.

But I don't know about other versions because only latest version exists for me.

[Slice]

 

Moreover I don't need to recompile boot file. Old one works fine.

ebuild.sh is good enough for me, except the last note. May be we can make a flag to not recompile CloverEFI.

 

 

I did this!

 

./ebuild.sh --module=rEFIt_UEFI/refit.inf

[CC] Net
[CC] AutoGen
[SLINK] CLOVERX64
[DLINK1] CLOVERX64
[MTOC] CLOVERX64
[GENFW] CLOVERX64

- Done -
Build end time: 20:24:43, Aug.23 2016
Build total time: 00:01:00

iMac:Clover slice$

See total time!

[Micky1979]

I did this!

 

./ebuild.sh --module=rEFIt_UEFI/refit.inf

[CC] Net
[CC] AutoGen
[SLINK] CLOVERX64
[DLINK1] CLOVERX64
[MTOC] CLOVERX64
[GENFW] CLOVERX64

- Done -
Build end time: 20:24:43, Aug.23 2016
Build total time: 00:01:00

iMac:Clover slice$

See total time!

1 second?       

EDIT, maybe. one minute, but is very fast!

Slice, I always + convinced that you should also rip out all the themes from the source, leave embedded and your theme. We can add it later, or inform buildpkg.sh to look also on another directory (~/src/AdditionalThemes ? and add also that if exist). Downloading Clover takes so much for non-code stuff.

[Micky1979]

But I have no such information because I didn't keep such file.

We can see a log in sf.net https://sourceforge.net/p/cloverefiboot/code/3720/log/?path=

[r3714] by  slice2009
sync with edk2-22435

And so on.

But I don't know about other versions because only latest version exists for me.

I promise that is the last time I ask  . Care to add that info manually (or otherwise) to Clover folder when you sync with edk2? ..so your "always latest" will be informative for all others. Consider that this way, in the future, we can compile old revisions compatible with "Patches_for_EDK2". A case? boot7-MCP79 no longer compilable (if I understand you well). plz plz

[Slice]

1 second?       

EDIT, maybe. one minute, but is very fast!

 

It is very vast for my computer taking into account this is full compilation. So why I asked for optimisation of compilation process.

 

 

I promise that is the last time I ask  . Care to add that info manually (or otherwise) to Clover folder when you sync with edk2? ..so your "always latest" will be informative for all others. Consider that this way, in the future, we can compile old revisions compatible with "Patches_for_EDK2". A case? boot7-MCP79 no longer compilable (if I understand you well). plz plz

 

You can recommend EDK2-22435 for all users that are not able to catch compilation errors. Developers may test newer EDK2 revision for his own responsibility.

 

boot7-MCP79 is a version compatible with nforce chipset, it has revision 2976. Revisions 3000+ somehow not working.

I have no a computer with such chipset and can't find a reason for this bug. This is definitely a bug hope will be resolved.

[Philip Petev]

Several potential problems:

1. I've noticed all repo addresses  are using the svn:// protocol. That however may lead to repo access error (happens to me from time to time with fully operational internet connection, maybe Sourceforge server issues), leading to repo access errors. There should be an option the repos to be accessed via the http:// protocol as well in case the svn:// protocol has failed.

2. Something is effectively replacing the path to nasm from ENV(HOME)/src/opt/local/bin/nasm to /opt/local/bin/nasm (where there is nothing to be found) and the build process fails.

 

..............................

*_GCC49_*_ASL_OUTFLAGS                 = DEF(IASL_OUTFLAGS)
#*_GCC49_*_NASM_PATH                    = ENV(HOME)/src/opt/local/bin/nasm
*_GCC49_*_NASM_PATH                    = /opt/local/bin/nasm

...............................

 

That can be seen all over the file. It just comments the path out. Not sure what causes it.

Edit: Oh, this is so by default, just checked the Clover source (r3720) directly from the repo. So, the Clover build script is suppose to change that, but it doesn't.

[Slice]

I think it can be ENV(NASM_PREFIX)/nasm

if this variable will be set by build script.

 

If not set then simple "nasm" is OK for me.

[AsusFreak]

This worked fine on 3718. But with the 3720 it fails...

build.py...
 : error 7000: Failed to execute command
	make tbuild [/Users/ken/src/edk2/Build/Clover/RELEASE_XCODE5/X64/MdePkg/Library/BaseLib/BaseLib]


build.py...
 : error F002: Failed to build module
	/Users/ken/src/edk2/MdePkg/Library/BaseLib/BaseLib.inf [X64, XCODE5, RELEASE]

- Failed -
Build end time: 03:58:29, Aug.24 2016
Build total time: 00:00:04

logout
Saving session...
...copying shared history...
...saving history...truncating history files...
...completed.

[Process completed]

Seems like every few Clover updates screws the script...

[Slice]

sudo cp ~/src/opt/local/bin/nasm  /opt/local/bin/

[Philip Petev]

Yeah.... symlink will do too, but that's only a workaround.

 

Edit: tried to set the NASM path to ENV(NASM_PREFIX)/nasm in tools_def.txt manually, but it still looks for it in /opt/local/bin/nasm. WTF!!!

 

Edit2: the problem is not EDK-related, reverted it to the recommended revision 22435 and got the same errors. I smell hardcoded path somewhere...

[Slice]

If the path is /opt/local/bin/nasm then it is written somewhere

[Micky1979]

Just install nasm to /opt/local/bin solve this "issue" (not really a problem), but I don't want anymore touch anything inside the source.

Sorry no time today to work on, but I promis tonight will work on it (and you all can participate if you want).

Already did something to download a specific revision of edk2 in one of my test scripts.

Also I want this Slice-compatible ... something in mind for this, and something for ebuild.sh about not building boot files again and again. 

 

PS my build time is 42 seconds (rebuilding boot files), 1 minute and 50 seconds to build also the full package (again rebuilding boot files).... wow

[Riley Freeman]

If the path is /opt/local/bin/nasm then it is written somewhere

 

https://sourceforge.net/p/cloverefiboot/code/3719/