Jump to content

Build_Clover.command, another Script to build standard Clover (or customized)


Micky1979
2,126 posts in this topic

Recommended Posts

Edit: tried to set the NASM path to ENV(NASM_PREFIX)/nasm in tools_def.txt manually, but it still looks for it in /opt/local/bin/nasm. WTF!!!

 

Edit2: the problem is not EDK-related, reverted it to the recommended revision 22435 and got the same errors. I smell hardcoded path somewhere...

I meant ENV(NASM_PREFIX)/nasm instead of /opt/local/bin/nasm
Link to comment
Share on other sites

I meant ENV(NASM_PREFIX)/nasm instead of /opt/local/bin/nasm

 

Yeah, yeah, figured that out already. My problem was I had to run ebuild.sh clean after those changes (commenting out the line with /opt/local/bin/nasm and then uncommenting  the above line). Now clover can be build correctly, but you two (you and Micky1979) really should decide where the nasm should be installed and change Clover or the script accordingly. Right now, they are sort of incompatible.

Link to comment
Share on other sites

Help me, please, in writing such bash script

if [file exists "/opt/local/bin/nasm"] then

   NASM_PREFIX="/opt/local/bin"

else if [file exists "(HOME)/src//opt/local/bin"] then

  NASM_PREFIX="(HOME)/src//opt/local/bin"

else

  NASM_PREFIX=""

fi

 

And then check for version

$ nasm -v
NASM version 2.11.06 compiled on Dec  4 2014
 
echo "bad nasm version, please install new one"
Link to comment
Share on other sites

Try this l0rd

if [[ -f "/opt/local/bin/nasm" ]]; then
  NASM_PREFIX="/opt/local/bin/nasm"
elif [[ -f "(HOME)/src//opt/local/bin" ]]; then
  NASM_PREFIX="(HOME)/src//opt/local/bin"
else
  NASM_PREFIX=""
fi

#echo $NASM_PREFIX

#NASM_VER=`nasm -v | awk '/version/ {print $3}'`
NASM_VER=`nasm -v | sed -nE 's/^.*version.([0-9\.]+).*$/\1/p'`

#echo $NASM_VER
  • Like 1
Link to comment
Share on other sites

take a look on my latest beta script some posts ago... it looks for each nasm version already in $PATH and perform a check and decide if good or not. Maybe something can be reused..

 

EDIT:


restoreIFS() {

    IFS=$' \t\n';

}

needNASM() {

    restoreIFS

    local nasmPath=""

    local nasmArray=( $(which -a nasm) )

    local needInstall=1

    local good=""

 

    if [ ${#nasmArray[@]} -ge "1" ]; then

 

        for i in "${nasmArray[@]}"

        do

            echo "found nasm v$(${i} -v | grep 'NASM version' | awk '{print $3}') at $(dirname ${i})"

        done

 

        # we have a good nasm?

        for i in "${nasmArray[@]}"

        do

            if isNASMGood "${i}"; then

                good="${i}"

                break

            fi

        done

 

        if [[ -x "${good}" ]] ; then

             # only nasm at index 0 is used!

            if [[ "${good}" == "${nasmArray[0]}" ]]; then

                echo "nasm is ok.."

            else

                echo "this one is good:"

                echo "${good}"

                echo "..but will not be used.."

                cp -R "${good}" "${NASM_PREFIX}"/

                echo "${good} copied to ${NASM_PREFIX}/!"

            fi

        else

            # no nasm versions suitable for Clover

            echo "nasm found, but is not good to build Clover.."

            needInstall=0

        fi

    else

        needInstall=0

        echo "nasm not found.."

    fi

    return $needInstall

}

 

isNASMGood() {

    restoreIFS

 

    IFS='.';

    local array=($( "${1}" -v | grep 'NASM version' | awk '{print $3}') )

 

    case "${#array[@]}" in

    "2" | "3")

        if [ "${array[0]}" -ge "3" ]; then

            return 0;

        fi

        if [ "${array[0]}" -eq "2" ] && [ "${array[1]}" -ge "12" ]; then

            return 0;

        fi

    ;;

    *)

        echo "Unknown nasm version format.."

    ;;

    esac

 

    return 1

}

  • Like 1
Link to comment
Share on other sites

Slice, take a look on this.

Usage:
(-nb = don't rebuild boot files)

./ebuild.sh -x64 -nb -t XCODE5
./ebuild.sh -ia32 -nb -t XCODE5

result:

build started at:
Thu Aug 25 00:43:00 CEST 2016
finished at
Thu Aug 25 00:43:27 CEST 2016

27 seconds for both architectures (12 for x64 + 11 for ia32. 4 seconds copying every new efi files to the Clover package directories ). Check if is what you need.

ebuild.sh.zip

Link to comment
Share on other sites

Hi Slice what is the result for you of:

echo $USER

on your build machine??

Sergeys-iMac:Clover slice$ echo $USER
slice
Sergeys-iMac:Clover slice$ 

@Slice: why don't you just change all paths to NASM in Patches_for_EDK2/Conf/tools_def.txt to ENV(NASM_PREFIX)/nasm, so every build script (incl. ebuild.sh) can define for itself which nasm instance to use via that variable?

Yes, I assumed this after accepting new ebuild.sh.

NASM_PREFIX is missing forward slash

e.g

NASM_PREFIX=~/src/opt/local/bin/

 

in tools_def.txt ( unmodified except for make GCC make path )

*_*_*_NASM_PATH                = ENV(NASM_PREFIX)nasm

No, slash should be out of prefix

*_*_*_NASM_PATH                = ENV(NASM_PREFIX)/nasm

take a look on my latest beta script some posts ago... it looks for each nasm version already in $PATH and perform a check and decide if good or not. Maybe something can be reused..

 

EDIT:


restoreIFS() {

    IFS=$' \t\n';

}

needNASM() {

    restoreIFS

    local nasmPath=""

    local nasmArray=( $(which -a nasm) )

    local needInstall=1

    local good=""

 

    if [ ${#nasmArray[@]} -ge "1" ]; then

 

        for i in "${nasmArray[@]}"

        do

            echo "found nasm v$(${i} -v | grep 'NASM version' | awk '{print $3}') at $(dirname ${i})"

        done

 

        # we have a good nasm?

        for i in "${nasmArray[@]}"

        do

            if isNASMGood "${i}"; then

                good="${i}"

                break

            fi

        done

 

        if [[ -x "${good}" ]] ; then

             # only nasm at index 0 is used!

            if [[ "${good}" == "${nasmArray[0]}" ]]; then

                echo "nasm is ok.."

            else

                echo "this one is good:"

                echo "${good}"

                echo "..but will not be used.."

                cp -R "${good}" "${NASM_PREFIX}"/

                echo "${good} copied to ${NASM_PREFIX}/!"

            fi

        else

            # no nasm versions suitable for Clover

            echo "nasm found, but is not good to build Clover.."

            needInstall=0

        fi

    else

        needInstall=0

        echo "nasm not found.."

    fi

    return $needInstall

}

 

isNASMGood() {

    restoreIFS

 

    IFS='.';

    local array=($( "${1}" -v | grep 'NASM version' | awk '{print $3}') )

 

    case "${#array[@]}" in

    "2" | "3")

        if [ "${array[0]}" -ge "3" ]; then

            return 0;

        fi

        if [ "${array[0]}" -eq "2" ] && [ "${array[1]}" -ge "12" ]; then

            return 0;

        fi

    ;;

    *)

        echo "Unknown nasm version format.."

    ;;

    esac

 

    return 1

}

Where to apply this?

Link to comment
Share on other sites

take a look on my latest beta script some posts ago... it looks for each nasm version already in $PATH and perform a check and decide if good or not. Maybe something can be reused..

 

EDIT:

restoreIFS() {

    IFS=$' \t\n';

}

needNASM() {

    restoreIFS

    local nasmPath=""

    local nasmArray=( $(which -a nasm) )

    local needInstall=1

    local good=""

 

    if [ ${#nasmArray[@]} -ge "1" ]; then

 

        for i in "${nasmArray[@]}"

        do

            echo "found nasm v$(${i} -v | grep 'NASM version' | awk '{print $3}') at $(dirname ${i})"

        done

 

        # we have a good nasm?

        for i in "${nasmArray[@]}"

        do

            if isNASMGood "${i}"; then

                good="${i}"

                break

            fi

        done

 

        if [[ -x "${good}" ]] ; then

             # only nasm at index 0 is used!

            if [[ "${good}" == "${nasmArray[0]}" ]]; then

                echo "nasm is ok.."

            else

                echo "this one is good:"

                echo "${good}"

                echo "..but will not be used.."

                cp -R "${good}" "${NASM_PREFIX}"/

                echo "${good} copied to ${NASM_PREFIX}/!"

            fi

        else

            # no nasm versions suitable for Clover

            echo "nasm found, but is not good to build Clover.."

            needInstall=0

        fi

    else

        needInstall=0

        echo "nasm not found.."

    fi

    return $needInstall

}

 

isNASMGood() {

    restoreIFS

 

    IFS='.';

    local array=($( "${1}" -v | grep 'NASM version' | awk '{print $3}') )

 

    case "${#array[@]}" in

    "2" | "3")

        if [ "${array[0]}" -ge "3" ]; then

            return 0;

        fi

        if [ "${array[0]}" -eq "2" ] && [ "${array[1]}" -ge "12" ]; then

            return 0;

        fi

    ;;

    *)

        echo "Unknown nasm version format.."

    ;;

    esac

 

    return 1

}

 

All that would be unnecessary, if the Clover patches are changed according to my proposal (post 112). You have already that variable defined in your current script version, all it takes is to use it.

Link to comment
Share on other sites

In buildnasn.sh since Clover already have this script. Need adjustment because was for my script and someting like copy operation is not needed. You can use that this way:

 

if ! needNASM; then

exit 0

fi

 

. . otherwise install nsam

philip, we need to ensure is the correct version....

Link to comment
Share on other sites

Sergeys-iMac:Clover slice$ ./ebuild.sh -nb
TOOLCHAIN_DIR: /Users/slice/src/edk2/Clover/../../opt/local
found nasm v2.12.01 at /Users/slice/src/edk2/Clover/../../opt/local/bin
found nasm v2.12.02 at /opt/local/bin
found nasm v2.12.01 at /usr/gcc-4.8/bin
found nasm v2.12.02 at /opt/local/bin
found nasm v0.98.40 at /usr/bin
nasm is ok..
Sergeys-iMac:Clover slice$ 

and finish?!

Link to comment
Share on other sites

In buildnasn.sh since Clover already have this script. Need adjustment because was for my script and someting like copy operation is not needed. You can use that this way:

 

if ! needNASM; then

exit 0

fi

 

. . otherwise install nsam

philip, we need to ensure is the correct version....

Well, it will be the correct version, doesn't your script always download and install the correct version (if not found any other)?

Link to comment
Share on other sites

@Slice, the script looks fine, but has one potential problem: the NASM_PREFIX variable should be definitely defined (non-empty) before the execution of ebuild.sh (it doesn't define it), otherwise that part may fail.

And it happen.

 

Guys, I know nothing about scripts, I don't know how to do this correctly.

Link to comment
Share on other sites

Yeah.... symlink will do too, but that's only a workaround.

 

Edit: tried to set the NASM path to ENV(NASM_PREFIX)/nasm in tools_def.txt manually, but it still looks for it in /opt/local/bin/nasm. WTF!!!

 

Edit2: the problem is not EDK-related, reverted it to the recommended revision 22435 and got the same errors. I smell hardcoded path somewhere...

If you need custom, always edit/compare both "/edk2/Clover/Patches_for_EDK2/Conf/tools_def.txt" and "/edk2/Conf/tools_def.txt". Sometimes that tools_def got tampered by other wrapper / maybe you. Or use this params to specify other conf path. Or do export '$CONF_PATH' & '$NASM_PREFIX' vars for build.

 

Link to comment
Share on other sites

Slice, take a look on this.

Usage:

(-nb = don't rebuild boot files)

./ebuild.sh -x64 -nb -t XCODE5
./ebuild.sh -ia32 -nb -t XCODE5

result:

build started at:
Thu Aug 25 00:43:00 CEST 2016
finished at
Thu Aug 25 00:43:27 CEST 2016

27 seconds for both architectures (12 for x64 + 11 for ia32. 4 seconds copying every new efi files to the Clover package directories ). Check if is what you need.

Anyway the boot file is created and compressing

- Done -
Build end time: 12:53:25, Aug.25 2016
Build total time: 00:02:17

Compressing DUETEFIMainFv.FV ...
Encoding
Compressing DxeCore.efi ...
Encoding
Compressing DxeIpl.efi ...
Encoding
Generate Loader Image ...

Link to comment
Share on other sites

 

Anyway the boot file is created and compressing

- Done -
Build end time: 12:53:25, Aug.25 2016
Build total time: 00:02:17

Compressing DUETEFIMainFv.FV ...
Encoding
Compressing DxeCore.efi ...
Encoding
Compressing DxeIpl.efi ...
Encoding
Generate Loader Image ...

 

We can do like that also at the beginning of the function:

    if (( $NOBOOTFILES == 0 )); then
      echo Compressing DUETEFIMainFv.FV ...
      "$BASETOOLS_DIR"/LzmaCompress -e -o "${BUILD_DIR}/FV/DUETEFIMAINFV${TARGETARCH}.z" "${BUILD_DIR}/FV/DUETEFIMAINFV${TARGETARCH}.Fv"

      echo Compressing DxeCore.efi ...
      "$BASETOOLS_DIR"/LzmaCompress -e -o "${BUILD_DIR}/FV/DxeMain${TARGETARCH}.z" "$BUILD_DIR_ARCH/DxeCore.efi"

      echo Compressing DxeIpl.efi ...
      "$BASETOOLS_DIR"/LzmaCompress -e -o "${BUILD_DIR}/FV/DxeIpl${TARGETARCH}.z" "$BUILD_DIR_ARCH/DxeIpl.efi"

      echo "Generate Loader Image ..."
    fi

and to the end of the "MainPostBuildScript" when it calls:

DESTDIR="$CLOVER_PKG_DIR"/BootSectors make -C $BOOTHFS

make should find all as before and rebuild only changed stuff. Otherwise the MakeFile is buggie

Sergeys-iMac:Clover slice$ ./ebuild.sh -nb
TOOLCHAIN_DIR: /Users/slice/src/edk2/Clover/../../opt/local
found nasm v2.12.01 at /Users/slice/src/edk2/Clover/../../opt/local/bin
found nasm v2.12.02 at /opt/local/bin
found nasm v2.12.01 at /usr/gcc-4.8/bin
found nasm v2.12.02 at /opt/local/bin
found nasm v0.98.40 at /usr/bin
nasm is ok..
Sergeys-iMac:Clover slice$ 

and finish?!

 

Actually my statement check only array[0] and array[1] of 2.12.02 so it says that is ok, but in reality you have a bad nasm at:

found nasm v2.12.01 at /Users/slice/src/edk2/Clover/../../opt/local/bin

that is the first, and this will not be used only if tools-def decide otherwise. "/Users/slice/src/opt/local/bin" is the good place to avoid any problems since added to $PATH by ebuild.sh before any other path. Also calling buildnasm.sh users cannot claim an unwanted installation since "src/opt/local/bin" is part of Clover..... is the best place IMHO, just looks here  "et voilà" because already added at beginning of $PATH, no chances to fail and will be a easy rule for all. Also all the scripts in Clover source already seem to stay with that.

Sorry to say that, but now I'm sure you used nasm v2.12.01 until commit of r3718... before change the nasm path in tools-def.txt :blush:

 

EDIT I'm at the job and cannot do anything other than speak until 6:00 PM

Link to comment
Share on other sites

This becomes too complicated. I think it would be better nasm to be present only in TOOLCHAIN_DIR and exactly that version to be used and if that version is not proper, the proper version to be downloaded and installed in TOOLCHAIN_DIR.

Link to comment
Share on other sites

I just don't want to double tools in all users homes when I have good one in common folder /opt/local/bin.

There is one problem that the /opt folder has other permission and can't be modified from the script. 


NASM_PREFIX is missing forward slash

e.g

NASM_PREFIX=~/src/opt/local/bin/

 

in tools_def.txt ( unmodified except for make GCC make path )

*_*_*_NASM_PATH                = ENV(NASM_PREFIX)nasm

Sorry, you are right. Prefix must contain / else empty prefix causes error like "/nasm"


Good way


Sergeys-iMac:Clover slice$ ./ebuild.sh --module=rEFIt_UEFI/refit.inf
TOOLCHAIN_DIR: /Users/slice/src/edk2/Clover/../../opt/local
NASM_PREFIX: /opt/local/bin/
NASM_VER: 2.12.02
Initializing workspace

...
Building ... /Users/slice/src/edk2/Clover/Library/OpensslLib/OpensslLibNull.inf [X64]
Building ... /Users/slice/src/edk2/MdeModulePkg/Library/DxeNetLib/DxeNetLib.inf [X64]
[CC] main
[CC] menu
[CC] DataHubCpu
[SLINK] CLOVERX64
[DLINK1] CLOVERX64
[MTOC] CLOVERX64
[GENFW] CLOVERX64

- Done -
Build end time: 15:03:10, Aug.25 2016
Build total time: 00:00:23

Sergeys-iMac:Clover slice$ 

Link to comment
Share on other sites

Hi all,

 

Slice ebuild.sh is almost ok but (HOME) does nothing (should be ${HOME} or $HOME), this is ok here (also corrected the check for nasm):

ebuild.sh.zip

 

 

This is the new script v3.2 rc1, and now revert edk2 to the r22435 (EDK2_REV="22435"), auto rebuild BaseTools if different after the update.

Nasm is 2.12.02 (downloaded but not compiled), and can be in /opt/local/bin or inside ~/src/opt/local/bin

A check if repos are online befor sync:

FAST_UPDATE="YES" # or FAST_UPDATE="NO" to be faster

disabled by default.

 

If the user is "slice", no downloads of any kind, no updates of any kind, no nasm/gettext/gcc (or other) are installed or touched...it simply cd to Clover folder and the menù for Slice is simple:

Please enter your choice: 
1) check revision
2) build with ./ebuild.sh -nb
3) build with ./ebuild.sh --module=rEFIt_UEFI/refit.inf
4) build all (boot3, 6 and 7)
5) build all with FORCEREBUILD (boot3, 6 and 7)
6) build pkg
7) build iso
8) build pkg+iso
#? 

also never exit (always call this menù after each operations.. to be fast). if you like and need adjustment.. tell me.

relative scripting:

            "build with ./ebuild.sh -nb")
                printHeader 'ebuild.sh -nb'
                cd "${DIR_MAIN}"/edk2/Clover
                ./ebuild.sh -nb
                break
            ;;
            "build with ./ebuild.sh --module=rEFIt_UEFI/refit.inf")
                cd "${DIR_MAIN}"/edk2/Clover
                printHeader 'ebuild.sh --module=rEFIt_UEFI/refit.inf'
                ./ebuild.sh --module=rEFIt_UEFI/refit.inf
                break
            ;;
            "build all (boot3, 6 and 7)")
                cd "${DIR_MAIN}"/edk2/Clover
                printHeader 'boot6'
                ./ebuild.sh -x64 -D NO_GRUB_DRIVERS_EMBEDDED -D CHECK_FLAGS -t XCODE5
                printHeader 'boot7'
                ./ebuild.sh -mc --no-usb -D NO_GRUB_DRIVERS_EMBEDDED -D CHECK_FLAGS -t XCODE5
                printHeader 'boot3'
                ./ebuild.sh -ia32 -D NO_GRUB_DRIVERS_EMBEDDED -D CHECK_FLAGS -t XCODE5
                break
            ;;
            "build all with FORCEREBUILD (boot3, 6 and 7)")
                cd "${DIR_MAIN}"/edk2/Clover
                printHeader 'boot6'
                ./ebuild.sh -fr -x64 -D NO_GRUB_DRIVERS_EMBEDDED -D CHECK_FLAGS -t XCODE5
                printHeader 'boot7'
                ./ebuild.sh -fr -mc --no-usb -D NO_GRUB_DRIVERS_EMBEDDED -D CHECK_FLAGS -t XCODE5
                printHeader 'boot3'
                ./ebuild.sh -fr -ia32 -D NO_GRUB_DRIVERS_EMBEDDED -D CHECK_FLAGS -t XCODE5
                break
            ;;
            "build pkg")
                cd "${DIR_MAIN}"/edk2/Clover
                printHeader 'make pkg'
                make pkg
                break
            ;;
            "build iso")
                cd "${DIR_MAIN}"/edk2/Clover
                printHeader 'make iso'
                make iso
                break
            ;;
            "build pkg+iso")
                cd "${DIR_MAIN}"/edk2/Clover
                printHeader 'make pkg + make iso'
                make pkg
                make iso
                break
            ;;

For all others it looks like before:

Please enter your choice: 
1) check revision
2) update Clover only (no building)
3) update & build Clover
4) run my script on the source
5) build existing revision (no update, standard build)
6) build existing revision with FORCEREBUILD (no update, standard build)
7) build existing revision with custom macros enabled
8) build Clover from scratch (delete /Users/Micky1979/src)
9) info and limitations about this script
#? 

if you change nasm path, please run option 6 once, i.e build existing revision with FORCEREBUILD (no update, standard build).

The new script to be working need the above ebuild.sh, so replace it after the update. Hope Slice will accept it.

Edited by Micky1979
attachment removed, look at the Download section
  • Like 2
Link to comment
Share on other sites

New script available here: http://www.insanelymac.com/forum/files/file/589-build-clovercommand/

(also uncommented "edk2" function that was disabled by mistake here due to personal tests made before upload there. now is ok)

 

EDIT 

updated also the following

By Micky1979 based on Slice, Zenith432, STLVNUB, JrCs, cecekpawon, cvad,
Rehabman, philip_petev

# Big thanks to the following testers:
# droples, Riley Freeman, pico joe, fantomas1, Fljagd, calibre, Mork vom Ork
# and all others (I'll be happy to increase this list)

  • Like 2
Link to comment
Share on other sites

×
×
  • Create New...