Jump to content

Build_Clover.command, another Script to build standard Clover (or customized)

Micky1979

By Micky1979,
in Clover

 Share
2,126 posts in this topic

Recommended Posts

apianti
apianti
Posted
Posted
5 hours ago, vector sigma said:

Hi Sir apianti, I'm sorry but I cannot understand what you mean or referring to AptioFix stories, but me at least, I was only referring on how the script (as per title of the topic) checkout/update AptioFix from github into a svn one and I was wondering if this can be done with edk2 ...and get rid of sourceforge. No some thing else. About the copy of the patches.. how this cannot work? cp in -R mode continue copying even on errors, so that I agree with -f, but errors are printed to the stdout and this is not the case. Those errors only happened to me when patches for edk2 inside Clover aren't good for the local revision of edk2 itself why I showned you this: 


The current local EDK2 revision is the suggested one (26300).

Sir Zenith432 committed new patches compatible with newer r26764 but the guy who showned a fail on the log was using 26300.

So, if you revert edk2, don't forget to revert Clover as well, otherwise errors for sure. You know, simply that. 

EDIT
unless you are talking about reverting edk2 to the same revision but untouched:D. Anyway mentioned cp command is inside clover() func, so to apply patches you have to ensure to update clover with an option of that script. So yes if you cannot update Clover because sf is down, patches cannot be applied but anyway are the old one. Speaking by using this script ^_^

EDIT II

maybe you have to move the cp command outside this function, or eventually have this code in ebuild.sh.

No, I had the patches copied over and had to revert the EDK2 repo to its original state. The revision actually matters very little, not as much as made out to be, sometimes when there are large changes to one of the headers or something there is an issue but most of the time it's fine because it overwrites the entire file, not just patch it. You only need the patches if you are building the firmware, the GUI alone shouldn't need the patches but then you have to specifically build each module. Which I doubt the script does. The script can definitely be changed to use the git repo, it's not my script, and was my initial point, switch to the github repo of EDK2 and you will probably have less issues.

Link to comment
Share on other sites
apianti
apianti
Posted
Posted
27 minutes ago, bronxteck said:

can't the edk repo just be changed in the BuildCloverConfig.txt on line 16

EDK2_REP=svn://svn.code.sf.net/p/edk2/code/trunk/edk2

or is there more to it then just that?

No it's more than that as the commands would need to be switched to use git instead of subversion. It would also require removing any working copy.

Link to comment
Share on other sites
vector sigma
vector sigma
Posted
Posted (edited)
8 hours ago, bronxteck said:

can't the edk repo just be changed in the BuildCloverConfig.txt on line 16

EDK2_REP=svn://svn.code.sf.net/p/edk2/code/trunk/edk2

or is there more to it then just that?

 

7 hours ago, apianti said:

No it's more than that as the commands would need to be switched to use git instead of subversion. It would also require removing any working copy.

I'm confident that this: 

EDK2_REP=https://github.com/tianocore/edk2/trunk

in BuildCloverConfig.txt will work as expected :D. Just need a clean src the first time. Bye bye sourceforge... now we need Clover on github..

Edited by vector sigma
Link to comment
Share on other sites
Philip Petev
Philip Petev
Posted
Posted (edited)
4 minutes ago, vector sigma said:

Hi Sir Philip Petev, of course yes. About the revision,by calling: 


Build_Clover.command --edk2rev 36852

everything went fine, but can we assume that differences is always the same?? 

The only way to be sure is to svn-checkout both the git and the SF repos in different folders and compare them with visual diff tool like Kaleidoscope. If they are identical, you won't see any occurrences when you load them in the app.

Edited by Philip Petev
Link to comment
Share on other sites
vector sigma
vector sigma
Posted
Posted (edited)
10 minutes ago, Philip Petev said:

The only way to be sure is to checkout both the git and the SF repos in different folders and compare them with visual diff tool like Kalleidoscope.

IMHO, is just the revision that comes with sf that is missleading, main repo is on github. I mean that if Slice will decide to use: 

https://github.com/tianocore/edk2/trunk

instead of  

svn://svn.code.sf.net/p/edk2/code/trunk/edk2

then the rivision will be just the latest he wants to be ha ha. He just need to checkout new modules with new url. Is always a svn repo and have to do this only once and syntax He's using will be always the same (i.e. He can continue to use ~/src/edk2/Clover/update.sh)

Edited by vector sigma
spelling
Link to comment
Share on other sites
apianti
apianti
Posted
Posted (edited)

There is no reason to need to continue with any revision for EDK2, it is only used to update the patches, which is why at some point there is a "suggested" revision. However, the git and svn repos are identical, except that the svn repo constantly syncs with the git repo. Clover v2 repo won't be changing git, just the script using git for EDK2, it still would need to use subversion for Clover. And if you do switch to git for the EDK2 repo you'll have to make sure the entire working copy is erased to pull from the git repo or you'll run into issues.

EDIT: Also I switched to the git EDK2 repo a long time ago when the svn repo switched to syncing instead of direct commits. Never had an issue.

Edited by apianti
Link to comment
Share on other sites
vector sigma
vector sigma
Posted
Posted
25 minutes ago, apianti said:

make sure the entire working copy is erased to pull from the git repo or you'll run into issues

yeah, just compare each packages by doing the following (in this case BaseTools): 

cd ~/src/edk2/BaseTools
svn info . | grep '^URL'

then if the output is not "URL: ${EDK2_REP}/BaseTools" you'll have delete it and checkout it again.

 

Link to comment
Share on other sites
Philip Petev
Philip Petev
Posted
Posted (edited)

SF is out for lunch... again.

Edit: guys, take a look at this:

Spoiler

================================================================================
boot6
                             <--------------------------------------------------
TOOLCHAIN_DIR: /Users/philip/src/opt/local
MTOC_PREFIX: /Users/philip/src/opt/local/bin/
NASM_PREFIX: /Users/philip/src/opt/local/bin/
NASM_VER: 2.13.03
Building from: /Users/philip/src/edk2

Running edk2 build for CloverX64 using the command:
build  -D NO_GRUB_DRIVERS_EMBEDDED --conf=/Users/philip/src/edk2/Conf -D USE_LOW_EBDA -p Clover/Clover.dsc  -a X64 -b RELEASE -t XCODE8 -n 9

Build environment: Darwin-16.7.0-x86_64-i386-64bit
Build start time: 22:24:40, Apr.02 2018

WORKSPACE        = /Users/philip/src/edk2
ECP_SOURCE       = /Users/philip/src/edk2/EdkCompatibilityPkg
EDK_SOURCE       = /Users/philip/src/edk2/EdkCompatibilityPkg
EFI_SOURCE       = /Users/philip/src/edk2/EdkCompatibilityPkg
EDK_TOOLS_PATH   = /Users/philip/src/edk2/BaseTools
CONF_PATH        = /Users/philip/src/edk2/Conf


Architecture(s)  = X64
Build target     = RELEASE
Toolchain        = XCODE8

Active Platform          = /Users/philip/src/edk2/Clover/Clover.dsc
Flash Image Definition   = /Users/philip/src/edk2/Clover/Clover.fdf

Processing meta-data ..

build.py...
/Users/philip/src/edk2/MdePkg/Library/BaseLib/BaseLib.inf(452): error 000E: File/directory not found in workspace
    /Users/philip/src/edk2/MdePkg/Library/BaseLib/X64/WriteTr.nasm


- Failed -
Build end time: 22:24:44, Apr.02 2018
Build total time: 00:00:04


o_Ops, ./ebuild.sh exited with error(s), aborting..

That's clean source download (no ~/src/edk2) from the GH repo.

Edited by Philip Petev
  • Haha 1
Link to comment
Share on other sites
apianti
apianti
Posted
Posted
11 hours ago, vector sigma said:

yeah, just compare each packages by doing the following (in this case BaseTools): 


cd ~/src/edk2/BaseTools
svn info . | grep '^URL'

then if the output is not "URL: ${EDK2_REP}/BaseTools" you'll have delete it and checkout it again.

 

No, if you switch to a git repo from using subversion repo, you need to erase the entire working copy. They are completely different tools and hold the commits in different ways, they cannot be checked out into the same directory. In fact git usually refuses to checkout into a non empty directory without flags and also if I recall correctly also complains that its already being tracked by subversion. You can however convert it to git but that won't help you because you want to have the remote git repo's commits/history, not create a new one.

7 hours ago, Philip Petev said:

SF is out for lunch... again.

Edit: guys, take a look at this:

  Hide contents

================================================================================
boot6
                             <--------------------------------------------------
TOOLCHAIN_DIR: /Users/philip/src/opt/local
MTOC_PREFIX: /Users/philip/src/opt/local/bin/
NASM_PREFIX: /Users/philip/src/opt/local/bin/
NASM_VER: 2.13.03
Building from: /Users/philip/src/edk2

Running edk2 build for CloverX64 using the command:
build  -D NO_GRUB_DRIVERS_EMBEDDED --conf=/Users/philip/src/edk2/Conf -D USE_LOW_EBDA -p Clover/Clover.dsc  -a X64 -b RELEASE -t XCODE8 -n 9

Build environment: Darwin-16.7.0-x86_64-i386-64bit
Build start time: 22:24:40, Apr.02 2018

WORKSPACE        = /Users/philip/src/edk2
ECP_SOURCE       = /Users/philip/src/edk2/EdkCompatibilityPkg
EDK_SOURCE       = /Users/philip/src/edk2/EdkCompatibilityPkg
EFI_SOURCE       = /Users/philip/src/edk2/EdkCompatibilityPkg
EDK_TOOLS_PATH   = /Users/philip/src/edk2/BaseTools
CONF_PATH        = /Users/philip/src/edk2/Conf


Architecture(s)  = X64
Build target     = RELEASE
Toolchain        = XCODE8

Active Platform          = /Users/philip/src/edk2/Clover/Clover.dsc
Flash Image Definition   = /Users/philip/src/edk2/Clover/Clover.fdf

Processing meta-data ..

build.py...
/Users/philip/src/edk2/MdePkg/Library/BaseLib/BaseLib.inf(452): error 000E: File/directory not found in workspace
    /Users/philip/src/edk2/MdePkg/Library/BaseLib/X64/WriteTr.nasm


- Failed -
Build end time: 22:24:44, Apr.02 2018
Build total time: 00:00:04


o_Ops, ./ebuild.sh exited with error(s), aborting..

 

I have been using SF all day and haven't had one problem, lol.

7 hours ago, Philip Petev said:

That's clean source download (no ~/src/edk2) from the GH repo.

What do you mean? I use the GH git repo for EDK2, and the SF svn repo for Clover, in multiple OSes and it works perfectly fine. Just built with newest commits from each repo, works great...?

Link to comment
Share on other sites
vector sigma
vector sigma
Posted
Posted (edited)
8 hours ago, apianti said:

No, if you switch to a git repo from using subversion repo, you need to erase the entire working copy. They are completely different tools and hold the commits in different ways, they cannot be checked out into the same directory. In fact git usually refuses to checkout into a non empty directory without flags and also if I recall correctly also complains that its already being tracked by subversion. You can however convert it to git but that won't help you because you want to have the remote git repo's commits/history, not create a new one.

We are saying a similar thing of Just checking if the repo is not the same, then you can erase it or not. But me I'm talking by using "svn co" on github not git, otherwise you can ignore me :)

16 hours ago, Philip Petev said:

That's clean source download (no ~/src/edk2) from the GH repo.

Wrong revision of edk2 (probably is old), take latest or use r36852 as I already tried. EDIT: just for testing purpose, assuming you're using github.

Edited by vector sigma
  • Like 1
Link to comment
Share on other sites
Philip Petev
Philip Petev
Posted
Posted (edited)

My point was subversion counts the revisions in the git repo the wrong way. Trying to pull out the latest preferred revision (r26764), subversion downloaded unusable copy (probably another revision), so changing the edk2 url with the GH repo doesn't work, at least no as it should be.

And the latest revision works only because nothing has been changed much (or at all) in the files from the Patches_for_edk2 folder.

Edited by Philip Petev
  • Thanks 1
Link to comment
Share on other sites
vector sigma
vector sigma
Posted
Posted (edited)
58 minutes ago, Philip Petev said:

subversion downloaded unusable copy (probably another revision)

there's probably a diferences of 10086: 

svn info https://github.com/tianocore/edk2/trunk
Path: trunk
URL: https://github.com/tianocore/edk2/trunk
Relative URL: ^/trunk
Repository Root: https://github.com/tianocore/edk2
Repository UUID: 72af8a33-8d04-ae46-851a-ce57b7db7bff
Revision: 36858
Node Kind: directory
Last Changed Author: carsey..jaben
Last Changed Rev: 36858
Last Changed Date: 2018-04-03 11:39:34 +0200 (Mar, 03 Apr 2018)

vs sourceforge: 

svn info svn://svn.code.sf.net/p/edk2/code/trunk/edk2
Path: edk2
URL: svn://svn.code.sf.net/p/edk2/code/trunk/edk2
Relative URL: ^/trunk/edk2
Repository Root: svn://svn.code.sf.net/p/edk2/code
Repository UUID: 6f19259b-4bc3-4df7-8a09-765794883524
Revision: 26772
Node Kind: directory
Last Changed Author: edk2buildsystem
Last Changed Rev: 26768
Last Changed Date: 2018-04-02 11:05:26 +0200 (Lun, 02 Apr 2018)

so by checking out r26764 from github (svn) is like having downloading r16678 from sf :D

EDIT

..but sf is actually one day older. Maybe We can checkout a single BaseTools from both url (r36858 and r26772) and see if the commit message is the same with: 

cd .../BaseTools
svn log --revision xxxxx

EDIT II

Anyway git is best, but the big problem is if Slice wants that??

Edited by vector sigma
typo
Link to comment
Share on other sites
vector sigma
vector sigma
Posted
Posted
On 1/4/2018 at 8:28 PM, apianti said:

So yes if you are getting those errors it's because the patches were not copied

Got it. I'm playing with Clover to make a customized iso with nvme+apfs drivers for virtualbox. I've entered the called "Developer mode", and this really doesn't copy patches. Don't know if this is a bug or is wanted, unsure.

Good night

Link to comment
Share on other sites
vector sigma
vector sigma
Posted
Posted
10 minutes ago, Slice said:

git is for beta versions, svn is for release version.

As well git has no native enumeration. We know version number exclusively due to svn.

cd into a git repository if you have one and type this: 

printf "%s\0\n" $(git cat-file commit HEAD | wc -c)

you'll get the commit number. If any day you need to switch to github and mantain your revision just trick ebuild.sh to add (or subtract) any number to match the latest revision you had on sf.

Link to comment
Share on other sites
apianti
apianti
Posted
Posted (edited)

You can determine the number of commits in a git branch though with 

git rev-list --count HEAD --

That should be the same as the revisions in svn. If not, that still gives an identifying number of commits that you can track.

Edited by apianti
Link to comment
Share on other sites
vector sigma
vector sigma
Posted
Posted (edited)

The following script can checkout (to your desktop) edkII using git (only packages needed by Clover). Can revert or update at the specified sha1 as you wish: 

#!/bin/bash

EDK2_REP='https://github.com/tianocore/edk2.git'
DIR_MAIN="${HOME}"/Desktop/EDK2_TEST
SHA1='0c9f2cb10b7ddec56a3440e77219fd3ab1725e5c' # empty if you want the latest commit
GIT_DEPTH=300 # this speed up the download but too old revisions of edk2 will not be in the history (so you cannot revert before)
BUILD_BASETOOLS=NO

# this script checkout only packages needed by Clover (as per instructions).
# if you need more add it to the below 'edk2array'
edk2array=(
	MdePkg
	MdeModulePkg
	CryptoPkg
	DuetPkg
	EdkCompatibilityPkg
	IntelFrameworkModulePkg
	IntelFrameworkPkg
	OvmfPkg
	OptionRomPkg
	PcAtChipsetPkg
	ShellPkg
	UefiCpuPkg
	BaseTools
	)

updateOrReverEDK2Packages() {
  cd "${DIR_MAIN}"
  local cmmt=$(git rev-parse HEAD | tr -d '\r')
  if [[ "$cmmt" == "$SHA1" ]]; then
    echo "edk2 is already at the specified commit!"
  else
    echo "Resetting edk2 at $SHA1 because is at $cmmt"
    git reset --hard $SHA1
    BUILD_BASETOOLS=YES
  fi
}

checkoutEDK2Packages() {
  local checkout=""
  mkdir -p "${DIR_MAIN}"
  for pkg in "${edk2array[@]}"
  do
    checkout+="${pkg}/*\n"
  done

  git init "${DIR_MAIN}"
  cd "${DIR_MAIN}"
  git remote add origin ${EDK2_REP}
  git config core.sparsecheckout true
  echo -e "${checkout}" >> .git/info/sparse-checkout
  git pull --depth $GIT_DEPTH origin master

  if [[ -n "$SHA1" ]]; then
    updateOrReverEDK2Packages
  fi
  BUILD_BASETOOLS=YES
}

edk2() {
  mkdir -p "${DIR_MAIN}"
  if [[ -d "${DIR_MAIN}/.git" ]]; then
    updateOrReverEDK2Packages
  else
    checkoutEDK2Packages
  fi
}

clear
edk2

if [[ "$BUILD_BASETOOLS" == YES ]]; then
  echo "We should clean/build BaseTools, checkout/update Clover and apply patches.."
fi

what about?

Edited by vector sigma
typo
Link to comment
Share on other sites
 Share
Go to topic listing
×
×
  • Create New...