Clover Problems and Solutions

[Slice]

I reverted UDK2018 and patches at 4522.

Now it works as intended as 

I just had more advanced solution with UDK2018 + patches_to_EDK2. And it worked.

OK, now this way will be common exactly by instructions.

Rev 4527.

[Slice]

I want to exclude fdisk440.

What can I do in this place?

partitionactive=$( "$fdisk440" -d ${bootrdisk} | grep -n "*" | awk -F: '{print $1}')
if [[ -n "$partitionactive" ]]; then
    echo "Current Active Partition: ${partitionactive}" >> "$install_log"
else
    echo "No Active Partition" >> "$install_log"
fi
echo "" >> "$install_log"

 

[vector sigma]

partitionactive=$( fdisk -d ${bootrdisk} | grep -n "*" | awk -F: '{print $1}')

1 hour ago, Slice said:

I want to exclude fdisk440.

What can I do in this place?

partitionactive=$( "$fdisk440" -d ${bootrdisk} | grep -n "*" | awk -F: '{print $1}')
if [[ -n "$partitionactive" ]]; then
    echo "Current Active Partition: ${partitionactive}" >> "$install_log"
else
    echo "No Active Partition" >> "$install_log"
fi
echo "" >> "$install_log"

 

 

[apianti]

Can't you just use the regular fdisk there? That's only reading the entries right? I thought the only difference between the two was the writing only the last 440 bytes when updating the sector, but that can be achieved with dd.

 

EDIT: Oh, lol. @Poco, Just got a weird bug where @vector sigma's post didn't appears until after I posted.

Edited June 13, 2018 by apianti

[Slice]

I want to propose exclude MTOC binary from the repository because ./buildmtoc.sh is good enough.

[vit9696]

@Zenith432, is there still any benefit in using XCODE8 over XCODE5? From what I know XCODE8 does not support -flto and produces bigger binaries. XCODE5 is meant to work with any Xcode from 5 and onwards, and I am not particularly sure why we need to maintain an extra toolchain which has various compatibility issues. Could we drop it without losing any vital features of the buildsystem?

[Zenith432]

8 minutes ago, vit9696 said:

@Zenith432, is there still any benefit in using XCODE8 over XCODE5? From what I know XCODE8 does not support -flto and produces bigger binaries. XCODE5 is meant to work with any Xcode from 5 and onwards, and I am not particularly sure why we need to maintain an extra toolchain which has various compatibility issues. Could we drop it without losing any vital features of the buildsystem?

XCODE5 and XCODE8 both support LTO, and LTO expands the binaries on both toolchains, so LTO is off.

I explained not long ago the difference.  You don't have to use XCODE8 if you don't want to and it has no compatibility issues.

[Zenith432]

@Slice

As you may remember. Thunk16.S compiled with clang in XCODE5 toolchain does not work (causes exception when run), which is why there's a patch for BaseLib.inf.

 

I have patch that fixes Thunk16.S

Spoiler

diff --git a/MdePkg/Library/BaseLib/X64/Thunk16.S b/MdePkg/Library/BaseLib/X64/Thunk16.S
--- a/MdePkg/Library/BaseLib/X64/Thunk16.S
+++ b/MdePkg/Library/BaseLib/X64/Thunk16.S
@@ -131,8 +131,8 @@ L_SavedCr4: .space      4
     #
     # rdi in the instruction below is indeed bx in 16-bit code
     #
-    .byte 0x66,0x2e                     # 2eh is "cs:" segment override
-    lgdt    (SavedGdt - L_Base)(%rdi)
+    .byte 0x66,0x67,0x2e                     # 2eh is "cs:" segment override
+    lgdt    (SavedGdt - L_Base)(%rbx)
     .byte 0x66
     movl    $0xc0000080,%ecx
     rdmsr
@@ -187,8 +187,8 @@ L_Base1:
     pushq   %rax
     lret                                # execution begins at next instruction
 L_RealMode: 
-    .byte 0x66,0x2e                     # CS and operand size override
-    lidt    (_16Idtr - L_Base1)(%rsi)
+    .byte 0x66,0x67,0x2e                     # CS and operand size override
+    lidt    (_16Idtr - L_Base1)(%rbp)
     .byte 0x66,0x61                     # popad
     .byte 0x1f                          # pop ds
     .byte 0x7                           # pop es
@@ -199,6 +199,8 @@ L_RealMode:
     .byte 0x66                          # make the following retf 32-bit
     lret                                # transfer control to user code
 
+.p2align 3
+
 .set  CODE16,  ASM_PFX(_16Code) - .
 .set  DATA16,  ASM_PFX(_16Data) - .
 .set  DATA32,  ASM_PFX(_32Data) - .

 

 

The reason for the crash is that two instructions lgdt and lidt make clang output 32-bit displacement in 16-bit mode, but the address-size override prefix (0x67) is not there, so it crashes.  I think Thunk16.S worked in older versions of clang that output 8-bit displacement.  Nowadays clang uses 32-bit displacement for subtraction of two addresses that cross a non-local symbol. This is to support subsections_via_symbols - however the 32-bit displacement is output even if subsections_via_symbols is turned off.

 

I tested that this with this patch doesn't crash building boot7+Clover GUI in XCODE5.

 

I'm not sure whether to check this in.  Thunk16.nasm works, and it's a replacement patch for patch to BaseLib.inf.  So either patches are good.

 

[Slice]

Yes, I remember. Thunk16.nasm is good enough.

But compiling *.S by Clang has an advantage. Those objects will be optimised by clang linker for unused codes while it can't optimise nasm binaries.

 

Thunk16 used in legacy Clover in BiosVideo and BiosBlockIo.

Legacy Clover can be tested in QEMU that I usually do.

[Zenith432]

12 minutes ago, Slice said:

But compiling *.S by Clang has an advantage. Those objects will be optimised by clang linker for unused codes while it can't optimise nasm binaries.

Yes, but this needs .S file to have ASM_FUNCTION_REMOVE_IF_UNREFERENCED to enable it (= .subsections_via_symbols of clang).

 

I searched all EDK2 source and it's only used on some Arm and Aarch64 .S files.  Never Intel CPU.

 

Most assembly source consists of just one function, or code that depends on stuff all over it, so nothing can be discarded.

So this feature may be important hypothetically - but not many actual uses.

 

For compiler it's useful - because compiler generates all functions and data items in source code with different symbols so they can be dead-stripped separately.

[Slice]

How do you think, can we exclude " -mno-sse -mno-mmx" from compilation rule? It presents in EDK2 because they wrote his software for all processors even for Pentium4.

But Clover is intended for macOS where SSE3 at least presents.

EDITED. O, I see, it is not in XCODE5 and 8!

[vit9696]

@Zenith432, I do see the following post in regards to Xcode toolchains:

And the only thing I can grasp from here is that you try to use Microsoft ABI linkage for all the functions instead of just EFIAPI ones meant to be used for external communication, and which only matter. Microsoft ABI calling convention reserves an extra shadow area and uses less registers for argument passing, so is slower by design than SYSV ABI. Given that prototypes of the functions meant to be used for external communication are marked EFIAPI, any calling convention misuse with XCODE5 properly results in a corresponding message, and I do not see why XCODE8 is necessary any longer.

 

As for -flto, it does not work for XCODE8 for Slice. And last time I tried XCODE8, it failed to compile 32-bit Clover and failed to properly handle -arch flags.

 

I may not see a full picture, but to my eyes XCODE8 feels like a legacy hack that gives us no benefit. While XCODE5 is maintained by Apple and edk2 teams, XCODE8 is only ours, and it brings no functional benefit but maintanance burden.

[Zenith432]

1 hour ago, Slice said:

How do you think, can we exclude " -mno-sse -mno-mmx" from compilation rule? It presents in EDK2 because they wrote his software for all processors even for Pentium4.

But Clover is intended for macOS where SSE3 at least presents.

EDITED. O, I see, it is not in XCODE5 and 8!

It's only in LLVM and XCLANG toolchains in Clover tools_def.txt.  These toolchains don't exist in tools_def.template anymore - they have CLANG38 which has this.  CLANG38 is for use with Linux/ELF LLVM.

[Slice]

TODO.

EDID section in config.plist assumes only monitor in the system. How to inject second EDID for second monitor?

How to inject properties for two monitors?

How to tunes connectors in a framebuffer for known monitors?

I need ideas how to do these.

[Needy]

22 hours ago, Slice said:

TODO.

EDID section in config.plist assumes only monitor in the system. How to inject second EDID for second monitor?

How to inject properties for two monitors?

How to tunes connectors in a framebuffer for known monitors?

I need ideas how to do these.

 

You could implement a hierarchical settings option for the EDID. Instead of having an EDID section, have a section that for example contains the id or any information that can uniquely identify a monitor. Clover would check for matching information with detected monitors and apply desired settings, such as EDID from sub-menus. Is quite efficient as well, no unnecessary information will be processed unless a match is found.

 

Would this work? 

 

Not sure about the frame buffer, but I believe it can be added to the same settings option mentioned above.

Edited June 17, 2018 by Needy

[Slice]

buildmtoc.sh is no working 10.13.5, Xcode 9.4.1

dyld: Library not loaded: /Users/slice/src/opt/local/lib/libisl.dyld: Library not loaded: /Usersdyld: Library not loaded: /Users10.dylib
  Referenced from: /Volumes/MacHD/Users/slice/src/opt/l/slice/src/opt/local/lib/libisl.dyld: Library not loaded: /Usersocal/bin/../libexec/gcc/x86_64-apple-darwin16.5.0/4.9.3/cc1
  Re/slice/src/opt/local/lib/libisl.10.dylib
  Referenced from: /Volumes/MacHD/Users/slice/src/opt/l10.dylib
  Referenced from: /Volumes/MacHD/Users/slice/src/opt/lason: Incompatible library versiocal/bin/../libexec/gcc/x86_64-aocal/bin/../libexec/gcc/x86_64-aon: cc1 requires version 13.0.0 or later, but libisl.10.dylib prpple-darwin16.5.0/4.9.3/cc1
  Reason: Incompatible library versiovides version 12.0.0
on: cc1 requires version 13.0.0 or later, but libisl.10.dylib pr/slice/src/opt/local/lib/libisl.10.dylib
  Referenced from: /Volovides version 12.0.0
umes/MacHD/Users/slice/src/opt/local/bin/../libexec/gcc/x86_64-apple-darwin16.5.0/4.9.3/cc1
  Reason: Incompatible library version: cc1 requires version 13.0.0 or later, but libisl.10.dylib provides version 12.0.0
pple-darwin16.5.0/4.9.3/cc1
  Reason: Incompatible library version: cc1 requires version 13.0.0 or later, but libisl.10.dylib provides version 12.0.0

 

[Sherlocks]

now, clover has problems in edid part

1. not working custom edid value in 10.6/10.7.

-injected edid in ioreg. but there is custom edid from darwin dump.

2. not consider 256 edid.

 

and

i think we need to find user who has multiple monitor with igpu and egpu. then get log and ioreg. check it

TODO.

EDID section in config.plist assumes only monitor in the system. How to inject second EDID for second monitor?

How to inject properties for two monitors?

How to tunes connectors in a framebuffer for known monitors?

I need ideas how to do these.

 

나의 LG-F800S 의 Tapatalk에서 보냄

 

 

 

[Zenith432]

1 hour ago, Slice said:

buildmtoc.sh is no working 10.13.5, Xcode 9.4.1

It works for me.  mtoc should be built with xcode build tools.  It looks from your errors that it's trying to use some old version of gcc 4.9.3 instead.  I don't know how this happens.  It's something about your PATH and the development tools you have in your PATH.

[Slice]

47 minutes ago, Zenith432 said:

It works for me.  mtoc should be built with xcode build tools.  It looks from your errors that it's trying to use some old version of gcc 4.9.3 instead.  I don't know how this happens.  It's something about your PATH and the development tools you have in your PATH.

I think some caches.

Or the script can't be used inside ebuild.sh script.

[Zenith432]

2 hours ago, Slice said:

I think some caches.

Or the script can't be used inside ebuild.sh script.

r4557

[vector sigma]

@Slice, and all, I want to propose some changes to ebuild.sh and in buildpkg.sh

 

ebuild.sh

I'm not a big supporter of pre built binaries inside my source, so I modified it a bit:

1. no download by default, '--no-ext' sobstituted with '--ext-pre' to enable it.
2. Added '--ext-co' to fresh clone & build latest AptioFixPkg/ApfsSupportPkg (and dependecies from the development branch as requested). Since I didn't like to to put external stuff inside the edk/udk folder these Pkg are placed inside a directory called 'EXT_PACKAGES' created at the same level of the edk2/udk folder .. so not inside it. ebuild.sh take the step to export this additional path in PACKAGES_PATH.
3. Added ''--ext-build', build or rebuild external Pkg(s) at your will. This allow us to make modification to the source... who knows.
4. Added missing drivers like XhciDxe/ApfsSupport to both drivers64/drivers64UEFI (see buildpkg.sh).

 

examples:

...to download and build external sources (..not needed on the second run):

./ebuild.sh --ext-co -x64 -D NO_GRUB_DRIVERS_EMBEDDED -t XCODE8
./ebuild.sh -mc --no-usb -D NO_GRUB_DRIVERS_EMBEDDED -t XCODE8

or if you want to build them again:

./ebuild.sh --ext-build -x64 -D NO_GRUB_DRIVERS_EMBEDDED -t XCODE8
./ebuild.sh -mc --no-usb -D NO_GRUB_DRIVERS_EMBEDDED -t XCODE8

 

Instead, if you want them again as prebuilt binaries from github:

./ebuild.sh --ext-pre -x64 -D NO_GRUB_DRIVERS_EMBEDDED -t XCODE8
./ebuild.sh -mc --no-usb -D NO_GRUB_DRIVERS_EMBEDDED -t XCODE8

in all cases existing external drivers will be copied to their respective destinations if found, may be downloaded/built a month ago..

 

buildpkg.sh: 

Can now add duplicate choices, in case of drivers.

 

Let me know, here works, but testers are welcome!

 

 

 

ebuild.sh.zip

buildpkg.sh.zip

Edited June 19, 2018 by vector sigma
added a link to PACKAGES_PATH Wiki

[Zenith432]

Systems with a xhci controller and UEFI bios have a builtin XHCI driver, so XhciDxe does not belong in drivers64UEFI

When using boot7, USB controller drivers should not be used because they either don't work or wrest the controller away from legacy bios that should own it.

When using boot6 on a system that only has a XHCI controller (like mine) - XhciDxe needs to be included in boot6 with -D ENABLE_USB_XHCI or it can't boot form USB sticks.

So XhciDxe should only be in drivers64 when using boot6, the system has additional older type USB controllers and you never plan to boot Clover from the XHCI controller.

It would be nice in ebuild.sh to have a feature like target.txt - where to put personal defaults so they don't have listed again and again.  Today I put my personal defaults by adding them to the call to MainBuildScript.  Maybe it's useful to take them from an environment variable or a text file.  If such a feature is added, there needs to be an option to disable the defaults.

 

Presonally I'd prefer if anything involving downloading is in separate script and not part of ebuild.sh - but I'm not complaining because I just mod ebuild.sh locally.

 

Since you mention the Grub drivers with that -D - I think maybe invert the Grub logic so they need a -D to be included instead of excluded - because I don't think they're used much.  Also, there are recent binaries of efifs drivers here instead of building them from outdated source code.

 

Edited June 20, 2018 by Zenith432

[Slice]

1 hour ago, Zenith432 said:

Since you mention the Grub drivers with that -D - I think maybe invert the Grub logic so they need a -D to be included instead of excluded - because I don't think they're used much.  Also, there are recent binaries of efifs drivers here instead of building them from outdated source code.

 

Very interesting! Did you compile these drivers in our environments?

There was a big problem to adopt Grub drivers to UEFI rules and libraries.

[Zenith432]

They're binaries - there's nothing to compile.  The efifs sources only build on linux with gnuefi, but EFI products run on all EFI firmware :)

 

The source in Clover was prepared by AndyV from efifs version 0.9.  Current source is 1.3.  Also grub is updated.  I have not tried to upgrade the source in Clover.  It's complicated because of grub.

[Slice]

For binaries we have nothing to do. Just recommend users to download those binaries.

Useful drivers are GrubEXFAT, GrubNTFS, GrubUDF but I didn't expect new version will be noticably better. Expected more protection about copying these sources.

GrubZFS I still can't compile.