C++ proposition

[Jief_Machak]

I've created a branch called c++_noglobals to show that a C++ compiler is a C compiler with object added. Contrary of what a lot of people think, the object mechanism does NOT need a runtime or library. Because calls to method internally become calls to function with an added parameter : "this" (this is a pointer to a struct containing all members variables). C++ was first implemented with the C preprocessor.

 

Now C++ added 2 things that can't be compiled as normal code : rtti and exceptions. Well, I just deactivated them, problem solved.

 

Please have a look at what I committed : there is no trick, no library, the compilation process is untouched, the link command is untouched. It compiles with XCODE5 and GCC53.

The only catch is that the global variables of object type are not initialised. Var of simple type are initialised like in C. I'll explain in an other post and commit a version that will initialise globals variables of an object type.

 

@Slice and Clover developer : switching to a C++ compiler and start introducing few simple objects (string, buffer, array, date) is just a PLUS. It'll make the code more reliable and easy to read. The only problem C++ compiler can cause is for some developer to want to use very advanced C++ features that makes the code unreadable at the end. You've already understood that we must not do that! But using a string as we use an int without worrying about allocation, reallocation and destruction is just very nice.

PS : the names "cpp_utils, XStringW" can change, I just forgot to rename them.

[Slice]

Yes, it works.

I see output to screen instead of message buffer?

 

Then I have a question  how to live.

How we will synchronize commits to master and to c++ branch?

What will be caveats to finally switch to c++? I want to support it but I'm rather afraid.

[Jief_Machak]

I defined DEBUG_MAIN to 2 to see messages, and I just add few lines of code at the beginning of main to check it's working.

 

We can't live in 2 branches, you're right. If we want to do it, the best, I think, is to switch compiler by renaming the files from .c to .cpp in the master branch. We don't do more than that first. Developer can continue to work as before, writing C code (or C-like or C-compatible code, whatever) in .cpp files. We don't introduce object yet. We wait some time (few releases) and if there isn't any problems, we can decide to stay with C++ compiler and introduce basics objects.

If strange behaviour happens and they seems to complicated to fix, we can rename files back to .c. No work would be lost that way.

I'm pretty sure C++ compiler will produce the same code given the same source and we won't have any problem caused be C++ compiler. But you are right to be afraid and cautious. If we try it that way, the worst case scenario is a lot of messages saying it's not working.

 

I don't see any caveats to switch to C++. It is just added features at no cost. There is nothing C++ can't do. There is no dependancy added.

 

I think that's the best way because development won't be stopped, and a C++ compiled Clover would be tested by a lot of people and platform that way, before we introduce object, meaning before it's hard to go back to C.

 

I don't how well git handles renaming. I know commit history will be preserved. Still, it'll be good to agree on a date for the renaming of the files, so developer could have committed all their changes before. Just to be extra cautious if git pull doesn't work well after renaming.

 

For now, I'll finish a version that allow global object initialisation and make a post to explain.

I'll also install a C++ compiled Clover on my own computer. I think it's fair that I'm the first to try :-).

[apianti]

5 hours ago, Jief_Machak said:

I'll also install a C++ compiled Clover on my own computer. I think it's fair that I'm the first to try :-).

 

No, it's really not since I already told you I have done this. You don't seem to understand that it won't work with any other compilers (sans clang and gcc because they have compiler specific extensions enabled by default). There is much more than just RTTI and exceptions, it is working because you have not disabled compiler specific extensions. This branch does not build with anything in windows. In fact, using a compiler specific keyword __typeof__, makes this completely unbuildable and a disaster to fix. Never use compiler specific extensions directly in code!

[Jief_Machak]

"You already told me" : are you implying that I'm supposed to obey and stop immediately my c++ proposition?

 

You should tried to be nice, it might work better.

 

If dear microsoft didn't implement __typeof__, it can be easily removed and replace by the actual type of the variable. I know it's an extension, but I thought Microsoft would have implemented it too.

My bad, but it's not a big deal, I'll do it.

 

I still don't understand why you are saying it's not possible when I have one working on my computer. Have a look at freestanding environment and article talking about using C++ to compile kernel, for example.

[apianti]

First, I'm just talking, I have not been mean or nice, I do not care. I am telling you that it does not work on windows because I just tried to compile it and it does not build even after fixing __typeof__. Which for your information is not part of any language standard specification and is a compiler specific extension, it makes no sense to allow this in c because you should always know what type you are using and there is decltype and auto in c++. Also, I never said it wasn't possible, I told you I have already done it. There's just too many compilers to support to make it realistic unless you suggest we only support one compiler. C++ also provides no advantage over c in this situation.

[Jief_Machak]

Yes, you don't care, we can feel it.

If you had done it, maybe you should have proposed it.

If you really have reverted the __typeof__, have you committed them ?

 

Forgot a question : is there really someone compiling Clover on Microsoft ? If Yes, could you tell which setup do you have (Visual C ? Cygwin ?) and maybe help configure that on my computer so I can make sure, from now on, that my modification compile on Windows.

Edited January 3, 2020 by Jief_Machak

[apianti]

Me? I use visual studio 2012/2013/2015/2017.

[Jief_Machak]

__typeof__ allows you not to have to go over all your cast if you change the type of a var. It makes as more sense in C or C++.

 

There isn't any compiler specific in the c++ branch, apart from __typeof__. So if it doesn't compile on Microsoft, it must be something else. Please help me setting up an env like yours (or at least tell me what you are using) and I'll have a look.

 

Nothing's is worth spending all that negative energy...

[mhaeuser]

A few questions / cents...

 

What are the expected advantages from C++ at this level? Which kind of libraries or objects would be best off ported in your opinion? Is this something that is going to be proposed at edk2-devel? Is there a sort of validation existing code continues to work (C++ is *not* C plus objects. even though the standards have been synced in a couple of ways in the past, that also proves exactly this point), how is ported code validated? Are there going to be guidelines of usage (e.g. disallowing exceptions), or some sort of checklist what works and what does not?

 

Yes, only a very few compilers should be supported for this low level - one cannot reasonably validated dozens of compilers / compiler versions. If I were you, I'd support XCODE5 and CLANG9 - former is the de facto standard on macOS and very similiar to CLANG9 in the first place, latter is available cross-OS and supported for "Reproducible Builds" by edk2. MSVC might work even worse with C++ than C due to intrinsics insertion that somehow cannot be disabled properly, even though there *is* a switch (which does not work...?). Also, {censored} LTO does not allow providing the intrinsics oneself without explicit references.

[Jief_Machak]

@apiantiWith which version exactly did you try to compile my C++ branch?

Can I have the version where you fixed the __typeof__, so it'll save me some time?

Edited January 3, 2020 by Jief_Machak

[apianti]

VS 2017. I already know what the issues are, there is no need to spend negative energy or whatever.

 

There is absolutely no advantage. Anything that is allocated will be released by exiting boot services so there is no need for cleanup, the object behavior can be achieved in c by the same method as the edk, there is higher overhead. There are some intrinsics you can't disable like memset and memcpy, which are used to initialize pretty much anything by default. The difference in behavior is that when you disable the standard library for msvc it also disables all the language support, clang/gcc does not because these are also enabled through the compiler extensions.

Edited January 3, 2020 by apianti

[Jief_Machak]

@Download-Fritz It's already working for XCODE5. You mean GCC9? It's also working with the configuration GCC53 that use Gcc9.2 on my computer.

The difference in term of language between C and C++ exists, sure. But, apart from the cast from void* (that I have mistakenly fixed with __typeof__), all the existing Clover code compiling fine with both compiler.

I mean by that is : C++ brought object mechanism (+ differences like const, references, etc.) Library is a different thing. We currently use C without printf, we can use C++ without std::string or C++ stream.

 

Because we don't modify edkII, there is no interest to change the compiler. That will be for rEFIt_UEFI first, and probably only.

What is not possible is RTTI and exception. I'm experimenting global object initialisation but that is compiler dependant but quite easy ; so we might choose to leave it not woking. All the rest works : which is just defining object with methods, operators, etc. and using them.

 

25 minutes ago, apianti said:

VS 2017. I already know what the issues are, there is no need to spend negative energy or whatever.

 

There is absolutely no advantage. Anything that is allocated will be released by exiting boot services so there is no need for cleanup, the object behavior can be achieved in c by the same method as the edk, there is higher overhead. There are some intrinsics you can't disable like memset and memcpy, which are used to initialize pretty much anything by default. The difference in behavior is that when you disable the standard library for msvc it also disables all the language support, clang/gcc does not because these are also enabled through the compiler extensions.

Please share what you've done (like fixing __typeof__) so I can have a look too.

[mhaeuser]

8 minutes ago, Jief_Machak said:

@Download-Fritz It's already working for XCODE5. You mean GCC9?

No, I meant CLANG9, as I said. GCC is not cross-platform (I do not consider Cygwin or such native, let alone "safe").

 

9 minutes ago, Jief_Machak said:

But, apart from the cast from void* (that I have mistakenly fixed with __typeof__), all the existing Clover code compiling fine with both compiler.

Compiling is not enough, consider "sizeof('a')" - this will be "sizeof(int)" for C and "sizeof(char)" for C++, albeit both compile fine obviously. I'm not familiar enough with C++ to give you more dangerous examples, I hoped you already thought of such.

 

It's still not clear to me what the exact advantages are, or what problem you are trying to solve, I do not know C++ well enough to judge.

[Jief_Machak]

I using ebuild with --xcode5. I didn't try --xcode8. I don't see any CLANG9

When I do --xcode5, it uses the clang I have on my computer, which is currently "Apple LLVM version 10.0.0 (clang-1000.11.45.5)".

What should I do for CLANG9 ?

 

I'll go back to yo with the second part of your message.

[mhaeuser]

2 minutes ago, Jief_Machak said:

I don't see any CLANG9

Oh sorry, it has been renamed to CLANGPDB: https://github.com/tianocore/edk2/commit/14672c34bdced0b09d4a4226e648e14701442b3e

It's the only truly cross-platform toolchain, it supports all relevant architectures, and it is supported for Reproducible Builds, so I think it is a must-have really. If XCODE5 compiles, CLANG9 should compile too, but I recommend runtime testing.

 

2 minutes ago, Jief_Machak said:

I'll go back to yo with the second part of your message.

You will what now?

[Jief_Machak]

1 minute ago, Download-Fritz said:

You will what now?

Answer your other questions soon.

 

2 minutes ago, Download-Fritz said:

CLANGPDB

It's not available in the current setup, isn't it ? I have to download edkII and setup a Clover into that full edkII, like it was before ? Or just update the tools_def.txt ?

[mhaeuser]

@Jief_Machak I have no clue about Clover building, I'm mainly asking out of curiousity (and for our projects). edk2 is taking many steps at once right now (staging both a Rust interface and a C stdlib implementation), C++ is yet another direction now. I wouldn't mind stuff making life easier, so I'm trying to find out why they are developed.

Updating tools_def should be enough, but I don't know how messed up the Clover edk2 fork is... for acidanthera, we run mostly unpatched.

[Jief_Machak]

Ok thanks.

[Slice]

Hey guys,

I tested Windows compilation with VS2017 toolset. It works... really? Mostly no. Normal Clover can be compiled only in macOS with Clang or with gcc. VS2017 compilation rules exists mostly for education purpose.

As well we can't use pure EDK2 no matter what acidanthera may say.

And this C++ version by Jief does work compiled by XCODE5 or GCC53. I will support this branch.

I don't know how to be with old branch master. It depends on other developers who will support it. I can imaging that all active developers will switch to c++.

[Jief_Machak]

Hi all,

to continue my experiments, I committed a version that enable globals objects being initialised. See https://github.com/CloverHackyColor/CloverBootloader/tree/c++_globals

 

Explanations :

C allows global variables. So C++ does it too. The problem is that a variable in C would be INT or CHAR or any simple type or any combination with struct. Initialising this is easy and doesn't require code. The compiler put the value in data section and set the address of the var.

In C++, a var can have a type which is a user defined object, which can have a constructor. Initialising variable of an object type requires to call some piece of code BEFORE the main function. To do that, the compiler generates function calls that will be called be a "launcher" before main. These function calls atexit to register the destructor and call the constructor. Because we don't have that launcher, we have to call these functions in our main. The way compilers do it is, of course, compiler dependant.

Gcc :

Gcc will create .ctor sections, one per module, containing call to make. To read this from our main, I modified the linker script GccBase.lds, adding this :

.ctorss ALIGN(CONSTANT(COMMONPAGESIZE)) : {
    __beginning_of_section_ctors = .;  /* create symbol for start of section */
    KEEP (*(.ctors))
    __end_of_section_ctors = .; /* create symbol for end of section */
  }

This will create a .ctorss section, containing all the .ctor sections, with a symbol to get the address of the beginning and the end of the section (__beginning_of_section_ctors and __end_of_section_ctors). Then I just iterate and call. See cpp_util/globals_ctor.cpp

Clang :

Clang creates a section __mod_init_func at the beginning of the __DATA segment. The segment __DATA will become a .data section in pecoff (mtoc does this). I didn't find a way to create a symbol before and after. So, to get the beginning, I look into the pecoff format and find the beginning of the section .data.

To get the end, I create a symbol at the beginning of the next section, which is __const. To be sure it'll be first, I created an order file for clang linker (BaseTools/Scripts/ClangLinkOrder.txt).

So there is 2 assumptions here (which can be annoying if clang changes that one day) :

  - __mod_init_func is at the beginning of the __DATA segment.

  - __const is the section immediately following __mod_init_func.

 

There is 2 other ways it can be done for Clang.

* One is to modify mtoc to keep the section __mod_init_func in pecoff header. I tried it. Works. It's probably better as so assumptions are made, but a patch must be maintain and apllied in buildmtoc.sh.

* An other is to call the functions "__GLOBAL__sub_I_[module name]". Each function in __mod_init_func are, in fact, calling a function named "__GLOBAL__sub_I_[module name]". We can just call these function. The only disadvantage is to remember to call "__GLOBAL__sub_I_[module name]" from main when you create the first global object in a module. The big advantage is not to have to modify anything about the linking and pecoff generation.

 

Cheers.

[apianti]

Sounds very easy. Also, that's only two of how many compilers...?

[Jief_Machak]

Here is the mtoc version : https://github.com/CloverHackyColor/CloverBootloader/tree/c++_globals_mtoc/rEFIt_UEFI

The mtoc.NEW_jief and its source are in BaseTools dir for the experiment. If we keep that solution, a source patch will be made and buildmtoc.sh will be modified to apply the patch.

 

18 hours ago, Slice said:

Normal Clover can be compiled only in macOS with Clang or with gcc

@apianti seems that Slice said that these 2 are the only supported. Which seems fair for a project that is made to run macOs. But if any developer would like another one, it's probably not hard to adapt.

[Jief_Machak]

And last, the GLOBAL__sub version : https://github.com/CloverHackyColor/CloverBootloader/tree/c++_globals_GLOBAL__sub

 

Any opinion ?

[Slice]

3 hours ago, Jief_Machak said:

And last, the GLOBAL__sub version : https://github.com/CloverHackyColor/CloverBootloader/tree/c++_globals_GLOBAL__sub

 

Any opinion ?

Need a time to have an opinion.

But I really respect your work.