New Driver for Realtek RTL8111

[Henry2010]

Thank you very much for developing this kext.

[InferiorYeast]

Thank you so much for the great driver! I was just wondering if you have considered incorporating the change dmazar made to checkLinkStatus() into your code. Without this change I too get cases where the link goes down and will not come back up. Would there be any downsides to other users by adding this change to your code?

 

- changed checkLinkStatus():

} else {
	 if (interrupt) {
		 /* Stop watchdog and statistics updates. */
		 timerSource->cancelTimeout();
		 setLinkDown();
		
		 if (tp->mcfg == CFG_METHOD_23) {
			 WriteReg32(ERIDR, 0x00000001);
			 WriteReg32(ERIAR, 0x8042f108);
		 }
	 } else {
		 /* Called from enable() - just notify that the link is down. */
		 setLinkStatus(kIONetworkLinkValid);
	 }
}

[Mieze]

Thank you so much for the great driver! I was just wondering if you have considered incorporating the change dmazar made to checkLinkStatus() into your code. Without this change I too get cases where the link goes down and will not come back up. Would there be any downsides to other users by adding this change to your code?

 

The issue dmazar reported has been resolved by moving setLinkStatus(kIONetworkLinkValid) from start() to enable(). In case you still have trouble with the latest version please report back with a complete kernel log using the debug build showing the issue.

 

Mieze

[RehabMan]

Noticed in 10.9 OS X Mavericks that Apple is going to use SMB2 as the default file sharing protocol.  I'm very surprised they are dumping AFP for a Microsoft protocol (did not see that one coming).

 

But this is great news, perhaps SMB access to Windows shares will finally work correctly on OS X!

 

From OSX_Mavericks_Core_Technology_Overview.pdf:

 

0. SMB2

   SMB2 is the new default protocol for sharing files in OS X Mavericks. SMB2 is superfast, increases security, and improves Windows compatibility.

   • Efficient. SMB2 features Resource compounding, allowing multiple requests to be sent in a single request. In addition, SMB2 can use large reads and writes to make better use of faster networks as well as large MTU support for blazing speeds on 10 Gigabit Ethernet. It aggressively caches file and folder properties and uses opportu- nistic locking to enable better caching of data. It’s even more reliable, thanks to the ability to transparently reconnect to servers in the event of a temporary disconnect.

   • Secure. SMB2 supports Extended Authentication Security using Kerberos and NTLMv2.

   • Compatible. SMB2 is automatically used to share files between two Mac computers running OS X Mavericks, or when a Windows client running Vista, Windows 7, or Windows 8 connects to your Mac. OS X Mavericks maintains support for AFP and SMB network file-sharing protocols, automatically selecting the appropriate protocol as needed. 

[Mieze]

 

Noticed in 10.9 OS X Mavericks that Apple is going to use SMB2 as the default file sharing protocol.  I'm very surprised they are dumping AFP for a Microsoft protocol (did not see that one coming).

 

But this is great news, perhaps SMB access to Windows shares will finally work correctly on OS X!

 

 

 

We'll have to wait for the outcome of this move. With Apple you can't be sure that they won't change directions completely within 1 or 2 years again. A few years ago it seemed almost certain that they were moving away from HFS+ in favor of ZFS until the project was suddenly cancelled. There is no guarantee that this won't happen again.

 

Mieze 

[beta992]

We'll have to wait for the outcome of this move. With Apple you can't be sure that they won't change directions completely within 1 or 2 years again. A few years ago it seemed almost certain that they were moving away from HFS+ in favor of ZFS until the project was suddenly cancelled. There is no guarantee that this won't happen again.

 

Mieze 

I have feeling since Steve Jobs (R.I.P.) is no more at Apple, more new and experimental things will go into the OS X system. Look at IOS7, the new MacPro, etc. I'm not saying Steve Jobs was a 'bad person', but just saying that more people at Apple have more influence because the 'main man' position is lower. Not want to be to much off-topic..

 

Thanks for your new driver Mieze. It seems my SMB performance has been increased.

[Mieze]

I have feeling since Steve Jobs (R.I.P.) is no more at Apple, more new and experimental things will go into the OS X system. Look at IOS7, the new MacPro, etc. I'm not saying Steve Jobs was a 'bad person', but just saying that more people at Apple have more influence because the 'main man' position is lower. Not want to be to much off-topic..

 

It's hard to tell where Apple is going now. Steve had a vision he was following which didn't help to make his decisions comprehensible for most people outside of Apple, but it was that thing that made Apple different than all those high tech companies doing business as usual. My worst fear is that Apple will be streamlined by the management resulting in the loss of their innovative capability.

 

Mieze 

[Mieze]

After watching some of WWDC sessions my enthusiasm for 10.9 is slowly fading. There are mayor changes with regard to kext development:

• Third party kexts should move to /Library/Extensions and need to be code signed in order to load successfully from that directory.
• For the time being we are still able to install legacy kexts in /System/Library/Extensions which do not need to be code signed but will produce that ugly message on load nozyczek posted a few days ago.
• They are planning to lock /System/Library/Extensions at some time in the future making it impossible to load unsigned kexts.

As I'm absolutely sure that you are all using the driver in compliance with Apple's license agreement, i.e. with Realtek based add-on cards in a MacPro, it shouldn't be a problem for me to get a Mac developer program membership (currently I'm only in the iOS developer program) and sign the prebuilt kext in order to cope with the new situation at some point in the future. 

 

As of now the warning on load is only a cosmetic issue. In case you already are a member of the Mac developer program you might also sign the kext with your own ID to get rid of it.

 

I understand that Apple's move is an important step towards improved security and might have been inevitable to prevent root kits from compromising the system but it will definitely make things harder in the future.

 

Mieze

[genzai]

 

I understand that Apple's move is an important step towards improved security and might have been inevitable to prevent root kits from compromising the system but it will definitely make things harder in the future.

 

 

Holy {censored}-o-la,

This is very interesting news, but doesn't this signal a major issue for FakeSMC once this future driver signing requirement is enforced? (not to mention that every hacint0sh using 10.9 is now going to suffer that ugly pop-up on boot?) The paranoid part of my brain can see this as a specific move against the hacint0sh. Or am i confusing the implications here?

Thanks,

g\

[Mieze]

Holy {censored}-o-la,

This is very interesting news, but doesn't this signal a major issue for FakeSMC once this future driver signing requirement is enforced? (not to mention that every hacint0sh using 10.9 is now going to suffer that ugly pop-up on boot?) The paranoid part of my brain can see this as a specific move against the hacint0sh. Or am i confusing the implications here?

Thanks,

g\

Well, I don't think this is a specific action against hackintoshing because they are introducing a lot of additional security features like address space randomization. It's rather a response to the growing threat by malware.

 

FakeSMC will be affected too, but it won't be the end of OS X running on third party hardware. In fact it will make things a little bit harder. What it effectively means will depend on the following questions:

• Will there be a backdoor or a way to completely disable verification of the code signatures?
• How tolerant will they be towards the hackintosh community? Will they take any actions in case  someone publishes a signed release of FakeSMC?

After all you can always get a Mac Developer Program Membership and sign kext's with your own ID. As long as you don't publish the signed kexts Apple won't notice what you are doing.

 

Mieze

[RehabMan]

I finally got around to looking at my SMB performance issue a bit more.  And I've made great progress.  I'm now getting 60MB/sec reads from my WHS2011 server and 30-40MB/sec writes using your driver with some modifications.  This is with a simple Finder based file copy of a large file (happens to be a 5.13GB MKV backup of the movie 2012).

 

I have created another fork of your github repo, just to contain the changes I made to deal with this issue.  It is at: https://github.com/RehabMan/RTL8111_driver_for_OS_X

 

You will see two commits.  The first implements SetProperties which allows one to change the intrMitigate on the fly. I have also published the source to my 'ioio' command line utility that allows this.  It is at: https://github.com/RehabMan/OS-X-ioio.  With it you can do things like:

ioio -s RTL8111 intrMitigate 0xaf51

I encourage you to pull this change into your own repo or otherwise experiment with mine, since it is much easier to experiment with different values of intrMitigate.  You can actually change it while doing a large file copy and see the difference (if any) immediately.

 

I have left the intrMitigate value in this repo at your value of 0xaf54 although I only get 50MB/sec reads from the server with that value.  If I change it to 0xaf51 (using either ioio or by building the change into the Info.plist), I get the read speeds I mention above at close to 60MB/sec.

 

The second commit is discussed below...

 

Read performance fix:

This one turned out to be simple.  Simply changing the code in interruptOccurred such that the IntrStatus is cleared before doing any work fixed the problem.  This change I had disclosed in a previous post.  I've re-organized the code in interruptOccurred a bit to accomodate the other change for write performance, but the only change required was to make the write to IntrStatus right after the read for IntrStatus instead of at the end of the function as it was before.

status = ReadReg16(IntrStatus);
WriteReg16(IntrStatus, status);

Write performance fix:

This one was a bit more involved.  You will remember that I had intended to try installing a "real" interrupt handler to see what would happen if I cleared the interrupt status sooner... right after the device asserts IRQ.  Well, this worked.  You can see the changes by searching for CLEAR_STATUS_IN_INTERRUPT.  It boils down to doing this in a real interrupt handler:

bool enable = ml_set_interrupts_enabled(false);
WriteReg16(IntrMask, 0x0000);
UInt16 status = ReadReg16(IntrStatus);
WriteReg16(IntrStatus, status);
_status |= status;
WriteReg16(IntrMask, intrMask);
ml_set_interrupts_enabled(enable);
interruptSource->interruptOccurred(0, 0, _msiIndex);

And then in the workloop based "fake" interrupt handler, reading/writing from/to _status instead of the IntrStatus register:

bool enable = ml_set_interrupts_enabled(false);
status = _status;
_status = 0;
ml_set_interrupts_enabled(enable);

I made the code conditional with #if so I could test easily with it on/off and compare results.  This makes me wonder if doing additional work (where practical) in the interrupt handler might be beneficial.

 

Now I'm off to compare performance against slice's version, so I can see where I'm at with performance so far.  I don't know if this is the true solution to my problem, but the findings are certainly interesting nonetheless.

 

[genzai]

Very interesting. sorry if this is getting a bit off topic but i have not seen this issue discussed elsewhere yet. Effectively it sounds like a good way for apple to get almost every hackintosh user to pay $99/ year for use of OS X. In fact that sounds about right to me

g\

After all you can always get a Mac Developer Program Membership and sign kext's with your own ID. As long as you don't publish the signed kexts Apple won't notice what you are doing.

 

Mieze

[Mieze]

Very interesting. sorry if this is getting a bit off topic but i have not seen this issue discussed elsewhere yet. Effectively it sounds like a good way for apple to get almost every hackintosh user to pay $99/ year for use of OS X. In fact that sounds about right to me

g\

 

Yes, and I hope it will have a positive side effect too: killing tonymacx86 and his x-beast tools.

 

Mieze

[RehabMan]

Very interesting. sorry if this is getting a bit off topic but i have not seen this issue discussed elsewhere yet. Effectively it sounds like a good way for apple to get almost every hackintosh user to pay $99/ year for use of OS X. In fact that sounds about right to me

g\

 

I'm a little more concerned about how this might affect the patching of system kexts.  Sure, we can always sign new (open source) kexts that support various devices, but what about patches to kexts that already exist and are signed.

 

Perfect example is AppleIntelCPUPowerManagement.kext and AppleHDA.kext, two commonly patched kexts required to make almost any hack work well.

 

It really isn't surprising that Apple is doing this.  More surprising is the fact they waited as long as they have before implementing it.

[Mieze]

I'm a little more concerned about how this might affect the patching of system kexts.  Sure, we can always sign new (open source) kexts that support various devices, but what about patches to kexts that already exist and are signed.

 

As far as I know 10.9 currently shows an error messages when kexts inside /S/L/E are unsigned or signature verification produced an error but it doesn't refuse to load them. The problem will arise when they get serious with signature verification and locking of /S/L/E but we'll cross that bridge when we come to it.

 

If it was their intention to dongle OS X to specific hardware they would have gone the Microsoft way, but the fact that they didn't, makes me believe that its not their intention to eliminate the hackintosh community.

 

Mieze

[Mieze]

In version 1.1.0 the driver hands over received packets to the network stack including the Ethernet CRC. Recently I found out that this results in hardware calculated TCP/IP checksums to be ignored so that the network stack calculates them again. Although this doesn't prevent the driver from working correctly, it causes slightly higher CPU load while receiving packets.

 

In order to fix that issue I already pushed version 1.1.1 to githup which removes the CRC before handing over received packets to the stack. I updated the binary to version 1.1.1 last night too.

 

Mieze

Edited June 30, 2013 by Mieze

[nozyczek]

Mieze,

Awesome work with 1.1.1. iperf is showing stable ~942 both ways. Copying from/to SMB appears to be quick too.

Just from curiosity what are you working on now? What is your focus, features, etc?

Thanks

nozyczek

[RehabMan]

In version 1.1.0 the driver hands over received packets to the network stack including the Ethernet CRC. Recently I found out that this results in hardware calculated TCP/IP checksums to be ignored so that the network stack calculates them again. Although this doesn't prevent the driver from working correctly, it causes slightly higher CPU load while receiving packets.

 

In order to fix that issue I already pushed version 1.1.1 to githup which removes the CRC before handing over received packets to the stack. I updated the binary to version 1.1.1 last night too.

 

Mieze

Any comments/questions/advice on my findings on the SMB perf issue and interrupt handling as described in post #211?  I still don't get quite what slice's driver can provide (for writes) but it is close.  I'll keep digging...

 

BTW, looking forward to your updating to the latest Linux code... maybe there is something in there that is different with my chip.

 

And I thought I'd let you know it is not just the ProBook.  I was contacted via PM from user 'Harkaitz' and he says "Finally I was able to extract [RehabMan note: meaning compile] the previous kext, it works much much better than lnx2mac or mieze´s kext on my system (realtek 8111F on a asus p8h77-i)." 

 

Can you tell what might be in common?  Mine being 8168/8111E chipset 14?

[Mieze]

Any comments/questions/advice on my findings on the SMB perf issue and interrupt handling as described in post #211?  I still don't get quite what slice's driver can provide (for writes) but it is close.  I'll keep digging...

 

BTW, looking forward to your updating to the latest Linux code... maybe there is something in there that is different with my chip.

 

And I thought I'd let you know it is not just the ProBook.  I was contacted via PM from user 'Harkaitz' and he says "Finally I was able to extract [RehabMan note: meaning compile] the previous kext, it works much much better than lnx2mac or mieze´s kext on my system (realtek 8111F on a asus p8h77-i)." 

 

Can you tell what might be in common?  Mine being 8168/8111E chipset 14?

 

The answer is simple: nothing with regard to the local machine! Basically it's not a problem of my driver but a bug of Apple's SMB implementation because Apple's drivers show the same behavior. If you are affected or not depends on the exact configuration of the remote machine which triggers it and in case you have this problem with my driver a real Mac will most probably show the same issue as my driver.

 

I've checked your changes and ran a lot of tests on my machines with regard to the SMB issue but I came to the conclusion that I will not adopt the patches because they result in a significantly increased CPU load affecting all users while only a few will benefit from them. Please keep in mind that most users won't ever experience any SMB performance issue. I created the driver as an open source project and I appreciate anyone modifying it to fit his or her personal needs. My intention was to create a driver that best fits general purpose applications and I'm not giving any guarantee that it's optimal under all circumstances.

 

Apple has recently introduced a new network driver architecture moving away from the push model towards a pulled model where the driver is actively dequeueing packets from the send queue giving driver programmers the ability to implement custom QoS mechanisms. Besides the interrupt driven reception they have also introduced a polled receive mode giving the network stack the ability to switch receive mode at runtime. Although these additions haven't made their way into the SDK you can take a look at it in the IONetworkingFamily-85.2 source code. By the way, starting with 10.8.3 Apples drivers (Broadcom and Intel 82574L) already implement this new new model. I'm very keen to see how this will affect driver performance and system.

 

Don't expect any progress in the near future because I'm too busy to spent much time on the driver at the moment. As a freelancer you don't have this strict obligations like an employee. I'm free to decide how to spent my time but I need to get my work done in order to make a living and I must admit that I neglected my iOS projects during the last two month in favor of hackintosh projects so that there is a bunch of unfinished work waiting for me.

 

Mieze

[RehabMan]

 

The answer is simple: nothing with regard to the local machine! Basically it's not a problem of my driver but a bug of Apple's SMB implementation because Apple's drivers show the same behavior. If you are affected or not depends on the exact configuration of the remote machine which triggers it and in case you have this problem with my driver a real Mac will most probably show the same issue as my driver.

 

I've checked your changes and ran a lot of tests on my machines with regard to the SMB issue but I came to the conclusion that I will not adopt the patches because they result in a significantly increased CPU load affecting all users while only a few will benefit from them. Please keep in mind that most users won't ever experience any SMB performance issue. I created the driver as an open source project and I appreciate anyone modifying it to fit his or her personal needs. My intention was to create a driver that best fits general purpose applications and I'm not giving any guarantee that it's optimal under all circumstances.

 

Apple has recently introduced a new network driver architecture moving away from the push model towards a pulled model where the driver is actively dequeueing packets from the send queue giving driver programmers the ability to implement custom QoS mechanisms. Besides the interrupt driven reception they have also introduced a polled receive mode giving the network stack the ability to switch receive mode at runtime. Although these additions haven't made their way into the SDK you can take a look at it in the IONetworkingFamily-85.2 source code. By the way, starting with 10.8.3 Apples drivers (Broadcom and Intel 82574L) already implement this new new model. I'm very keen to see how this will affect driver performance and system.

 

Don't expect any progress in the near future because I'm too busy to spent much time on the driver at the moment. As a freelancer you don't have this strict obligations like an employee. I'm free to decide how to spent my time but I need to get my work done in order to make a living and I must admit that I neglected my iOS projects during the last two month in favor of hackintosh projects so that there is a bunch of unfinished work waiting for me.

 

Mieze

 

Hey, no worries!

 

If I run across people who experience the problem, I'll point them to my fork.

 

The only reason I went down this path is because the only use I have for ethernet on my laptop is when I need to transfer large files (usually video), because generally ethernet is much much faster than WiFi.  With the performance of your driver without my patches, that was not the case, so I needed to come up with a solution.

 

Thanks for the feedback and great starting point.  Good luck with your iOS projects.

[imacX86]

Hey, no worries!

 

If I run across people who experience the problem, I'll point them to my fork.

 

The only reason I went down this path is because the only use I have for ethernet on my laptop is when I need to transfer large files (usually video), because generally ethernet is much much faster than WiFi.  With the performance of your driver without my patches, that was not the case, so I needed to come up with a solution.

 

Thanks for the feedback and great starting point.  Good luck with your iOS projects.

I was able to pull from your fork. Thanks for merging it with latest master branch! 

[Mieze]

Hey RehabMan,

 

please try this! While doing some research on the interrupt rate, I had an idea how to improve performance without causing much additional CPU load.

 

Mieze

RealtekRTL8111-RehabMan5.zip

[imacX86]

Hey RehabMan,

 

please try this! While doing some research on the interrupt rate, I had an idea how to improve performance without causing much additional CPU load.

 

Mieze

I have tried this on my ProBook 4440s. Not very good results: barely ~5MBs with drops to 200KBs. With latest RahabMan driver ~60MBs. with drops to 39MBs.

 

To get a prospective on my network raw performance: NFS - 117MBs sustained on RehabMan driver.

[Mieze]

I have tried this on my ProBook 4440s. Not very good results: barely ~5MBs with drops to 200KBs. With latest RahabMan driver ~60MBs. with drops to 39MBs.

 

To get a prospective on my network raw performance: NFS - 117MBs sustained on RehabMan driver.

 

As the remote machine is causing this issue, please describe it's configuration, CPU, NIC and OS.

 

Mieze

[imacX86]

As the remote machine is causing this issue, please describe it's configuration, CPU, NIC and OS.

 

Mieze

The remote is Open Solaris variant (Nexenta NAS) 3-core AMD CPU with Intel NIC (Intel Gigabit CT PCI-E Network Adapter EXPI9301CTBLK). It is running full-duplex from Windows (120MBs both ways) and real Macs. I have (RahabMan) comparable speed readings with iMacs/MBP/mini/Hacks with Marvell Yukon Gigabit Adapter 88E8053. So no there is no any issues with remote machine it has been rock solid performer for years.

 

SMB is not exactly best performing on Lion, it is even worse on ML, but not that slow (5MBs). On AFP I am getting ~124MBs Mac-to-Mac, Mac-to-Hack, Hack-to-Hack (on Marvell NICs).  I did not try AFP with Realtek NIC though. When on WiFi (5GHz) - I am getting 15MBs (ProBooks 4540s). Wired 1Gbs should be faster then any WiFi for sure.

 

Just to be clear, I use MBs as Mega Bytes per Seconds.