New Driver for Realtek RTL8111

[dmazar]

Windows driver: Realtek, 10.6.2011., 7.46.610.2011. I think it's installed from mobo CD, but I do not know if it's updated from Windows Update.

 

Linux: there are two drivers:

- r8169 which is part of Linux - Windows -> reboot into Linux (net works) -> reboot into OSX (net works)

- r8168 from Realtek - Windows -> reboot into Linux (net works) -> reboot into OSX (net does not work)

 

Thanks for still thinking about this. If you think that some registers dumps at some point will help, just tell. I can do them from Linux and OSX driver.

[Mieze]

Windows driver: Realtek, 10.6.2011., 7.46.610.2011. I think it's installed from mobo CD, but I do not know if it's updated from Windows Update.

 

Realtek released new Windows drivers during the last days. Maybe you should try them to see if they resolve the problem as I assume that they use a similar setup/firmware as the Linux drivers?

 

http://www.realtek.c...3&GetDown=false

 

Thanks for still thinking about this. If you think that some registers dumps at some point will help, just tell. I can do them from Linux and OSX driver.

 

We've found a lot of facts but what we need most now is a new theory how to reconcile them. As far as I know only chipset 14 is affected of the problem so that it might be the best approach to check the initialization steps that are specific to this chip. As soon as I have an idea I'll let you know.

 

Mieze

[genzai]

Hi Mieze,

Thank you so much for this dev work. Using lnx2mac which had a very slow dev cycle, i was having problems with extended high traffic use of the nic one one of my systems. It would just die until a reboot. I would also have to inactivate and reactivate if i unplugged and replugged the cable. Both these issues seem solved by your driver, and i am also getting a good %10 more bandwidth it seems. I have not tried the Slice driver yet but i am happy with yours. Maybe you guys should join forces

 

Anyway, i do use windows occasionally and would lobe to see that issue resolved. I will test it on my system to see if i have the same issue. One request i have is, since you say that offload wont work with jumbo frame is if you can allow offload to disabled when jumbo frames are used or if thats not possible release a version with jumbo frame support (and no offload). That would be helpful as sometimes i use this feature on internal network setups.

Thanks!

g\

[Mieze]

Anyway, i do use windows occasionally and would lobe to see that issue resolved. I will test it on my system to see if i have the same issue. One request i have is, since you say that offload wont work with jumbo frame is if you can allow offload to disabled when jumbo frames are used or if thats not possible release a version with jumbo frame support (and no offload). That would be helpful as sometimes i use this feature on internal network setups.

 

As far as I know only chipset 14 is affected by the problem while the 8111E-VL (Chipset 16) and 8111F (Chipset 17) found on recent mainboards seem to work fine.

 

On the receiver side the driver should tolerate jumbo frames as long as they fit into a single buffer. Eventually you'll have to increase kRxBufferPktSize (current value 2000) to cope with your setup.

 

On the transmitter side it makes no sense to use jumbo frames as every devices works well with standard frames and disabling the offload features will result in a significant performance impact even on fast systems.

 

Mieze

[saltspork]

Limitations

• Support for the Realtek RTL8111B/8168B is still experimental and might not work at all. Therefore it is only included in debug builds and has never been tested successfully because I don't have access to a board with one of these outdated chips.
  

Known Issues

• The code for RTL8111B/8168B NICs is untested and will probably not work as expected.
  

 

Hoping I can help out with that. I've got an old EP35-DS3 w/ 8111B that I don't really value for anything but testing.

 

I installed the debug version of your driver and it works mostly fine with the massive exception of TCP. There's also a lot of spam in the system log from a timer loop.

 

I've attached my system.log and netstat -s output. Let me know what else I can do to help test.

 

netstat.txt

system.log.txt

[Mieze]

I installed the debug version of your driver and it works mostly fine with the massive exception of TCP. There's also a lot of spam in the system log from a timer loop.

 

Hello saltspork,

 

thank you very much for the debug data. In order to get rid of the timer log messages comment out the following two lines in timerActionRTL8111B(IOTimerEventSource *timer):

 

DebugLog("timerActionRTL8111B() ===>\n");

 

DebugLog("timerActionRTL8111B() <===\n");

 

As only TCP isn't working I guess that this is an issue with TCP segmentation offload. Please change this method

 

UInt32 RTL8111::getFeatures() const

{

return (kIONetworkFeatureMultiPages | kIONetworkFeatureHardwareVlan | kIONetworkFeatureTSOIPv4 /*| kIONetworkFeatureTSOIPv6*/);

}

 

into

 

UInt32 RTL8111::getFeatures() const

{

return (kIONetworkFeatureMultiPages | kIONetworkFeatureHardwareVlan);

}

 

to disable it. Good luck!

 

I'm currently preparing a new release and will include improved 8111B support if we can get it working with TCP.

 

Mieze

[saltspork]

Hi Mieze.

 

I've commented out all the bits above and while the debug messages are cleared up TCP still isn't working.

 

An updated system.log is attached although I doubt it will help much.

 

Thanks for your efforts in any case.

 

system.log.txt

[Mieze]

Hi Mieze.

 

I've commented out all the bits above and while the debug messages are cleared up TCP still isn't working.

 

An updated system.log is attached although I doubt it will help much.

 

Thanks for your efforts in any case.

 

system.log.txt

 

Hello saltspork!

 

Thanks for the log data. What makes me wonder is the fact that TCP packets seem to go out but there is nothing coming back as if they all ended up in a black hole. No bad packets, no errors. Can you please describe the way you tested TCP?

 

Mieze

[saltspork]

Thanks for the log data. What makes me wonder is the fact that TCP packets seem to go out but there is nothing coming back as if they all ended up in a black hole. No bad packets, no errors. Can you please describe the way you tested TCP?

 

At first I wasn't testing very scientifically but I've gone back and done things properly with Wireshark on both ends. First trying to SSH to a remote system, and then the same remote system trying to SSH to the Hackintosh.

 

I don't know enough about TCP to say anything for certain, but it appears that the driver spits out invalid TCP packets that aren't even worthy of acknowledgement from a remote system, literally.

 

I've attached pcap files from each end, nicely synchronised.

 

pcap.tar.gz

[Mieze]

Update Notification

 

I released version 1.0.1 (source and binary) which includes two important changes:

• Improved behavior when rx checksum offload isn't working. When hardware checksum verification of received packets fails, the driver now considers the packet as unchecked and hands it over to the network stack letting it repeat the verification in software instead of marking the packet as bad.
  
• The NIC's model name is added to IORegistry so that System Profiler will show it properly.
  

The download URL is still the same.

 

Mieze

[dmazar]

Realtek released new Windows drivers during the last days. Maybe you should try them to see if they resolve the problem as I assume that they use a similar setup/firmware as the Linux drivers?

Tried. No luck with new drivers.

 

Since I know that

r8169 from Linux leaves controller in a good condition and your driver works after it, while after

r8168 it does not, maybe comparing "shutdown" code would help? Can you give me the hint what routines to check?

[Mieze]

Tried. No luck with new drivers.

 

Since I know that

r8169 from Linux leaves controller in a good condition and your driver works after it, while after

r8168 it does not, maybe comparing "shutdown" code would help? Can you give me the hint what routines to check?

 

The drivers entry points are usually a good starting point:

 

static struct pci_driver rtl8169_pci_driver = {

.name = MODULENAME,

.id_table = rtl8169_pci_tbl,

.probe = rtl_init_one,

.remove = rtl_remove_one,

.shutdown = rtl_shutdown,

.driver.pm = RTL8169_PM_OPS,

};

 

 

static const struct net_device_ops rtl_netdev_ops = {

.ndo_open = rtl_open,

.ndo_stop = rtl8169_close,

.ndo_get_stats64 = rtl8169_get_stats64,

.ndo_start_xmit = rtl8169_start_xmit,

.ndo_tx_timeout = rtl8169_tx_timeout,

.ndo_validate_addr = eth_validate_addr,

.ndo_change_mtu = rtl8169_change_mtu,

.ndo_fix_features = rtl8169_fix_features,

.ndo_set_features = rtl8169_set_features,

.ndo_set_mac_address = rtl_set_mac_address,

.ndo_do_ioctl = rtl8169_ioctl,

.ndo_set_rx_mode = rtl_set_rx_mode,

#ifdef CONFIG_NET_POLL_CONTROLLER

.ndo_poll_controller = rtl8169_netpoll,

#endif

 

I assume that you are familiar with driver programming for Linux but in case you need an introduction "Linux Device Drivers" will help you. The 3rd edition is available as a PDF file for free. Personally I would start with the driver's remove or shutdown routines. If you have further questions, in particular with regard to the mapping between Linux and OS X routines, feel free to contact me again (PM or mail are also ok).

 

Good luck!

 

Mieze

[Scellow]

Hi there, thanks for your hard work !

I tried with 8105E but it don't work, if it's not too hard to do could you add the suport for 8105E please ^^ ? i only have one PCIe slot for my HD7770 so i can't add an working one :/

 

The original driver for the 8105E work but only in 10Mbps i can't get it working in 100Mbps mode , it connect but give me an false IP so nothing work ..

10Mbps is okay but i work with big files so it's a little annoying at times ..

here is my original post :

http://www.insanelym...ce-for-100mbps/

[Mieze]

Hi there, thanks for your hard work !

I tried with 8105E but it don't work, if it's not too hard to do could you add the suport for 8105E please ^^ ? i only have one PCIe slot for my HD7770 so i can't add an working one :/

 

The original driver for the 8105E work but only in 10Mbps i can't get it working in 100Mbps mode , it connect but give me an false IP so nothing work ..

10Mbps is okay but i work with big files so it's a little annoying at times ..

here is my original post :

http://www.insanelym...ce-for-100mbps/

Hello scellow,

 

unfortunately its not that easy just to add support for the 8105E and I'm very busy at the moment. My driver is based on the Realtek's r8168 linux driver for their gigabit NICs but they have a different linux driver for their FE devices. Basically we would have to create a completely new driver or try to merge both.

 

Maybe you should talk to Slice because he has already implemented basic support for the 8105E. Although the code isn't working for this chip and there is still a lot of work to be done, he has the the better starting point.

 

Mieze

[Scellow]

Hi Mieze,

 

Thanks for your reply , i guess this is hard to port even an app so i bet a driver is painfull :/

I asked Slice about a way to modify the actual official driver for the 8105E to correct the problem of the 100Mbps mode, maybe it'll be easier to do than to make an full port since we have something wich is partially ok

 

I found the datasheet for this model maybe it could help. Everything is on my original post here :http://www.insanelymac.com/forum/topic/287568-realtek-rtl8105e-only-10mbps-any-chance-for-100mbps/

 

Or maybe i just need to send an mail to Realtek for the 100Mbps issue ? If anyone have their mail adress just pm me

 

Thanks again to Insanelymac public, you are all awesome

[refinery]

would anyone have any information on making this work on snow leopard? im running an asus p7p55d-e pro on 10.6.8, the 8111D's performance is awful on this board, i can never get above 50mb/sec (usually more around 30-35) while all my other devices can pull the same file off my server at close to maxing out the gigabit connection (the files in question are on a raid50, so its not a bottleneck on the server side)

using the driver as provided compiled does not see the NIC at all (as to be expected, it says 10.8 only)

i tried to build it using the xcode project in xcode 4.2 on my machine but it just throws a lot of errors, mostly about "use of undeclared identifier 'kChecksumTCPIPv6'" or 'kChecksumUDPIPv6'; did you mean 'kChecksumUDP'

im no coding expert so I dont really know what to do about these.

 

i'd very much love to get better networking performance out of my machine without having to resort to buying an additional NIC. whole point of building a hackintosh was having extra expansion slots over a mac pro. ive already gotten stung once with my original logic board choice (evga p55 ftw200) as there is no osx support at all and never will be for the marvell 88e8057 chipset.

any help would be appreciated to get this working. and before people suggest it, going to lion or mountain lion is *NOT* an option. i have multiple specific reasons for staying on snow leopard both technical and personal.

[Mieze]

i tried to build it using the xcode project in xcode 4.2 on my machine but it just throws a lot of errors, mostly about "use of undeclared identifier 'kChecksumTCPIPv6'" or 'kChecksumUDPIPv6'; did you mean 'kChecksumUDP'

im no coding expert so I dont really know what to do about these.

Try to update Xcode to 4.4.1 or newer and select "10.6" as "OS X Deployment Target". This should make the errors go away. At least on my machine there are no build errors with this configuration. If the driver isn't working as expected after a successful build please send me the debug messages from kernel.log, ifconfig -a and the netstat -s output. Please keep in mind that this driver is 64bit-only. In case your system is still running in 32bit-mode it won't work at all.

 

Good luck!

 

Mieze

[RehabMan]

Try to update Xcode to 4.4.1 or newer and select "10.6" as "OS X Deployment Target". This should make the errors go away. At least on my machine there are no build errors with this configuration. If the driver isn't working as expected after a successful build please send me the debug messages from kernel.log, ifconfig -a and the netstat -s output. Please keep in mind that this driver is 64bit-only. In case your system is still running in 32bit-mode it won't work at all.

 

Using SDK 10.8 and target 10.6 will make it build, but it will not load on Snow Leopard. This is because the definition of IONetworkController has changed in both Lion and ML (with additional virtual member functions). As a result, it is necessary to build with SDK 10.6 where these items are not defined.

 

I have forked this code in an effort to provide one (64-bit) binary for ML, Lion, and SL, as we are considering using this in the ProBook Installer. My fork is located at: https://github.com/R...Realtek-Network. The README.md there provides details on installation (there are special considerations on SL), where to download, and some background on what's changed (my change log is a bit behind, but you can see the commits).

 

A side effect of me pushing the archive (which is a git repo) to github is that it retains Mieze's commit log to her own local repo. For me, merging future changes is a little tricky, in that I have a separate local copy of your repo and when you make an update, I overwrite that repo with the new contents of the archive you post, then I'm able to treat it as a normal git remote and do a git fetch/merge to merge in those changes. It would be easier if your repo was available on a public site such as github, but for now this works...

 

Initially, I tried to make the kext work building with any SDK and still targeting all three systems (ML, Lion, SL) and was able to get it to work, but only with some ugly hacks. These hacks are disabled right now, and I'm just building with SDK 10.6 (Note: current Xcode doesn't come with SDK 10.6, so you have to copy it from an older Xcode that does, and copy it into the appropriate place (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs). If you want to see the ugly hacks that enable it to work on the three OS X versions with any SDK, search the project for DISABLE_ALL_HACKS.

 

To get around the undefined symbols with 10.6 SDK:

//HACK: from 10.7 SDK headers...
#ifndef __MAC_10_7
enum {
kChecksumTCPIPv6 = 0x0020,
kChecksumUDPIPv6 = 0x0040,
};
#define kIOMessageDeviceSignaledWakeup iokit_common_msg(0x350)
#endif

 

And to set those bits only when running on SL (note check for running kernel):

IOReturn RTL8111::getChecksumSupport(UInt32 *checksumMask, UInt32 checksumFamily, bool isOutput)
{
IOReturn result = kIOReturnUnsupported;

if ((checksumFamily == kChecksumFamilyTCPIP) && checksumMask) {
if (isOutput) {
*checksumMask = (kChecksumTCP | kChecksumUDP | kChecksumIP);
if (GetKernelVersion() >= MakeKernelVersion(11, 0, 0))
*checksumMask |= (kChecksumTCPIPv6 | kChecksumUDPIPv6);
}
else
*checksumMask = (kChecksumTCP | kChecksumUDP | kChecksumIP);

result = kIOReturnSuccess;
}
return result;
}

 

Provided the following is in the header:

#include <Availability.h>

#define MakeKernelVersion(maj,min,rev) (maj<<16|min<<8|rev)
#include <libkern/version.h>
#define GetKernelVersion() MakeKernelVersion(version_major,version_minor,version_revision)

 

BTW, we were previously using lnx2mac's driver. And while it works on the ProBook 4530s, it does not on the ProBook 4540s (next generation). And a nice side effect of using your driver on the ProBook 4530s is that it shaves ~7-8 seconds off of the boot time compared with using lnx2mac (my Chameleon to Login time is down from 15 sec to 8). Thanks for that. There is the problem posted here elsewhere about the driver not working after a warm boot from Windows, but small price to pay for a driver that 1) has source, 2) works on the 4540s, 3) trims boot time by 7 seconds.

[Mieze]

Thanks for figuring out how to build the driver under 10.6 but using an outdated SDK in order to build a driver for Lion and Mountain Lion is definitely not a reasonable solution with regard to future changes in particular as networks drivers have been evolving in Lion and ML. But with the information you provided it becomes clear what is needed to make a version that can be build with any of the mentioned SDKs without using ugly hacks. As far as I can see most differences are related to the introduction of checksum offload under IPv6 with Lion so that only a handful of methods need an alternate implementation for Snow Leopard.

 

I hope that I will find some time to add the missing code next week.

 

Mieze

[RehabMan]

Thanks for figuring out how to build the driver under 10.6 but using an outdated SDK in order to build a driver for Lion and Mountain Lion is definitely not a reasonable solution with regard to future changes in particular as networks drivers have been evolving in Lion and ML. But with the information you provided it becomes clear what is needed to make a version that can be build with any of the mentioned SDKs without using ugly hacks. As far as I can see most differences are related to the introduction of checksum offload under IPv6 with Lion so that only a handful of methods need an alternate implementation for Snow Leopard.

 

I hope that I will find some time to add the missing code next week.

 

Mieze

 

Good luck there. The problem stems from the games Apple is playing in the base classes IOEthernetController : IONetworkController

 

In the 10.7 and 10.8 IONetworkController they define additional virtual methods in IONetworkController (inherited into IOEthernetController and thus RTL8111):

virtual UInt32 getDebuggerLinkStatus(void);
virtual bool setDebuggerMode(bool active);

 

So, if you build against 10.8 headers (or 10.7), RTL8111's vtable imports them. Of course, they don't exist on SL, so the system cannot load the binary (there are no weak references in kernel/IOKit). Maybe you, being more experienced with IOKit drivers, know of some way around this fundamental problem.

 

And if you look at 10.8.3 sources, you will see even more funny business going on with the vtable of IONetworkController... I'm really not sure what Apple is up to and what the strategy for backward compatibilty (building w/ old headers running on new system)/forward compatibility (building w/ new headers running on old system). The whole system looks pretty fragile to me.

[Mieze]

So, if you build against 10.8 headers (or 10.7), RTL8111's vtable imports them. Of course, they don't exist on SL, so the system cannot load the binary (there are no weak references in kernel/IOKit). Maybe you, being more experienced with IOKit drivers, know of some way around this fundamental problem.

 

Of couse you'll have to use 10.6 SDK to build for Snow Leopard. There is no way to get around this requirement for a clean solution. Creating one binary for SL, L and ML won't work and is not the way it was designed by Apple. Even Apple's Lion drivers cause problems when used under ML.

 

And if you look at 10.8.3 sources, you will see even more funny business going on with the vtable of IONetworkController... I'm really not sure what Apple is up to and what the strategy for backward compatibilty (building w/ old headers running on new system)/forward compatibility (building w/ new headers running on old system). The whole system looks pretty fragile to me.

 

It's not fragile, they are improving the network subsystem making it fit for 10GE because without offloading most of the network related tasks to the NIC the next generation of Ethernet would be almost useless due to CPU load and speed limitations.

 

Mieze

[RehabMan]

Of couse you'll have to use 10.6 SDK to build for Snow Leopard. There is no way to get around this requirement for a clean solution. Creating one binary for SL, L and ML won't work and is not the way it was designed by Apple. Even Apple's Lion drivers cause problems when used under ML.

 

Working here. I've tested my kext on ML 10.8.3, Lion 10.7.5 and SL 10.6.8. I guess I'll roll with it until it is proven not to work.

 

It's not fragile, they are improving the network subsystem making it fit for 10GE because without offloading most of the network related tasks to the NIC the next generation of Ethernet would be almost useless due to CPU load and speed limitations.

 

I'm saying modifying the base classes the way they do and even using C++ as a driver model is fragile. Perhaps Apple has intended hardware vendors to release new drivers each time the OS is updated... but that, to me, is the definition of "fragile." The ability for the OS to evolve, still work with older drivers (not taking advantage of new OS features, of course), and providing a way for new drivers to take advantage of new features in the OS, should they be present, would be "not fragile."

[Mieze]

Working here. I've tested my kext on ML 10.8.3, Lion 10.7.5 and SL 10.6.8. I guess I'll roll with it until it is proven not to work.

 

Have you thoroughly tested IPv6? As its hardly used problems are not obvious but may cause serious problems including KPs in certain conditions.

 

I'm saying modifying the base classes the way they do and even using C++ as a driver model is fragile. Perhaps Apple has intended hardware vendors to release new drivers each time the OS is updated... but that, to me, is the definition of "fragile." The ability for the OS to evolve, still work with older drivers (not taking advantage of new OS features, of course), and providing a way for new drivers to take advantage of new features in the OS, should they be present, would be "not fragile."

Since SL OS X's internal architecture has dramatically changed. By the way, have you studied the linux code the driver is based on? OS X has a much cleaner driver architecture than linux ever had and the kernel interfaces are quite stable, compared to Linux, provided you use it the way it was meant to be. The only difference is that on the linux side no one is complaining about the drivers because a new kernel release always comes with a complete set of matching drivers.

 

Mieze

[RehabMan]

Have you thoroughly tested IPv6? As its hardly used problems are not obvious but may cause serious problems including KPs in certain conditions.

 

Thanks for the tip. My ability to test IPv6 is limited. I have an IPv6 enabled router, but my ISP is IPv4 only. So far I don't see any outward signs of IPv6 being supported (w/ my setup) in OS X, but I'll see if I can find some tools w/in OS X for IPv6 status.

 

Since SL OS X's internal architecture has dramatically changed. By the way, have you studied the linux code the driver is based on? OS X has a much cleaner driver architecture than linux ever had and the kernel interfaces are quite stable, compared to Linux, provided you use it the way it was meant to be. The only difference is that on the linux side no one is complaining about the drivers because a new kernel release always comes with a complete set of matching drivers.

 

I don't have any experience w/ Linux drivers. More experience in Windows... And yes, it is an amazing feat that Linux does to be able to rev. everything at once like that. Of course, last time I tried Linux on my desktop, the LAN drivers for Intel 82579 won't work...

[DOCa Cola]

27 days uptime with the 1.0 driver. So far no issues at all. Seems very stable. Great work